User Welfare Optimization in Recommender Systems with Competing Content Creators
Abstract
Driven by the new economic opportunities created by the creator economy, an increasing number of content creators rely on and compete for revenue generated from online content recommendation platforms. This burgeoning competition reshapes the dynamics of content distribution and profoundly impacts long-term user welfare on the platform. However, the absence of a comprehensive picture of global user preference distribution often traps the competition, especially the creators, in states that yield sub-optimal user welfare. To encourage creators to best serve a broad user population with relevant content, it becomes the platform’s responsibility to leverage its information advantage regarding user preference distribution to accurately signal creators. In this study, we perform system-side user welfare optimization under a competitive game setting among content creators. We propose an algorithmic solution for the platform, which dynamically computes a sequence of weights for each user based on their satisfaction of the recommended content. These weights are then utilized to design mechanisms that adjust the recommendation policy or the post-recommendation rewards, thereby influencing creators’ content production strategies. To validate the effectiveness of our proposed method, we report our findings from a series of experiments, including: 1. a proof-of-concept negative example illustrating how creators’ strategies converge towards sub-optimal states without platform intervention; 2. offline experiments employing our proposed intervention mechanisms on diverse datasets; and 3. results from a three-week online experiment conducted on a leading short-video recommendation platform.
1 Introduction
Online content recommendation platforms have evolved into an indispensable component of our daily lives (Bobadilla et al., 2013). These platforms play a pivotal role in assisting their users in navigating the vast ocean of content generated by revenue-seeking creators, including various social media platforms (e.g., Facebook, Instagram), streaming services (e.g., YouTube, TikTok), and many more. One of the primary functions of these recommendation platforms is to advance user welfare, defined as the overall volume and quality of interactions between users and content. This metric is widely regarded as a fundamental indicator of the well-being of an online ecosystem and is also closely tied to the platform’s revenue.
After decades of effort in relevance-driven matching between users and content, industry practitioners and researchers have reached the consensus that user welfare optimization cannot be achieved through myopic approaches that merely target at eliciting and predicting user preferences (Qian and Jain, 2022; Boutilier et al., 2023; Zhan et al., 2021; Mladenov et al., 2020; Yao et al., 2022a, b; Dean and Morgenstern, 2022; Biyik et al., 2023). One primary reason is because any matching strategy has a profound impact on content creators’ beliefs about the users’ demand and consequently their reactions, i.e., what to produce next, leading to a shift in the distribution of content available for recommendation. This influence pathway is unfortunately overlooked in existing recommendation algorithm design; and therefore, there is a great need for a robust recommendation strategy that operates with respect to creators’ strategic responses and the resultant content dynamics. It is imperative for the platform to encourage creators in generating content that continuously contributes to the overall health of the ecosystem.
Typically, creators’ well-being is intricately linked to the exposure of their content and the economic incentives they accrue from the platform, compelling them to continuously strive for maximized benefits (Glotfelter, 2019; Hodgson, 2021). This dynamic creates a competitive environment that leads to intriguing phenomena in terms of welfare guarantees at equilibrium (Fleder and Hosanagar, 2009; Jagadeesan et al., 2022; Zhu et al., 2023). For instance, Yao et al. (2023a) introduced a game theoretical framework to investigate competition dynamics among content creators. Their research revealed that social welfare loss can be attributed to factors such as the degree of exploration in users’ decision making and the span of recommendation slots. As indicated by many previous studies, the platform suffers from sub-optimal social welfare and thus undermines long-term revenue when content distribution lacks necessary diversity to cater to various users’ preferences. This issue is also observed in empirical studies, where content creators often exhibit a tendency to chase trends (Holmbom, 2015; Nandagiri and Philip, 2018). In essence, creators tend to produce content that arouses the interests of the majority user group, owing to the group’s high visibility and the creators’ myopic creation strategies (Yao et al., 2023a; Jagadeesan et al., 2022). However, it is our contention that the platform should not simply blame creators for their perceived selfishness and myopia. This is because creators do not possess a holistic view of the demand distribution, i.e., user preferences. Instead, it is the platform’s responsibility to disseminate knowledge about user demand to creators. By doing so, creators can make better informed decisions that mutually benefit their own interest and enhance user welfare (and hence platform’s revenue).
In this study, we extend the Content Creator Competition () framework introduced by Yao et al. (2023a, b), to model the dynamics of competition among content creators. We relax the behavioral assumptions about creators’ updating strategies in the original framework and explore how the platform can design mechanisms to optimize user welfare accordingly. Our key idea is to direct creators’ attention towards currently under-served users, by manipulating creators’ received utilities with respect to the cumulative user satisfaction about the recommended content. We present a series of approaches to implement the interventions with theoretical justifications.
To validate the effectiveness of our approach, we conducted offline experiments using both synthetic data and the MovieLens dataset, and demonstrated how our mechanism improves user welfare over time under a creator response simulator. Additionally, we deployed an online experiment on a leading short-video recommendation platform over a span of three weeks and observed statistically significant and positive result in terms of the overall user engagement and content diversity. Our model and online experiments offer valuable insights into the design of incentive-aware recommender platforms. To summarize, our contributions can be listed as follows:
-
1.
We formalize the user welfare optimization problem in a competitive content creation environment and identify the primary cause for potential sub-optimal outcomes: the information asymmetry between content creators and the platform.
-
2.
We propose a dynamic user importance reweighting approach with theoretical justifications for optimizing user welfare and three implementation schemes which can be applied to various practical scenarios.
-
3.
We demonstrate the effectiveness of our solution with both offline simulations and online testing on real traffic.
2 Related Work
The characterization and optimization of long-term dynamics on content platform involving strategic content creators has garnered increasing attention from both theoretical (Ben-Porat and Tennenholtz, 2017, 2018; Yao et al., 2023b, a; Zhu et al., 2023; Hu et al., 2023; Immorlica et al., 2024; Hron et al., 2022; Jagadeesan et al., 2022; Dean et al., 2024; Immorlica et al., 2024; Xu et al., 2024; Yao et al., 2024) and empirical (Mladenov et al., 2020; Prasad et al., 2023) fields. Seminal works from Ben-Porat and Tennenholtz (2017, 2018) introduced a game theoretical setting to model interactions between content creators and users, and proposed the Shapley mediator to ensure the existence of a pure Nash Equilibrium (Nash Jr, 1950).
Recently, Yao et al. (2023a) demonstrated that due to creators’ competition, the user welfare loss under a top- recommender systems can be upper-bounded by . This finding suggests that the platform can improve user welfare by providing more recommendations. Building on this, the authors further proposed a category of mechanisms for the platform to ensure a stable equilibrium and developed a computational solution to identify the optimal mechanism for social welfare optimization (Yao et al., 2023b). Additionally, Zhu et al. (2023) introduced an online learning method to jointly optimize recommendation policy and payment contracts for creators to maximize accumulated utility. Hu et al. (2023) designed a learning algorithm to incentivize the creation of high-quality content. However, all these studies rely on strong behavioral assumptions about content creators, e.g., they can perform no-regret learning (Yao et al., 2023a), or have oracle access to their utility functions (Yao et al., 2023b; Ben-Porat and Tennenholtz, 2017, 2018), so that the Nash equilibium is achievable. Our work bridges this gap by develo** a system-side solution to optimize user welfare that even when creators are not able to achieve Nash equilibria.
On the empirical side, Mladenov et al. (2020) explored a scenario where content creators may leave the platform if their user engagement falls below a threshold. The study optimized social welfare by solving a constrained matching problem. In a similar spirit, Prasad et al. (2023) introduced a sequential prompting policy aimed at optimizing user welfare in equilibrium. The optimal policy was determined through mixed integer programming. The solutions were reported to be effective under specific behavioral assumptions or environmental contexts, e.g., the platform can send prompts to creators as additional signals. However, the platforms are often constrained in their ability to influence the ecosystem. They may primarily rely on monetary incentives to motivate creators and have limited flexibility to manipulate factors beyond matching strategies and post-matching rewards. Our solution addresses this broader range of scenarios, making it applicable, for example, when creators are highly responsive to monetary incentives, and the platform’s influence is primarily exerted through adjustments to matching probabilities and post-matching rewards.
3 The Modeling of Content Creation Competition
In this section, we formulate the competition among content creators (i.e., players) as a strategic game, which will serve as an environment for the subsequent mechanism design problem. At a high level, each creator’s utility is determined by the platform’s matching strategy and the post-matching reward function. Creators adhere to simple, local update principles to sequentially alter their strategies, resulting in a dynamic content distribution on the platform. The primary objective of the platform is to optimize the cumulative user welfare by designing its matching strategy and post-matching reward function. Our strategic game setup builds upon and extends the framework of Content Creator Competition () game introduced by Yao et al. (2023a, b). For the sake of simplicity in nomenclature, we retain the name of and refer to our game as , i.e., an extension of the . Formally, a instance is defined by the following tuple: , which we explain in details below.
-
1.
Basic setups: a user distribution with finite support , and a set of content creators denoted by . Each creator can take an action , is often referred to as a pure strategy in game-theoretic literature, from an action set . can be understood as the embedding of content that creator will produce. Without loss of generality, we assume the norms of any and are upper bounded by 1.
-
2.
Relevance function: the relevance function measures the relevance score between a user and content . Without loss of generality, we normalize to , where suggests perfect matching. We focus on modeling the strategic behavior of creators and thus abstract away the estimation of 111We assume is learned from the offline data and is an unbiased estimation of user ’s satisfaction when exposed to . . For simplicity, we use to denote when the joint strategy profile and user profile are clear in the context of our discussion.
-
3.
Matching function: Given any user and when each creator commits to a strategy , the platform retrieves the top- ranked content in terms of the relevance scores and match one of them to . Specifically, let be a permutation of , we assume that the platform would pick using a softmax distribution with temperature 222The formulation in (Yao et al., 2023a) also assumes the platform retrieve top- content for each user, but let the user to choose one according to the Random Utility model. The resulting matching probability shares the same form as in Eq. (1), but differs in the sense that the in our setting is a parameter controlled by the platform while it is the user decision noise in (Yao et al., 2023a)., i.e.,
(1) A small makes the matching strategy more deterministic, and corresponds to random matching.
-
4.
User utility and welfare: When user is matched with , the user’s perceived utility is given by a function . The user welfare is thus defined as the total expected utility resulted from the matching,
(2) To simplify the technical discussions, we assume the learned relevance function is an unbiased estimation of , and therefore can be simplified to
(3) However, our proposed solution works for general welfare function defined in Eq (2).
-
5.
Creator utility: For creator , her utility is given by
(4) where is the system-provided reward for this matching.
Natural choices of include being proportional to the user’s perceived utility, or simply setting (i.e., reward creators by the amount of traffic). Therefore, we have
(5) (6) Throughout the paper we adopt Eq (5) as the platform’s default choice, as it is demonstrated in (Yao et al., 2023a) that rewarding creators by user utility enjoys a better welfare guarantee than rewarding them by traffic.
The most well established concept for characterizing a game’s outcome is pure Nash equilibrium (PNE) (Nash Jr, 1950). At a PNE, any possible deviation from a player’s current strategy would not increase her utility conditioned on other players’ strategies. Under some mild assumptions, we can prove that the PNE of our game exists and is unique as stated in the following theorem.
Theorem 1
Any game with has a unique pure Nash equilibrium (PNE) under the utility function (6) if is sufficiently smooth and concave and each creator has a convex strategy set.
Theorem 1 guarantees the existence of a unique PNE and thus theoretically allows the platform to establish a stable outcome. However, in practical scenarios, we find it uninteresting to either generalize this result or delve further into its properties for two reasons. First, it is rare for to hold in practice because no system will present the entire collection of content to each user. When , the existence of a PNE becomes challenging to establish, due to the discontinuity of the utility functions caused by the top- ranking operator during the matching process. Second, even when a PNE does exist, it does not suggest that creators can consistently reach it through sequential updates. Furthermore, the existence of a PNE does not necessarily imply it is easily achievable in practice, nor does it suggest an improved user welfare. In fact, as we will demonstrate in Section 4.1, even in a simple environment with a unique PNE, a natural updating dynamics among creators fails to converge to the PNE and results in sub-optimal user welfare.
Therefore, we focus on a more practical solution concept called Local Nash equilibria (LNE). While a PNE requires that all players do not want to deviate to any other strategy in the entire space, an LNE merely stipulates players are satisfied with their strategies in a local region. Its formal definition is given as follows.
Definition 1
A profile of creator strategies forms a local Nash equilibrium (LNE), if for every creator , there exists an open set such that is a best response strategy within ; formally,
(7) |
We argue that LNE offers a more intuitive and practical solution concept for consideration due to two observations. First, the strategic evolution of content creation is often deeply intertwined with creators’ historical decisions (Kajander, 2019). This correlation stems from content generation being anchored in domain-specific expertise and accumulated experiences, which are inherently stable attributes. As a result, the produced content usually demonstrates path dependency, posing significant challenges for creators in implementing drastic modifications. Second, creators are typically constrained by a lack of comprehensive insights into their utility functions due to a limited understanding of the user demographic and the distribution of user preferences. Given these constraints, creators are likely to resort to incremental adjustments for strategy update.
Hence, we focus on the setting where creators engage in a repeated play of and employ a local searching rule termed local better response (LBR) update for improving their strategies. The details of LBR is presented in Algorithm 2 in Appendix A. LBR characterizes two fundamental properties of content creation: 1. it relies solely on point estimations of the utility function; and 2. it only incurs local changes at each update. At each step, a creator who decides to update her strategy would first generate an exploration direction and then she would evaluate whether adjusting her strategy in this direction results in a higher utility. If so, she proceeds to update her strategy along in a pace of ; otherwise, she maintains her current strategy. This procedure closely emulates real-world scenarios where creators strive to optimize their utilities while having merely black-box access to the utility functions. In practice, finding a clear direction that guarantees improved utility can be a challenging and, at times, unrealistic task. Consequently, we model their strategy evolution as an iterative process of trial and error. By definition, when LBR converges in , it must converge to an LNE. Our primary interest lies in understanding how the platform can devise a dynamic rewarding or matching principle that maximizes cumulative user welfare within a given time period.
4 Intervention Mechanism Design
In this section, we introduce the new intervention mechanism designed to optimize user welfare. These mechanisms are intended for the platform to influence creators’ perceived utilities, thereby guiding the evolution of their strategies toward more desirable outcomes. We will first establish the need for platform-driven mechanism design by illustrating how suboptimal results can arise in a simplified example without any intervention. Subsequently, we will delve into the specifics of our proposed methods.
4.1 The Necessity of Intervention
We start with a simple illustrative example to show how the competition among creators could result in quite inferior user welfare in when creators employ local update dynamics specified in Algorithm 2. This example exhibits a stark contrast to the sound welfare guarantee for no-regret learning (Belmega et al., 2018) equipped creators in (Yao et al., 2023a). Consider a instance described below. The user population is evenly distributed over the finite set and there are content creators, each with action set . The reward function is defined as and . It is evident that the user welfare defined in Eq (3) is maximized when each creator precisely targets a single user, i.e., , which also represents the PNE of this game. However, as we will illustrate through simulations, creators’ strategies do not converge to the PNE nor optimize the user welfare under the LBR dynamics when the platform does not intervene.
First, let’s examine what happens when the platform takes no action to guide the creators. The left panel of Figure 1 visualizes the trajectories of strategy evolution in our constructed environment. Initially, creators’ strategies are randomly distributed in the region between and . Over time, and are exclusively occupied by one creator each, while has two creators competing for it. The remaining creator chooses not to target either or and hovers around the region between and , leaving both and unsatisfied.
From the observed strategy evolution paths, we can deduce how this sub-optimal situation arises. Initially, creators move in different directions: two creators quickly converge to and , while the remaining three compete for the attention of the central user . However, after this point, no creator has a strong incentive to move closer to or , as the marginal utility gained from getting closer to or does not compensate for the loss incurred by moving away from . Consequently, two creators decide to remain around and one creator settles in a region between and .
The above observations highlight the pivotal role played by the central user in the occurrence of sub-optimal results. Since is close to other users in the embedding space, targeting becomes a popular and safe choice for creators. It secures a fraction of attention from without completely sacrificing the utility gained from other user groups. Thus, users like act as “popular states” when creators dynamically adjust their strategies. Whenever a creator is located near , they are likely to be trapped and reluctant to explore potentially better strategies. Consequently, such “popular” users end up attracting more creators, leaving other users unattended.
One immediate solution for the platform is to identify and reduce the impact of these “popular” users. For instance, the platform can halve the utility gained from the central user for each creator. This simple mechanism works effectively in this example, as illustrated in the right panel of Figure 1. Initially, there are still three creators converging to . However, due to the reduced reward from , two creators find it less profitable to stay, driving them to deviate towards and . By assigning different importance weight for each user, the platform can reshape each creator utility landscape and therefore influence their local search based dynamical behaviors.
4.2 Platform’s Intervention Mechanisms
The observations above motivate our design of intervention mechanisms that can be employed by the platform to influence creators’ perceived utilities. These mechanisms lay the foundation for the adaptive optimization methods we will delve into later. As a reminder, as defined in Eq (4), a creator’s expected utility from a specific user is influenced by two key factors: the probability of creator being matched with user denoted as , and the post-matching reward assigned by the platform, denoted as . The default choice of the platform is to set the reward function as in Eq (5) and the matching probability function as the softmax over the top- ranked content as demonstrated in Eq (1).
In the example provided in Section 4.1, the primary factors leading to sub-optimal welfare is the presence of popular user groups that attract excessive creator attention, making minority user groups unnoticed by creators. To enhance overall user welfare, it is crucial for the platform to guide creators’ attention toward these overlooked user groups by re-emphasizing their significance. In this way, creators who were previously unaware of these user groups or found them less lucrative may consider adjusting their strategies to align more closely with those users’ preferences. To achieve this objective, we introduce and study three different approaches for modifying the schemes of and , namely User Importance Reweighting (UIR), Soft Matching Truncation (SMT), and Hard Matching Truncation (HMT). These three mechanisms share a common underlying principle, but they are designed to operate under different scenarios, taking into account potential constraints faced by a platform.
User Importance Reweighting (UIR)
The most straightforward approach is UIR,
(8) |
where the platform simply adjusts the post-matching rewards for creators based on the measured importance of each user. Specifically, if the platform believes a user has been under-served under the current content distribution, it raises the reward for creators whose content is consumed by such a user. Intuitively, this sends a message to creators that “if you shift your content towards such users, you will get a higher marginal reward compared to sticking to your current content.” As a result, the platform can carefully design the user weights such that a reasonable number of creators can be successfully incentivized to serve the targeted users.
Soft Matching Truncation (SMT) and Hard Matching Truncation (HMT)
Both SMT and HMT function in a similar manner as UIR but focus on manipulating the matching probability rather than the post-matching reward by utilizing the weight . Recall that the probabilistic matching function is characterized by two parameters: the truncation number (which, in practice, corresponds to the total number of recommendation candidates retrieved for ranking) and the temperature (which can be viewed as a measure of the exploration strength in the ranking model). When the platform needs to signal the importance of a specific user , it enhances ’s visibility among creators, increasing the chance that creators who were previously unaware of start realizing the potential benefits of catering to . This can be achieved by either increasing or : increasing flattens the distribution of ’s matches among the top- candidates, while increasing enlarges the pool of creators exposed to . Therefore, both of them augment the expected number of creators exposed to . Since imposes a rigid threshold on the number of creators exposed to , while offers a more flexible threshold, we refer to them as Hard Matching Truncation (HMT) and Soft Matching Truncation (SMT), respectively:
(9) | |||
(10) |
We remark that UIR is more suitable when the platform possesses the flexibility to design payment incentives for creators. However, if the platform has limited control over payment, such as budget constraints or other factors, SMT or HMT can be employed, as they only require minor adjustments to the matching function. The specific choices of increasing functions are flexible and we leave it to the experiments.
4.3 Welfare Optimization through Adaptive Reweighing
To implement our proposed intervention mechanisms, we need to compute the corresponding user-specific weighting functions, namely , , and . In this section we will use UIR as an example to illustrate our method and let the user distribution be a uniform distribution over its support so that can be parameterized by a vector . When the platform commits to an intervention mechanism , the content creators’ strategic updates according to LBR (i.e., algorithm 2) will lead their joint strategy to an LNE , which determines the content distribution and the total user welfare . Therefore, the task of finding the optimal maximizing under can be formulated as the following bi-level optimization problem:
(11) | ||||
s.t., | (12) |
We adopt the formulation in Eq (11) simply for presentation purpose, as the constraint in Eq (12) is not well-defined due to the non-uniqueness of LNE of in general. When we takcle problem in Eq (11), we employ either LBR for simulating an in offline experiments, or we directly observe based on the creators’ actual responses over a period of time for online experiments. An straightforward approach to solve Eq (11) is to use an iterative method to dynamically adjust , and the main challenge is to pin down an improving direction of . Ideally, we can apply first-order optimization if an estimation of the gradient is available. However, the interplay between and is generally intractable to analyze and we have to resort to heuristic methods. To get an intuitive idea about an improving direction of , we consider a stylized setting where the user population is perfectly separated and the relevance function is given by dot-product . In such a structured environment, the following theorem reveals a useful principle for finding an improving direction of .
Theorem 2
When the number of creators is large enough and the user population is a uniform distribution over an orthogonal basis in , updating with the following formula guarantees an improvement in defined in Eq (3):
(13) |
where is a small scalar denoting the learning rate, and is the expected utility of user at .
By the definition in Eq (4), rescaling each by a constant does not alter the nature of problem in Eq (11). Therefore, the insight conveyed by Eq (13) is clear: if a user enjoys a high expected utility under the current content distribution, the platform should reduce her weight when rewarding creators. Conversely, if a user’s expected utility is relatively low, the platform needs to highlight her significance for motivating a larger set of creators to develop content that caters to the needs of this user. Despite the fact that Eq (13) is derived from a significantly simplified user distribution, we will leverage it as a foundational element in the development of our adaptive reweighing algorithm and demonstrate in our experiments that this simple heuristic works pretty well for real user distributions.
Next, we formally introduce our proposed adaptive reweighting algorithm for optimizing the intervention mechanism . Each user is initially assigned a unit weight . During subsequent iterations, the platform continuously monitors the average utility of user , denoted as , within a specified time window, and updates according to the following (14), where is a tunable parameter. This adjustment process employs the meta-algorithm structure of multiplicative weight update method (Arora et al., 2012).
(14) |
In practice, we can choose the user utility function as the metric used for defining the user welfare function Eq (2). Up to this point, our discussion has primarily focused on the assumption that . However, it is important to highlight that in Eq (14) can also take alternative forms to optimize empirical performance. For instance, it can be a function of any numerical measurement related to user satisfaction (e.g., click-through rate). To reduce the dimension of the user weight vector and enhance the robustness of weight updates, we recommend that algorithm designers pre-cluster users into groups based on their static features so that users within the same group maintain identical weights. The platform’s intervention strategy is thus parameterized by an -dimensional vector, , with each entry denoting the weight assigned to the corresponding user group.
For a fixed time horizon in which the platform plans to perform intervention, the platform divides the horizon into epochs, each with an equal length of (i.e., ). At the start of each epoch , the platform commits to a weight vector and deploy it to one of the intervention mechanisms UIR, SMT or HMT. After that, the platform observes and records the sequence of creators’ strategic responses, denoted as from the online environment. Subsequently, the algorithm estimates the average user welfare for each group . It then employs values in to update the weights at the beginning of the -th epoch using Eq (14). To prevent from growing or declining excessively, after each update we first normalize and then clip its values within a predetermined interval . The formal description of this process is presented in Algorithm 1. The implementation details for the deployment of UIR, SMT and HMT in Line 4 are deferred to Appendix B.
5 Experiments
In this section, we evaluate our proposed intervention mechanisms on both offline datasets and an online environment on a leading short-video recommendation platform in the industry.
5.1 Experiments on Offline Data
We conduct simulations on game instances constructed from synthetic data and MovieLens-1m dataset (Harper and Konstan, 2015). In the following, we first introduce the specification of these two simulation environments and then report the results.
5.1.1 Synthetic environment
For the synthetic environment, we first construct the user population as follows: we fix an embedding dimension and independently sample cluster centers, denoted as , from the unit sphere . For each center , we generate users belonging to cluster- by independently sampling from a Gaussian distribution . The sizes of the user clusters are denoted by a vector . In this manner, we generate a population of size . The number of creators is set to , and each action set is set to the unit ball in . The user utility and relevance score function are set to . We set to by default. Such synthetic datasets characterize a class of clustered user preference distributions (e.g., majority vs., minority user groups).
On the creators’ side, we let their initial strategies to be close the center of the largest user group. This environment models a situation where creators tend to chase popular trends by exclusively producing content tailored to the taste of the largest user group. We aim to investigate whether our proposed mechanisms can assist the platform to escape from such sub-optimal states.
5.1.2 Environment constructed from MovieLens-1m
We use deep matrix factorization (Fan and Cheng, 2018) to train user and movie embeddings (with dimension set to ) by fitting the observed ratings in the range of 1 to 5. To ensure the quality of the trained embeddings, we performed a 5-fold cross-validation and obtained an averaged RMSE on the test sets. Then with the same hyper-parameter settings, we train the user/item embeddings with the complete dataset.
We select active users with more than 200 ratings, resulting in a population comprising users. We set the number of creators to , with each creator’s action set consisting of different movies. All share a common part – the most popular 700 movies based on the number of ratings they received, and each also has a private part – a randomly sampled 300 movies. Our choice of the user utility and matching score functions is , and then normalized to the region Additionally, we set and initialize creators’ strategies to the most preferred movie among all users (i.e., the movie that enjoys the highest average rating among ).
5.1.3 Configurations of adaptive reweighting algorithm and intervention mechanisms
For the adaptive reweighting algorithm, we set the epoch length and the simulation time horizon for both environments. During each time step within an epoch, we simulate creators’ responses by letting each of them update her strategy once using Algorithm 2 in a random order. Creators’ learning rate is set to . On the platform side, we use -means clustering to determine user groups and set the number of clusters to for synthetic environment and for MovieLens environment, respectively. We should note as in practice, even the system does not have the exact knowledge about user distribution, we do not use the ground-truth clustering of users set in the simulation. In addition, we set the temperature parameter for the first half of the time period and reduce it to for the remaining period. The clip** constants are set to and the map** used in SMT and HMT are set to and .
5.1.4 Results
Figure 2(a) illustrates the user welfare resulted from creators’ evolving strategies under the three intervention mechanisms: UIR, SMT, and HMT, compared to the baseline (no platform intervention). Over time, all three mechanisms consistently outperform the baseline. In the baseline (shown in blue), the welfare plateaus quickly and remains stagnant. Conversely, the welfare curves under the other mechanisms exhibit “double-ascent” patterns. Initially, they also plateau, but eventually, they begin to rise again and surpass the baseline. This is because, without platform’s intervention, creators tend to remain in sub-optimal equilibria as illustrated in Section 4.1. However, our proposed mechanisms gradually accumulate user group weights, which, when significant enough, encourage creators to explore unattended user groups, leading to increased welfare. Among the three mechanisms, HMT demonstrates the most substantial gain with the least variance. UIR, while showing a lower marginal gain, maintains stability with minimal variance. SMT, which achieves a moderate gain, exhibits higher variance, suggesting that directly manipulating the matching temperature may be overly aggressive.
Figure 2(b) shows the learned group weights at the last iteration of simulation. As it demonstrates, all three mechanisms emphasize on small groups over larger ones. This outcome aligns with our expectation: on one hand, larger user groups are more likely to “trap” unnecessarily many creators and thus should be deprioritized; on the other hand, increasing weights of niche user groups also improve their chances of being discovered by more creators.
Figure 2(c) breaks down the average utilities across user groups. The blue dashed line (i.e., the no-intervention baseline) exhibits a positive correlation between averaged group utility and group size, mirroring real-world observations. The orange bars show that UIR strikes a balance by improving the utility of niche groups while slightly trading off utility in larger groups. HMT achieves a remarkable Pareto improvement across all groups, as indicated by the red bars. However, SMT’s gains come at the cost of even greater skewness in the average utility distribution across groups.
To summarize, all three mechanisms show promising improvements in overall user welfare, but their nature of gains differs, introducing considerations for the platform. When condition allows, HMT is the top choice due to its strong performance, stability, and fairness. For platforms that prioritize fairness and stability, UIR is also a viable option. However, SMT, despite improving overall welfare, may suffer from potential drawbacks such as instability and fairness issues. In-depth analysis of the merits and limitations of these mechanisms remains a topic for future research.
The results in the MovieLens environment align with the insights from the synthetic environment (refer to Figure 3). However, it’s worth noting that the trends in learned group weights and realized group utilities do not always align with group size, which is expected in real-world data where unattended user groups may not necessarily have small sizes. Nevertheless, our proposed mechanisms continue to improve overall welfare by identifying and prioritizing these groups.
5.2 Online Experiments
We conducted online evaluations on one of the world’s leading short-video content creation and recommendation platforms (referred to as the “platform” hereafter due to the anonymity requirement), spanning over 3 weeks. We observe that the platform’s intervention can indeed influence creator behavior because, on average, there is a positive correlation between the delivery volume and content creation volume for each topic. (The Pearson correlation is 0.2, and there are hundreds of topics in total). In this experiment, we employed the “like-through-rate” (LTR) as the user utility function. LTR is calculated as the ratio of total likes to the number of impressions of a specific short video. We opted for LTR as the chosen metric because it not only serves as a reflection of user satisfaction but also offers a straightforward and easily interpretable signal for content creators to assess their content’s perceived quality. The selection of the HMT mechanism for testing was deliberate, driven not only by its strong performance against the baseline and other mechanisms in our offline experiments, but also due to its ease of integration into production: HMT solely requires changing the number of candidate content retrieved for different users within the deployed relevance-based ranking model.
5.2.1 Experiment Setups
We list the experiment setups below.
User clustering: We utilized explicit user characteristics such as demographics including country and gender and their level of activeness including video consumption volume and watch time. This approach led to the creation of over 10,000 user groups and we retained groups that had a sufficient number of users, resulting in hundreds of user groups.
Cluster weight update: We implemented a daily weight updating cadence. Each day, we assessed the satisfaction of every user group by calculating the relative change of LTR over its average in the previous two days. Subsequently, we recalculated the user weights in accordance with the method outlined in Algorithm 1.
A/B test configurations: To evaluate changes in both user and creator behavior, we employed a symmetric A/B test setup on the platform. This symmetric A/B test consisted of an experiment arm and a control arm to measure performance. At the beginning, we randomly pair 3% creators with 3% users from entire platform for each arm. Under this setup, users within each arm exclusively received content created by creators within the same arm, and content created by these creators was exclusively exposed to users within the same arm throughout the testing period. This stringent separation prevents any cross-group treatment leakage and maintains a closed feedback loop within each arm. In our online experiment, we ran these two arms for a duration of 3 weeks: a control arm adhering to the existing production setup and a test arm where we applied our proposed mechanism, HMT.
HMT specifics: We implemented HMT during the cold start content retrieval phase, which pertains to content created within a few days and has not yet garnered a predefined number of impressions. Specifically, within the platform’s production pipeline, we integrated an audience matching stage to retrieve cold start content. During this stage, content is exclusively delivered to the most suitable user candidates based on relevance scores generated by a pre-trained model. In the existing production setup, a fixed relevance score percentile of 99% is uniformly applied to all users. This means that every user is only matched with the top 1% of cold-start content in terms of relevance scores to ensure a high level of personalization. When tuning the percentile, we typically observe a trade-off between overall user satisfaction and the volume of cold-start content. In our experiment, we leveraged HMT to intelligently adjust this threshold for different user groups, anticipating improvements in both of thees metrics. Consequently, user groups with higher weights were granted a higher chance to be selected by content creators, while those with lower weights were deprioritized. The map** from the group weight to the percentile of retrieved cold start content proportion was designed as a piece-wise constant function, with details specified in Table 1.
Weight | |||||||
---|---|---|---|---|---|---|---|
Percentile | 0.99 | 0.95 | 0.90 | 0.85 | 0.75 | 0.7 | 0.1 |
5.2.2 Results
Positive results were obtained in three key aspects.
User-side engagement: The core utility metric LTR increased by and the total impression number of cold-start content increased by , leading to a increase in impressions for fresh content created within 2 hours. These improvements are statistically significant and demonstrate increased user welfare while enhancing the freshness and diversity of content. The gains in both user satisfaction and the volume of cold-start content indicate that HMT influenced many creators to produce more targeted content that benefits niche user groups. Table 2 provides a breakdown of performance improvement per user group. We indexed all groups in descending order by their sizes and divided them into four columns, with each column constituting approximately 25% of the total traffic during the experiment period. As shown, smaller groups enjoyed a higher gain in terms of LTR, which echoes the observations in offline results. The gain in cold-start content impression volume shows an opposite trend. This is because the absolute number of cold-start impressions for larger user groups was smaller as the distribution of relevance scores in this group was more skewed, resulting in a larger relative gain in this metric.
Content diversity: The average number of consumed topics per user during the experimental period increased by 0.71%, and this increase is also statistically significant.
Creator-side engagement: For popular creators (those with more than 1000 followers), the number of daily active users (Creator DAU) increased by an average of 0.17%, while for the remaining creators, the gain is 0.06%. Additionally, there is a promising increasing trend in Creator DAU for popular creators over the three weeks of the experiment: the increases over the first, second, and third weeks are -0.2%, 0.24%, and 0.48%. This suggests that the three-week duration of the experiment may have been too short to influence the majority of creators to respond accordingly, and more time may be needed to fully observe the positive feedback from creators.
User Groups | 1-5 | 6-20 | 21-74 | 75+ | TOTAL |
---|---|---|---|---|---|
LTR | +0.43% | +1.40% | +0.75% | +1.36% | |
Impression | +2.64% | +0.62% | +1.42% | +0.11% |
6 Conclusion
In this study, we tackle the user welfare optimization challenge faced by online content recommendation platforms through the lens of mechanism design. We identified myopic strategy updates among creators caused by their limited information access as the culprit of sub-optimal welfare and introduced platform interventions to address this issue. Our three proposed mechanisms, based on adaptive user importance reweighting, enable platforms to convey global user preference information, reshape creators’ perceived utilities, and influence their behaviors. Empirical experiments in both offline and online environments demonstrated the effectiveness of our approach, highlighting its potential for practical impact.
For future work, there remains an intriguing need for a comprehensive understanding of the merits and limitations of UIR, SMT, and HMT to aid practitioners in selecting the most suitable mechanism for real-world applications. It is also important to address practical constraints when applying the developed mechanisms. For instance, can we find ways to jointly optimize user welfare and platform costs? Can the mechanism explicitly ensure fairness on the user side and producer side? Deeper insights into these questions hold the potential to greatly impact the rapidly evolving online content landscape and industry practices.
References
- Arora et al. (2012) Sanjeev Arora, Elad Hazan, and Satyen Kale. The multiplicative weights update method: a meta-algorithm and applications. Theory of computing, 8(1):121–164, 2012.
- Belmega et al. (2018) E Veronica Belmega, Panayotis Mertikopoulos, Romain Negrel, and Luca Sanguinetti. Online convex optimization and no-regret learning: Algorithms, guarantees and applications. arXiv preprint arXiv:1804.04529, 2018.
- Ben-Porat and Tennenholtz (2017) Omer Ben-Porat and Moshe Tennenholtz. Shapley facility location games. In International Conference on Web and Internet Economics, pages 58–73. Springer, 2017.
- Ben-Porat and Tennenholtz (2018) Omer Ben-Porat and Moshe Tennenholtz. A game-theoretic approach to recommendation systems with strategic content providers. Advances in Neural Information Processing Systems, 31, 2018.
- Biyik et al. (2023) Erdem Biyik, Fan Yao, Yinlam Chow, Alex Haig, Chih-wei Hsu, Mohammad Ghavamzadeh, and Craig Boutilier. Preference elicitation with soft attributes in interactive recommendation. arXiv preprint arXiv:2311.02085, 2023.
- Bobadilla et al. (2013) Jesús Bobadilla, Fernando Ortega, Antonio Hernando, and Abraham Gutiérrez. Recommender systems survey. Knowledge-based systems, 46:109–132, 2013.
- Boutilier et al. (2023) Craig Boutilier, Martin Mladenov, and Guy Tennenholtz. Modeling recommender ecosystems: Research challenges at the intersection of mechanism design, reinforcement learning and generative models. arXiv preprint arXiv:2309.06375, 2023.
- Bravo et al. (2018) Mario Bravo, David Leslie, and Panayotis Mertikopoulos. Bandit learning in concave n-person games. Advances in Neural Information Processing Systems, 31, 2018.
- Dean and Morgenstern (2022) Sarah Dean and Jamie Morgenstern. Preference dynamics under personalized recommendations. In Proceedings of the 23rd ACM Conference on Economics and Computation, pages 795–816, 2022.
- Dean et al. (2024) Sarah Dean, Evan Dong, Meena Jagadeesan, and Liu Leqi. Recommender systems as dynamical systems: Interactions with viewers and creators. In Workshop on Recommendation Ecosystems: Modeling, Optimization and Incentive Design, 2024.
- Fan and Cheng (2018) Jicong Fan and Jieyu Cheng. Matrix completion by deep matrix factorization. Neural Networks, 98:34–41, 2018.
- Fleder and Hosanagar (2009) Daniel Fleder and Kartik Hosanagar. Blockbuster culture’s next rise or fall: The impact of recommender systems on sales diversity. Management science, 55(5):697–712, 2009.
- Glotfelter (2019) Angela Glotfelter. Algorithmic circulation: how content creators navigate the effects of algorithms on their work. Computers and composition, 54:102521, 2019.
- Harper and Konstan (2015) F Maxwell Harper and Joseph A Konstan. The movielens datasets: History and context. Acm transactions on interactive intelligent systems (tiis), 5(4):1–19, 2015.
- Hodgson (2021) Thomas Hodgson. Spotify and the democratisation of music. Popular Music, 40(1):1–17, 2021.
- Holmbom (2015) Mattias Holmbom. The youtuber: A qualitative study of popular content creators, 2015.
- Hron et al. (2022) Jiri Hron, Karl Krauth, Michael I Jordan, Niki Kilbertus, and Sarah Dean. Modeling content creator incentives on algorithm-curated platforms. arXiv preprint arXiv:2206.13102, 2022.
- Hu et al. (2023) Xinyan Hu, Meena Jagadeesan, Michael I Jordan, and Jacob Steinhard. Incentivizing high-quality content in online recommender systems. arXiv preprint arXiv:2306.07479, 2023.
- Immorlica et al. (2024) Nicole Immorlica, Meena Jagadeesan, and Brendan Lucier. Clickbait vs. quality: How engagement-based optimization shapes the content landscape in online platforms. In Proceedings of the ACM Web Conference 2024, 2024.
- Jagadeesan et al. (2022) Meena Jagadeesan, Nikhil Garg, and Jacob Steinhardt. Supply-side equilibria in recommender systems. arXiv preprint arXiv:2206.13489, 2022.
- Kajander (2019) Hanna Kajander. Challenges of a content creator in the era of digital marketing. 2019.
- Krantz and Parks (2002) Steven George Krantz and Harold R Parks. The implicit function theorem: history, theory, and applications. Springer Science & Business Media, 2002.
- Mladenov et al. (2020) Martin Mladenov, Elliot Creager, Omer Ben-Porat, Kevin Swersky, Richard Zemel, and Craig Boutilier. Optimizing long-term social welfare in recommender systems: A constrained matching approach. In International Conference on Machine Learning, pages 6987–6998. PMLR, 2020.
- Nandagiri and Philip (2018) Vaibhavi Nandagiri and Leena Philip. Impact of influencers from instagram and youtube on their followers. International Journal of Multidisciplinary Research and Modern Education, 4(1):61–65, 2018.
- Nash Jr (1950) John F Nash Jr. Equilibrium points in n-person games. Proceedings of the national academy of sciences, 36(1):48–49, 1950.
- Prasad et al. (2023) Siddharth Prasad, Martin Mladenov, and Craig Boutilier. Content prompting: Modeling content provider dynamics to improve user welfare in recommender ecosystems. arXiv preprint arXiv:2309.00940, 2023.
- Qian and Jain (2022) Kun Qian and Sanjay Jain. Digital content creation: An analysis of the impact of recommendation systems. Available at SSRN 4311562, 2022.
- Rosen (1965) J Ben Rosen. Existence and uniqueness of equilibrium points for concave n-person games. Econometrica: Journal of the Econometric Society, pages 520–534, 1965.
- Xu et al. (2024) Renzhe Xu, Haotian Wang, Xingxuan Zhang, Bo Li, and Peng Cui. Ppa-game: Characterizing and learning competitive dynamics among online content creators. arXiv preprint arXiv:2403.15524, 2024.
- Yao et al. (2022a) Fan Yao, Chuanhao Li, Denis Nekipelov, Hongning Wang, and Haifeng Xu. Learning from a learning user for optimal recommendations. In International Conference on Machine Learning, pages 25382–25406. PMLR, 2022a.
- Yao et al. (2022b) Fan Yao, Chuanhao Li, Denis Nekipelov, Hongning Wang, and Haifeng Xu. Learning the optimal recommendation from explorative users. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 9457–9465, 2022b.
- Yao et al. (2023a) Fan Yao, Chuanhao Li, Denis Nekipelov, Hongning Wang, and Haifeng Xu. How bad is top- recommendation under competing content creators? In International Conference on Machine Learning. PMLR, 2023a.
- Yao et al. (2023b) Fan Yao, Chuanhao Li, Karthik Abinav Sankararaman, Yiming Liao, Yan Zhu, Qifan Wang, Hongning Wang, and Haifeng Xu. Rethinking incentives in recommender systems: Are monotone rewards always beneficial? arXiv preprint arXiv:2306.07893, 2023b.
- Yao et al. (2024) Fan Yao, Chuanhao Li, Denis Nekipelov, Hongning Wang, and Haifeng Xu. Human vs. generative ai in content creation competition: Symbiosis or conflict? arXiv preprint arXiv:2402.15467, 2024.
- Zhan et al. (2021) Ruohan Zhan, Konstantina Christakopoulou, Ya Le, Jayden Ooi, Martin Mladenov, Alex Beutel, Craig Boutilier, Ed Chi, and Minmin Chen. Towards content provider aware recommender systems: A simulation study on the interplay between user and provider utilities. In Proceedings of the Web Conference 2021, pages 3872–3883, 2021.
- Zhu et al. (2023) Banghua Zhu, Sai Praneeth Karimireddy, Jiantao Jiao, and Michael I Jordan. Online learning in a creator economy. arXiv preprint arXiv:2305.11381, 2023.
A Details of content creators’ strategy update dynamics
The Local Better Response (LBR) procedure described in Algorithm 2 captures the evolution of creators’ strategies in a snapshot, and characterizes two fundamental properties of content creation: 1. it relies solely on point estimations of the utility function (Line 3); and 2. it only incurs local changes at each update (Line 4). At each step, a creator who decides to update her strategy would first generate an exploration direction (Line 2); then she would evaluate whether adjusting her strategy in this direction results in a higher utility. If so, she proceeds to update her strategy along in a pace of ; otherwise, she maintains her current strategy.
Algorithm 2 closely emulates real-world scenarios where creators strive to optimize their utilities while having merely black-box access to the utility functions. In practice, finding a clear direction that guarantees improved utility can be a challenging and, at times, unrealistic task. Consequently, we model their strategy evolution as an iterative process of trial and error. By definition, when LBR converges in , it must converge to an LNE. Our primary interest lies in understanding how the platform can devise a dynamic rewarding or matching principle that maximizes cumulative user welfare within a given time period.
B Implementation Details of UIR, SMT and HMT Mechanisms
The following sub-routine, denoted as Algorithm 3, outlines how the platform deploys the weights obtained from Line 8, Algorithm 1 as an intervention mechanism in Line 4. In Algorithm 3, the weight vector is directly employed to modify the reward or payment associated with each creator-user interaction.
In the case of SMT or HMT intervention types, the platform requires a function to map to or . This map** can be implemented as a piecewise constant function and determined empirically. The specifics of this process are elucidated in Algorithm 4 and 5.
C Proof of Theorem 1
We restate Theorem 1 as the following with more rigorous characterizations, and then provide its detailed proof.
Theorem 3
Any game with has a unique pure Nash equilibrium (PNE) if each creator’s srtategy set is convex and is twice-differentiable and satisfies
(15) |
Proof
We prove that under the proposed conditions, the is a strictly monotone game Rosen (1965) and thus possesses a unique PNE. According to Appendix A in Bravo et al. (2018), a sufficient condition that establishes strictly monotonicity for any -person game is convex action sets and a negative definite Hessian of , which is defined as
For game, the convexity of strategy sets are satisfied. Next we prove the property of the game’s Hessian matrix with associated utility function
(16) |
Without loss of generality, let . Denote , we have
Next we show that for any and , the block matrix is always positive semi-definite (PSD). For simplicity, let
we obtain
Therefore, it suffices to prove that the matrix
is PSD. For any where , we can verify that
Therefore, the block matrix is always PSD for any and . A sufficient condition for to be negative definite is thus being positive definite (PD), i.e., . It remains to show that
(17) |
And a sufficient condition for Eq (17) to hold is
(18) |
which completes the proof.
D Proof of Theorem 2
Proof Since the utility functions of are twice differentiable, any LNE of satisfies the following definition
(19) |
must also satisfy the first-order condition . If we let
(20) |
be a vector-valued function, the constraint (12) can be rewritten into
(21) |
From the implicit function theorem (Krantz and Parks, 2002), the derivative of w.r.t. can be written as
where are the Jacobian matrices, and
(22) |
where is the partial derivative of w.r.t. .
Since , we apply a change of variable and denote each as instead. Next we calculate each term of the RHS of (22) to obtain an estimation of the gradient of our objective welfare function to the user weight vector . Without loss of generality we let the user distribution be a uniform distribution on unit basis and . The utility functions given in Eq (6) and the user welfare function read
(23) |
(24) |
If we denote , then is exactly the probability of matching content to . Given the assumption that is sufficiently large, we have is sufficiently small for any and therefore we ignore the high-order infiintesimal terms such as in the following derivation.
(25) |
where .
Next we calculate each term in the RHS of Eq (22). The -th block of is a -dimensional vector given by
(26) |
the -th block of is a -dimensional vector given by
(27) |
Since are orthogonal basis, the non-diagonal blocks of matrix are all zero matrices and the -th diagonal block of matrix is given by
(28) | ||||
Therefore, we can derive a approximation of as below:
(29) | ||||
(30) |
where (29) holds because .
Therefore, Eq (30) suggests that the following update rule
(31) |
aligns with the gradient direction of , which yields Eq (13).