-
Learning Rich Rankings
Authors:
Arjun Seshadri,
Stephen Ragain,
Johan Ugander
Abstract:
Although the foundations of ranking are well established, the ranking literature has primarily been focused on simple, unimodal models, e.g. the Mallows and Plackett-Luce models, that define distributions centered around a single total ordering. Explicit mixture models have provided some tools for modelling multimodal ranking data, though learning such models from data is often difficult. In this…
▽ More
Although the foundations of ranking are well established, the ranking literature has primarily been focused on simple, unimodal models, e.g. the Mallows and Plackett-Luce models, that define distributions centered around a single total ordering. Explicit mixture models have provided some tools for modelling multimodal ranking data, though learning such models from data is often difficult. In this work, we contribute a contextual repeated selection (CRS) model that leverages recent advances in choice modeling to bring a natural multimodality and richness to the rankings space. We provide rigorous theoretical guarantees for maximum likelihood estimation under the model through structure-dependent tail risk and expected risk bounds. As a by-product, we also furnish the first tight bounds on the expected risk of maximum likelihood estimators for the multinomial logit (MNL) choice model and the Plackett-Luce (PL) ranking model, as well as the first tail risk bound on the PL ranking model. The CRS model significantly outperforms existing methods for modeling real world ranking data in a variety of settings, from racing to rank choice voting.
△ Less
Submitted 22 December, 2023;
originally announced December 2023.
-
MiCRO: Multi-interest Candidate Retrieval Online
Authors:
Frank Portman,
Stephen Ragain,
Ahmed El-Kishky
Abstract:
Providing personalized recommendations in an environment where items exhibit ephemerality and temporal relevancy (e.g. in social media) presents a few unique challenges: (1) inductively understanding ephemeral appeal for items in a setting where new items are created frequently, (2) adapting to trends within engagement patterns where items may undergo temporal shifts in relevance, (3) accurately m…
▽ More
Providing personalized recommendations in an environment where items exhibit ephemerality and temporal relevancy (e.g. in social media) presents a few unique challenges: (1) inductively understanding ephemeral appeal for items in a setting where new items are created frequently, (2) adapting to trends within engagement patterns where items may undergo temporal shifts in relevance, (3) accurately modeling user preferences over this item space where users may express multiple interests. In this work we introduce MiCRO, a generative statistical framework that models multi-interest user preferences and temporal multi-interest item representations. Our framework is specifically formulated to adapt to both new items and temporal patterns of engagement. MiCRO demonstrates strong empirical performance on candidate retrieval experiments performed on two large scale user-item datasets: (1) an open-source temporal dataset of (User, User) follow interactions and (2) a temporal dataset of (User, Tweet) favorite interactions which we will open-source as an additional contribution to the community.
△ Less
Submitted 28 October, 2022;
originally announced October 2022.
-
Birdwatch: Crowd Wisdom and Bridging Algorithms can Inform Understanding and Reduce the Spread of Misinformation
Authors:
Stefan Wojcik,
Sophie Hilgard,
Nick Judd,
Delia Mocanu,
Stephen Ragain,
M. B. Fallin Hunzaker,
Keith Coleman,
Jay Baxter
Abstract:
We present an approach for selecting objectively informative and subjectively helpful annotations to social media posts. We draw on data from on an online environment where contributors annotate misinformation and simultaneously rate the contributions of others. Our algorithm uses a matrix-factorization (MF) based approach to identify annotations that appeal broadly across heterogeneous user group…
▽ More
We present an approach for selecting objectively informative and subjectively helpful annotations to social media posts. We draw on data from on an online environment where contributors annotate misinformation and simultaneously rate the contributions of others. Our algorithm uses a matrix-factorization (MF) based approach to identify annotations that appeal broadly across heterogeneous user groups - sometimes referred to as "bridging-based ranking." We pair these data with a survey experiment in which individuals are randomly assigned to see annotations to posts. We find that annotations selected by the algorithm improve key indicators compared with overall average and crowd-generated baselines. Further, when deployed on Twitter, people who saw annotations selected through this bridging-based approach were significantly less likely to reshare social media posts than those who did not see the annotations.
△ Less
Submitted 27 October, 2022;
originally announced October 2022.
-
Choosing to Rank
Authors:
Stephen Ragain,
Johan Ugander
Abstract:
Ranking data arises in a wide variety of application areas but remains difficult to model, learn from, and predict. Datasets often exhibit multimodality, intransitivity, or incomplete rankings---particularly when generated by humans---yet popular probabilistic models are often too rigid to capture such complexities. In this work we leverage recent progress on similar challenges in discrete choice…
▽ More
Ranking data arises in a wide variety of application areas but remains difficult to model, learn from, and predict. Datasets often exhibit multimodality, intransitivity, or incomplete rankings---particularly when generated by humans---yet popular probabilistic models are often too rigid to capture such complexities. In this work we leverage recent progress on similar challenges in discrete choice modeling to form flexible and tractable choice-based models for ranking data. We study choice representations, maps from rankings (complete or top-$k$) to collections of choices, as a way of forming ranking models from choice models. We focus on the repeated selection (RS) choice representation, first used to form the Plackett-Luce ranking model from the conditional multinomial logit choice model. We fully characterize, for a prime number of alternatives, the choice representations that admit ranking distributions with unit normalization, a desirably property that greatly simplifies maximum likelihood estimation. We further show that only specific minor variations on repeated selection exhibit this property. Our choice-based ranking models provide higher out-of-sample likelihood when compared to Plackett-Luce and Mallows models on a broad collection of ranking tasks including food preferences, ranked-choice elections, car racing, and search engine relevance tasks.
△ Less
Submitted 28 January, 2019; v1 submitted 13 September, 2018;
originally announced September 2018.
-
Improving pairwise comparison models using Empirical Bayes shrinkage
Authors:
Stephen Ragain,
Alexander Peysakhovich,
Johan Ugander
Abstract:
Comparison data arises in many important contexts, e.g. shop**, web clicks, or sports competitions. Typically we are given a dataset of comparisons and wish to train a model to make predictions about the outcome of unseen comparisons. In many cases available datasets have relatively few comparisons (e.g. there are only so many NFL games per year) or efficiency is important (e.g. we want to quick…
▽ More
Comparison data arises in many important contexts, e.g. shop**, web clicks, or sports competitions. Typically we are given a dataset of comparisons and wish to train a model to make predictions about the outcome of unseen comparisons. In many cases available datasets have relatively few comparisons (e.g. there are only so many NFL games per year) or efficiency is important (e.g. we want to quickly estimate the relative appeal of a product). In such settings it is well known that shrinkage estimators outperform maximum likelihood estimators. A complicating matter is that standard comparison models such as the conditional multinomial logit model are only models of conditional outcomes (who wins) and not of comparisons themselves (who competes). As such, different models of the comparison process lead to different shrinkage estimators. In this work we derive a collection of methods for estimating the pairwise uncertainty of pairwise predictions based on different assumptions about the comparison process. These uncertainty estimates allow us both to examine model uncertainty as well as perform Empirical Bayes shrinkage estimation of the model parameters. We demonstrate that our shrunk estimators outperform standard maximum likelihood methods on real comparison data from online comparison surveys as well as from several sports contexts.
△ Less
Submitted 24 July, 2018;
originally announced July 2018.
-
Pairwise Choice Markov Chains
Authors:
Stephen Ragain,
Johan Ugander
Abstract:
As datasets capturing human choices grow in richness and scale -- particularly in online domains -- there is an increasing need for choice models that escape traditional choice-theoretic axioms such as regularity, stochastic transitivity, and Luce's choice axiom. In this work we introduce the Pairwise Choice Markov Chain (PCMC) model of discrete choice, an inferentially tractable model that does n…
▽ More
As datasets capturing human choices grow in richness and scale -- particularly in online domains -- there is an increasing need for choice models that escape traditional choice-theoretic axioms such as regularity, stochastic transitivity, and Luce's choice axiom. In this work we introduce the Pairwise Choice Markov Chain (PCMC) model of discrete choice, an inferentially tractable model that does not assume any of the above axioms while still satisfying the foundational axiom of uniform expansion, a considerably weaker assumption than Luce's choice axiom. We show that the PCMC model significantly outperforms the Multinomial Logit (MNL) model in prediction tasks on both synthetic and empirical datasets known to exhibit violations of Luce's axiom. Our analysis also synthesizes several recent observations connecting the Multinomial Logit model and Markov chains; the PCMC model retains the Multinomial Logit model as a special case.
△ Less
Submitted 14 May, 2021; v1 submitted 8 March, 2016;
originally announced March 2016.
-
A Leapfrog Strategy for Pursuit-Evasion in a Polygonal Environment
Authors:
Brendan Ames,
Andrew Beveridge,
Rosalie Carlson,
Claire Djang,
Volkan Isler,
Stephen Ragain,
Maxray Savage
Abstract:
We study pursuit-evasion in a polygonal environment with polygonal obstacles. In this turn based game, an evader $e$ is chased by pursuers $p_1, p_2, ..., p_{\ell}$. The players have full information about the environment and the location of the other players. The pursuers are allowed to coordinate their actions. On the pursuer turn, each $p_i$ can move to any point at distance at most 1 from his…
▽ More
We study pursuit-evasion in a polygonal environment with polygonal obstacles. In this turn based game, an evader $e$ is chased by pursuers $p_1, p_2, ..., p_{\ell}$. The players have full information about the environment and the location of the other players. The pursuers are allowed to coordinate their actions. On the pursuer turn, each $p_i$ can move to any point at distance at most 1 from his current location. On the evader turn, he moves similarly. The pursuers win if some pursuer becomes co-located with the evader in finite time. The evader wins if he can evade capture forever.
It is known that one pursuer can capture the evader in any simply-connected polygonal environment, and that three pursuers are always sufficient in any polygonal environment (possibly with polygonal obstacles). We contribute two new results to this field. First, we fully characterize when an environment with a single obstacles is one-pursuer-win or two-pursuer-win. Second, we give sufficient (but not necessary) conditions for an environment to have a winning strategy for two pursuers. Such environments can be swept by a \emph{leapfrog strategy} in which the two cops alternately guard/increase the currently controlled area. The running time of this algorithm is $O(n \cdot h \cdot {diam}(P))$ where $n$ is the number of vertices, $h$ is the number of obstacles and ${diam}(P)$ is the diameter of $P$.
More concretely, for an environment with $n$ vertices, we describe an $O(n^2)$ algorithm that (1) determines whether the obstacles are well-separated, and if so, (2) constructs the required partition for a leapfrog strategy.
△ Less
Submitted 19 October, 2014; v1 submitted 13 January, 2014;
originally announced January 2014.
