-
Statistical Inference and A/B Testing in Fisher Markets and Paced Auctions
Authors:
Luofeng Liao,
Christian Kroer
Abstract:
We initiate the study of statistical inference and A/B testing for two market equilibrium models: linear Fisher market (LFM) equilibrium and first-price pacing equilibrium (FPPE). LFM arises from fair resource allocation systems such as allocation of food to food banks and notification opportunities to different types of notifications. For LFM, we assume that the data observed is captured by the c…
▽ More
We initiate the study of statistical inference and A/B testing for two market equilibrium models: linear Fisher market (LFM) equilibrium and first-price pacing equilibrium (FPPE). LFM arises from fair resource allocation systems such as allocation of food to food banks and notification opportunities to different types of notifications. For LFM, we assume that the data observed is captured by the classical finite-dimensional Fisher market equilibrium, and its steady-state behavior is modeled by a continuous limit Fisher market. The second type of equilibrium we study, FPPE, arises from internet advertising where advertisers are constrained by budgets and advertising opportunities are sold via first-price auctions. For platforms that use pacing-based methods to smooth out the spending of advertisers, FPPE provides a hindsight-optimal configuration of the pacing method. We propose a statistical framework for the FPPE model, in which a continuous limit FPPE models the steady-state behavior of the auction platform, and a finite FPPE provides the data to estimate primitives of the limit FPPE. Both LFM and FPPE have an Eisenberg-Gale convex program characterization, the pillar upon which we derive our statistical theory. We start by deriving basic convergence results for the finite market to the limit market. We then derive asymptotic distributions, and construct confidence intervals. Furthermore, we establish the asymptotic local minimax optimality of estimation based on finite markets. We then show that the theory can be used for conducting statistically valid A/B testing on auction platforms. Synthetic and semi-synthetic experiments verify the validity and practicality of our theory.
△ Less
Submitted 20 June, 2024;
originally announced June 2024.
-
Interference Among First-Price Pacing Equilibria: A Bias and Variance Analysis
Authors:
Luofeng Liao,
Christian Kroer,
Sergei Leonenkov,
Okke Schrijvers,
Liang Shi,
Nicolas Stier-Moses,
Congshan Zhang
Abstract:
Online A/B testing is widely used in the internet industry to inform decisions on new feature roll-outs. For online marketplaces (such as advertising markets), standard approaches to A/B testing may lead to biased results when buyers operate under a budget constraint, as budget consumption in one arm of the experiment impacts performance of the other arm. To counteract this interference, one can u…
▽ More
Online A/B testing is widely used in the internet industry to inform decisions on new feature roll-outs. For online marketplaces (such as advertising markets), standard approaches to A/B testing may lead to biased results when buyers operate under a budget constraint, as budget consumption in one arm of the experiment impacts performance of the other arm. To counteract this interference, one can use a budget-split design where the budget constraint operates on a per-arm basis and each arm receives an equal fraction of the budget, leading to ``budget-controlled A/B testing.'' Despite clear advantages of budget-controlled A/B testing, performance degrades when budget are split too small, limiting the overall throughput of such systems. In this paper, we propose a parallel budget-controlled A/B testing design where we use market segmentation to identify submarkets in the larger market, and we run parallel experiments on each submarket.
Our contributions are as follows: First, we introduce and demonstrate the effectiveness of the parallel budget-controlled A/B test design with submarkets in a large online marketplace environment. Second, we formally define market interference in first-price auction markets using the first price pacing equilibrium (FPPE) framework. Third, we propose a debiased surrogate that eliminates the first-order bias of FPPE, drawing upon the principles of sensitivity analysis in mathematical programs. Fourth, we derive a plug-in estimator for the surrogate and establish its asymptotic normality. Fifth, we provide an estimation procedure for submarket parallel budget-controlled A/B tests. Finally, we present numerical examples on semi-synthetic data, confirming that the debiasing technique achieves the desired coverage properties.
△ Less
Submitted 11 February, 2024;
originally announced February 2024.
-
Bootstrap** Fisher Market Equilibrium and First-Price Pacing Equilibrium
Authors:
Luofeng Liao,
Christian Kroer
Abstract:
The linear Fisher market (LFM) is a basic equilibrium model from economics, which also has applications in fair and efficient resource allocation. First-price pacing equilibrium (FPPE) is a model capturing budget-management mechanisms in first-price auctions. In certain practical settings such as advertising auctions, there is an interest in performing statistical inference over these models. A po…
▽ More
The linear Fisher market (LFM) is a basic equilibrium model from economics, which also has applications in fair and efficient resource allocation. First-price pacing equilibrium (FPPE) is a model capturing budget-management mechanisms in first-price auctions. In certain practical settings such as advertising auctions, there is an interest in performing statistical inference over these models. A popular methodology for general statistical inference is the bootstrap procedure. Yet, for LFM and FPPE there is no existing theory for the valid application of bootstrap procedures. In this paper, we introduce and devise several statistically valid bootstrap inference procedures for LFM and FPPE. The most challenging part is to bootstrap general FPPE, which reduces to bootstrap** constrained M-estimators, a largely unexplored problem. We devise a bootstrap procedure for FPPE under mild degeneracy conditions by using the powerful tool of epi-convergence theory. Experiments with synthetic and semi-real data verify our theory.
△ Less
Submitted 11 February, 2024; v1 submitted 3 February, 2024;
originally announced February 2024.
-
Statistical Inference and A/B Testing for First-Price Pacing Equilibria
Authors:
Luofeng Liao,
Christian Kroer
Abstract:
We initiate the study of statistical inference and A/B testing for first-price pacing equilibria (FPPE). The FPPE model captures the dynamics resulting from large-scale first-price auction markets where buyers use pacing-based budget management. Such markets arise in the context of internet advertising, where budgets are prevalent.
We propose a statistical framework for the FPPE model, in which…
▽ More
We initiate the study of statistical inference and A/B testing for first-price pacing equilibria (FPPE). The FPPE model captures the dynamics resulting from large-scale first-price auction markets where buyers use pacing-based budget management. Such markets arise in the context of internet advertising, where budgets are prevalent.
We propose a statistical framework for the FPPE model, in which a limit FPPE with a continuum of items models the long-run steady-state behavior of the auction platform, and an observable FPPE consisting of a finite number of items provides the data to estimate primitives of the limit FPPE, such as revenue, Nash social welfare (a fair metric of efficiency), and other parameters of interest. We develop central limit theorems and asymptotically valid confidence intervals. Furthermore, we establish the asymptotic local minimax optimality of our estimators. We then show that the theory can be used for conducting statistically valid A/B testing on auction platforms. Numerical simulations verify our central limit theorems, and empirical coverage rates for our confidence intervals agree with our theory.
△ Less
Submitted 28 June, 2023; v1 submitted 5 January, 2023;
originally announced January 2023.
-
Statistical Inference for Fisher Market Equilibrium
Authors:
Luofeng Liao,
Yuan Gao,
Christian Kroer
Abstract:
Statistical inference under market equilibrium effects has attracted increasing attention recently. In this paper we focus on the specific case of linear Fisher markets. They have been widely use in fair resource allocation of food/blood donations and budget management in large-scale Internet ad auctions. In resource allocation, it is crucial to quantify the variability of the resource received by…
▽ More
Statistical inference under market equilibrium effects has attracted increasing attention recently. In this paper we focus on the specific case of linear Fisher markets. They have been widely use in fair resource allocation of food/blood donations and budget management in large-scale Internet ad auctions. In resource allocation, it is crucial to quantify the variability of the resource received by the agents (such as blood banks and food banks) in addition to fairness and efficiency properties of the systems. For ad auction markets, it is important to establish statistical properties of the platform's revenues in addition to their expected values. To this end, we propose a statistical framework based on the concept of infinite-dimensional Fisher markets. In our framework, we observe a market formed by a finite number of items sampled from an underlying distribution (the "observed market") and aim to infer several important equilibrium quantities of the underlying long-run market. These equilibrium quantities include individual utilities, social welfare, and pacing multipliers. Through the lens of sample average approximation (SSA), we derive a collection of statistical results and show that the observed market provides useful statistical information of the long-run market. In other words, the equilibrium quantities of the observed market converge to the true ones of the long-run market with strong statistical guarantees. These include consistency, finite sample bounds, asymptotics, and confidence. As an extension, we discuss revenue inference in quasilinear Fisher markets.
△ Less
Submitted 29 September, 2022;
originally announced September 2022.
-
Online Market Equilibrium with Application to Fair Division
Authors:
Yuan Gao,
Christian Kroer,
Alex Peysakhovich
Abstract:
Computing market equilibria is a problem of both theoretical and applied interest. Much research to date focuses on the case of static Fisher markets with full information on buyers' utility functions and item supplies. Motivated by real-world markets, we consider an online setting: individuals have linear, additive utility functions; items arrive sequentially and must be allocated and priced irre…
▽ More
Computing market equilibria is a problem of both theoretical and applied interest. Much research to date focuses on the case of static Fisher markets with full information on buyers' utility functions and item supplies. Motivated by real-world markets, we consider an online setting: individuals have linear, additive utility functions; items arrive sequentially and must be allocated and priced irrevocably. We define the notion of an online market equilibrium in such a market as time-indexed allocations and prices which guarantee buyer optimality and market clearance in hindsight. We propose a simple, scalable and interpretable allocation and pricing dynamics termed as PACE. When items are drawn i.i.d. from an unknown distribution (with a possibly continuous support), we show that PACE leads to an online market equilibrium asymptotically. In particular, PACE ensures that buyers' time-averaged utilities converge to the equilibrium utilities w.r.t. a static market with item supplies being the unknown distribution and that buyers' time-averaged expenditures converge to their per-period budget. Hence, many desirable properties of market equilibrium-based fair division such as no envy, Pareto optimality, and the proportional-share guarantee are also attained asymptotically in the online setting. Next, we extend the dynamics to handle quasilinear buyer utilities, which gives the first online algorithm for computing first-price pacing equilibria. Finally, numerical experiments on real and synthetic datasets show that the dynamics converges quickly under various metrics.
△ Less
Submitted 2 October, 2021; v1 submitted 23 March, 2021;
originally announced March 2021.
-
Contextual Standard Auctions with Budgets: Revenue Equivalence and Efficiency Guarantees
Authors:
Santiago Balseiro,
Christian Kroer,
Rachitesh Kumar
Abstract:
The internet advertising market is a multi-billion dollar industry, in which advertisers buy thousands of ad placements every day by repeatedly participating in auctions. An important and ubiquitous feature of these auctions is the presence of campaign budgets, which specify the maximum amount the advertisers are willing to pay over a specified time period. In this paper, we present a new model to…
▽ More
The internet advertising market is a multi-billion dollar industry, in which advertisers buy thousands of ad placements every day by repeatedly participating in auctions. An important and ubiquitous feature of these auctions is the presence of campaign budgets, which specify the maximum amount the advertisers are willing to pay over a specified time period. In this paper, we present a new model to study the equilibrium bidding strategies in standard auctions, a large class of auctions that includes first- and second-price auctions, for advertisers who satisfy budget constraints on average. Our model dispenses with the common, yet unrealistic assumption that advertisers' values are independent and instead assumes a contextual model in which advertisers determine their values using a common feature vector. We show the existence of a natural value-pacing-based Bayes-Nash equilibrium under very mild assumptions. Furthermore, we prove a revenue equivalence showing that all standard auctions yield the same revenue even in the presence of budget constraints. Leveraging this equivalence, we prove Price of Anarchy bounds for liquid welfare and structural properties of pacing-based equilibria that hold for all standard auctions. In recent years, the internet advertising market has adopted first-price auctions as the preferred paradigm for selling advertising slots. Our work thus takes an important step toward understanding the implications of the shift to first-price auctions in internet advertising markets by studying how the choice of the selling mechanism impacts revenues, welfare, and advertisers' bidding strategies.
△ Less
Submitted 9 October, 2022; v1 submitted 20 February, 2021;
originally announced February 2021.
-
Infinite-Dimensional Fisher Markets and Tractable Fair Division
Authors:
Yuan Gao,
Christian Kroer
Abstract:
Linear Fisher markets are a fundamental economic model with applications in fair division as well as large-scale Internet markets. In the finite-dimensional case of $n$ buyers and $m$ items, a market equilibrium can be computed using the Eisenberg-Gale convex program. Motivated by large-scale Internet advertising and fair division applications, this paper considers a generalization of a linear Fis…
▽ More
Linear Fisher markets are a fundamental economic model with applications in fair division as well as large-scale Internet markets. In the finite-dimensional case of $n$ buyers and $m$ items, a market equilibrium can be computed using the Eisenberg-Gale convex program. Motivated by large-scale Internet advertising and fair division applications, this paper considers a generalization of a linear Fisher market where there is a finite set of buyers and a continuum of items. We introduce generalizations of the Eisenberg-Gale convex program and its dual to this infinite-dimensional setting, which leads to Banach-space optimization problems. We establish existence of optimal solutions, strong duality, as well as necessity and sufficiency of KKT-type conditions. All these properties are established via non-standard arguments, which circumvent the limitations of duality theory in optimization over infinite-dimensional Banach spaces. Furthermore, we show that there exists a pure equilibrium allocation, i.e., a division of the item space. When the item space is a closed interval and buyers have piecewise linear valuations, we show that the Eisenberg-Gale-type convex program over the infinite-dimensional allocations can be reformulated as a finite-dimensional convex conic program, which can be solved efficiently using off-the-shelf optimization software based on primal-dual interior-point methods. Based on our convex conic reformulation, we develop the first polynomial-time cake-cutting algorithm that achieves Pareto optimality, envy-freeness, and proportionality. For general buyer valuations or a very large number of buyers, we propose computing market equilibrium using stochastic dual averaging, which finds approximate equilibrium prices with high probability. Finally, we discuss how the above results easily extend to the case of quasilinear utilities.
△ Less
Submitted 5 April, 2021; v1 submitted 6 October, 2020;
originally announced October 2020.
-
Dominant Resource Fairness with Meta-Types
Authors:
Steven Yin,
Shatian Wang,
Lingyi Zhang,
Christian Kroer
Abstract:
Inspired by the recent COVID-19 pandemic, we study a generalization of the multi-resource allocation problem with heterogeneous demands and Leontief utilities. Unlike existing settings, we allow each agent to specify requirements to only accept allocations from a subset of the total supply for each resource. These requirements can take form in location constraints (e.g. A hospital can only accept…
▽ More
Inspired by the recent COVID-19 pandemic, we study a generalization of the multi-resource allocation problem with heterogeneous demands and Leontief utilities. Unlike existing settings, we allow each agent to specify requirements to only accept allocations from a subset of the total supply for each resource. These requirements can take form in location constraints (e.g. A hospital can only accept volunteers who live nearby due to commute limitations). This can also model a type of substitution effect where some agents need 1 unit of resource A \emph{or} B, both belonging to the same meta-type. But some agents specifically want A, and others specifically want B. We propose a new mechanism called Dominant Resource Fairness with Meta Types which determines the allocations by solving a small number of linear programs. The proposed method satisfies Pareto optimality, envy-freeness, strategy-proofness, and a notion of sharing incentive for our setting. To the best of our knowledge, we are the first to study this problem formulation, which improved upon existing work by capturing more constraints that often arise in real life situations. Finally, we show numerically that our method scales better to large problems than alternative approaches.
△ Less
Submitted 12 August, 2021; v1 submitted 21 July, 2020;
originally announced July 2020.
-
Scalable Fair Division for 'At Most One' Preferences
Authors:
Christian Kroer,
Alexander Peysakhovich
Abstract:
Allocating multiple scarce items across a set of individuals is an important practical problem. In the case of divisible goods and additive preferences a convex program can be used to find the solution that maximizes Nash welfare (MNW). The MNW solution is equivalent to finding the equilibrium of a market economy (aka. the competitive equilibrium from equal incomes, CEEI) and thus has good propert…
▽ More
Allocating multiple scarce items across a set of individuals is an important practical problem. In the case of divisible goods and additive preferences a convex program can be used to find the solution that maximizes Nash welfare (MNW). The MNW solution is equivalent to finding the equilibrium of a market economy (aka. the competitive equilibrium from equal incomes, CEEI) and thus has good properties such as Pareto optimality, envy-freeness, and incentive compatibility in the large. Unfortunately, this equivalence (and nice properties) breaks down for general preference classes. Motivated by real world problems such as course allocation and recommender systems we study the case of additive `at most one' (AMO) preferences - individuals want at most 1 of each item and lotteries are allowed. We show that in this case the MNW solution is still a convex program and importantly is a CEEI solution when the instance gets large but has a `low rank' structure. Thus a polynomial time algorithm can be used to scale CEEI (which is in general PPAD-hard) for AMO preferences. We examine whether the properties guaranteed in the limit hold approximately in finite samples using several real datasets.
△ Less
Submitted 24 September, 2019;
originally announced September 2019.
