Grading and Ranking Large number of candidates
Authors:
Rida Laraki,
Estelle Varloot
Abstract:
It is common that a jury must grade a set of candidates in a cardinal scale such as {1,2,3,4,5} or an ordinal scale such as {Great, Good, Average, Bad }. When the number of candidates is very large such as hotels (BOOKING), restaurants (GOOGLE), apartments (AIRBNB), drivers (UBER), or papers (EC), it is unreasonable to assume that each jury member will provide a separate grade for each candidate.…
▽ More
It is common that a jury must grade a set of candidates in a cardinal scale such as {1,2,3,4,5} or an ordinal scale such as {Great, Good, Average, Bad }. When the number of candidates is very large such as hotels (BOOKING), restaurants (GOOGLE), apartments (AIRBNB), drivers (UBER), or papers (EC), it is unreasonable to assume that each jury member will provide a separate grade for each candidate. Each jury member is more likely to abstain for some candidates, cast a blank vote, or be associated at random, or as a function of its expertise, with only a small subset of the candidates and is asked to grade each of those. Extending the classical theory, we study aggregation methods in which a voter will not be eligible to grade all the candidates, and the candidates are not eligible for the same sets of voters. Moreover, each candidate on which they are eligible, the voter will have the choice between: a blank vote, grade the candidate, or abstain. Assuming single-peaked preferences over the grades, we axiomatically characterise a broad class of strategy-proof grading mechanisms satisfying axioms such as unanimity, anonymity, neutrality, participation or consistency. Finally, when a strict ranking is necessary (to distinguish let say between two borderline papers in a conference), some tie-breaking rules, extending the leximin and majority judgment, are defined and are shown to be equivalent to some strategy-proof grading functions on a richer space of outcome. Our paper will propose new rules, called phantom-proxy mechanisms, to aggregate the votes in the examples above or others, which differ from the usual average mark, that are easily manipulable. Moreover, the phantom-proxy are able to reduce the injustices caused by some candidates juries too generous or severe.
△ Less
Submitted 23 February, 2023;
originally announced February 2023.
Level-strategyproof Belief Aggregation Mechanisms
Authors:
Rida Laraki,
Estelle Varloot
Abstract:
In the problem of aggregating experts' probabilistic predictions over an ordered set of outcomes, we introduce the axiom of level-strategy\-proofness (level-SP) and prove that it is a natural notion with several applications. Moreover, it is a robust concept as it implies incentive compatibility in a rich domain of single-peakedness over the space of cumulative distribution functions (CDFs). This…
▽ More
In the problem of aggregating experts' probabilistic predictions over an ordered set of outcomes, we introduce the axiom of level-strategy\-proofness (level-SP) and prove that it is a natural notion with several applications. Moreover, it is a robust concept as it implies incentive compatibility in a rich domain of single-peakedness over the space of cumulative distribution functions (CDFs). This contrasts with the literature which assumes single-peaked preferences over the space of probability distributions. Our main results are: (1) a reduction of our problem to the aggregation of CDFs; (2) the axiomatic characterization of level-SP probability aggregation functions with and without the addition of other axioms; (3) impossibility results which provide bounds for our characterization; (4) the axiomatic characterization of two new and practical level-SP methods: the proportional-cumulative method and the middlemost-cumulative method; and (5) the application of proportional-cumulative to extend approval voting, majority rule, and majority judgment methods to situations where voters/experts are uncertain about how to grade the candidates/alternatives to be ranked.\footnote{We are grateful to Thomas Boyer-Kassem, Roger Cooke, Aris Filos-Ratsikas, Hervé Moulin, Clemens Puppe and some anonymous EC2021 referees for their helpful comments and suggestions.}
\keywords{Probability Aggregation Functions \and ordered Set of Alternatives \and Level Strategy-Proofness \and Proportional-Cumulative \and Middlemost-Cumulative}
△ Less
Submitted 13 September, 2022; v1 submitted 10 August, 2021;
originally announced August 2021.
New Characterizations of Strategy-Proofness under Single-Peakedness
Authors:
Andrew Jennings,
Rida Laraki,
Clemens Puppe,
Estelle Varloot
Abstract:
We provide novel simple representations of strategy-proof voting rules when voters have uni-dimensional single-peaked preferences (as well as multi-dimensional separable preferences). The analysis recovers, links and unifies existing results in the literature such as Moulin's classic characterization in terms of phantom voters and Barberà, Gul and Stacchetti's in terms of winning coalitions ("gene…
▽ More
We provide novel simple representations of strategy-proof voting rules when voters have uni-dimensional single-peaked preferences (as well as multi-dimensional separable preferences). The analysis recovers, links and unifies existing results in the literature such as Moulin's classic characterization in terms of phantom voters and Barberà, Gul and Stacchetti's in terms of winning coalitions ("generalized median voter schemes"). First, we compare the computational properties of the various representations and show that the grading curve representation is superior in terms of computational complexity. Moreover, the new approach allows us to obtain new characterizations when strategy-proofness is combined with other desirable properties such as anonymity, responsiveness, ordinality, participation, consistency, or proportionality. In the anonymous case, two methods are single out: the -- well know -- ordinal median and the -- most recent -- linear median.
△ Less
Submitted 16 June, 2022; v1 submitted 23 February, 2021;
originally announced February 2021.
