Showing 1–2 of 2 results for author: Agrawal, A
-
Debiasing classifiers: is reality at variance with expectation?
Authors:
Ashrya Agrawal,
Florian Pfisterer,
Bernd Bischl,
Francois Buet-Golfouse,
Srijan Sood,
Jiahao Chen,
Sameena Shah,
Sebastian Vollmer
Abstract:
We present an empirical study of debiasing methods for classifiers, showing that debiasers often fail in practice to generalize out-of-sample, and can in fact make fairness worse rather than better. A rigorous evaluation of the debiasing treatment effect requires extensive cross-validation beyond what is usually done. We demonstrate that this phenomenon can be explained as a consequence of bias-va…
▽ More
We present an empirical study of debiasing methods for classifiers, showing that debiasers often fail in practice to generalize out-of-sample, and can in fact make fairness worse rather than better. A rigorous evaluation of the debiasing treatment effect requires extensive cross-validation beyond what is usually done. We demonstrate that this phenomenon can be explained as a consequence of bias-variance trade-off, with an increase in variance necessitated by imposing a fairness constraint. Follow-up experiments validate the theoretical prediction that the estimation variance depends strongly on the base rates of the protected class. Considering fairness--performance trade-offs justifies the counterintuitive notion that partial debiasing can actually yield better results in practice on out-of-sample data.
△ Less
Submitted 30 May, 2021; v1 submitted 4 November, 2020;
originally announced November 2020.
-
Authoritarian Governments Appear to Manipulate COVID Data
Authors:
Mudit Kapoor,
Anup Malani,
Shamika Ravi,
Arnav Agrawal
Abstract:
Because SARS-Cov-2 (COVID-19) statistics affect economic policies and political outcomes, governments have an incentive to control them. Manipulation may be less likely in democracies, which have checks to ensure transparency. We show that data on disease burden bear indicia of data modification by authoritarian governments relative to democratic governments. First, data on COVID-19 cases and deat…
▽ More
Because SARS-Cov-2 (COVID-19) statistics affect economic policies and political outcomes, governments have an incentive to control them. Manipulation may be less likely in democracies, which have checks to ensure transparency. We show that data on disease burden bear indicia of data modification by authoritarian governments relative to democratic governments. First, data on COVID-19 cases and deaths from authoritarian governments show significantly less variation from a 7 day moving average. Because governments have no reason to add noise to data, lower deviation is evidence that data may be massaged. Second, data on COVID-19 deaths from authoritarian governments do not follow Benford's law, which describes the distribution of leading digits of numbers. Deviations from this law are used to test for accounting fraud. Smoothing and adjustments to COVID-19 data may indicate other alterations to these data and a need to account for such alterations when tracking the disease.
△ Less
Submitted 18 July, 2020;
originally announced July 2020.
