-
Quantifying Privacy Risks of Public Statistics to Residents of Subsidized Housing
Authors:
Ryan Steed,
Diana Qing,
Zhiwei Steven Wu
Abstract:
As the U.S. Census Bureau implements its controversial new disclosure avoidance system, researchers and policymakers debate the necessity of new privacy protections for public statistics. With experiments on both published statistics and synthetic data, we explore a particular privacy concern: respondents in subsidized housing may deliberately not mention unauthorized children and other household…
▽ More
As the U.S. Census Bureau implements its controversial new disclosure avoidance system, researchers and policymakers debate the necessity of new privacy protections for public statistics. With experiments on both published statistics and synthetic data, we explore a particular privacy concern: respondents in subsidized housing may deliberately not mention unauthorized children and other household members for fear of being evicted. By combining public statistics from the Decennial Census and the Department of Housing and Urban Development, we demonstrate a simple, inexpensive reconstruction attack that could identify subsidized households living in violation of occupancy guidelines in 2010. Experiments on synthetic data suggest that a random swap** mechanism similar to the Census Bureau's 2010 disclosure avoidance measures does not significantly reduce the precision of this attack, while a differentially private mechanism similar to the 2020 disclosure avoidance system does. Our results provide a valuable example for policymakers seeking a trustworthy, accurate census.
△ Less
Submitted 5 July, 2024;
originally announced July 2024.
-
Towards AI Accountability Infrastructure: Gaps and Opportunities in AI Audit Tooling
Authors:
Victor Ojewale,
Ryan Steed,
Briana Vecchione,
Abeba Birhane,
Inioluwa Deborah Raji
Abstract:
Audits are critical mechanisms for identifying the risks and limitations of deployed artificial intelligence (AI) systems. However, the effective execution of AI audits remains incredibly difficult. As a result, practitioners make use of various tools to support their efforts. Drawing on interviews with 35 AI audit practitioners and a landscape analysis of 390 tools, we map the current ecosystem o…
▽ More
Audits are critical mechanisms for identifying the risks and limitations of deployed artificial intelligence (AI) systems. However, the effective execution of AI audits remains incredibly difficult. As a result, practitioners make use of various tools to support their efforts. Drawing on interviews with 35 AI audit practitioners and a landscape analysis of 390 tools, we map the current ecosystem of available AI audit tools. While there are many tools designed to assist practitioners with setting standards and evaluating AI systems, these tools often fell short of supporting the accountability goals of AI auditing in practice. We thus highlight areas for future tool development beyond evaluation -- from harms discovery to advocacy -- and outline challenges practitioners faced in their efforts to use AI audit tools. We conclude that resources are lacking to adequately support the full scope of needs for many AI audit practitioners and recommend that the field move beyond tools for just evaluation, towards more comprehensive infrastructure for AI accountability.
△ Less
Submitted 14 March, 2024; v1 submitted 27 February, 2024;
originally announced February 2024.
-
AI auditing: The Broken Bus on the Road to AI Accountability
Authors:
Abeba Birhane,
Ryan Steed,
Victor Ojewale,
Briana Vecchione,
Inioluwa Deborah Raji
Abstract:
One of the most concrete measures to take towards meaningful AI accountability is to consequentially assess and report the systems' performance and impact. However, the practical nature of the "AI audit" ecosystem is muddled and imprecise, making it difficult to work through various concepts and map out the stakeholders involved in the practice. First, we taxonomize current AI audit practices as c…
▽ More
One of the most concrete measures to take towards meaningful AI accountability is to consequentially assess and report the systems' performance and impact. However, the practical nature of the "AI audit" ecosystem is muddled and imprecise, making it difficult to work through various concepts and map out the stakeholders involved in the practice. First, we taxonomize current AI audit practices as completed by regulators, law firms, civil society, journalism, academia, consulting agencies. Next, we assess the impact of audits done by stakeholders within each domain. We find that only a subset of AI audit studies translate to desired accountability outcomes. We thus assess and isolate practices necessary for effective AI audit results, articulating the observed connections between AI audit design, methodology and institutional context on its effectiveness as a meaningful mechanism for accountability.
△ Less
Submitted 25 January, 2024;
originally announced January 2024.
-
Image Representations Learned With Unsupervised Pre-Training Contain Human-like Biases
Authors:
Ryan Steed,
Aylin Caliskan
Abstract:
Recent advances in machine learning leverage massive datasets of unlabeled images from the web to learn general-purpose image representations for tasks from image classification to face recognition. But do unsupervised computer vision models automatically learn implicit patterns and embed social biases that could have harmful downstream effects? We develop a novel method for quantifying biased ass…
▽ More
Recent advances in machine learning leverage massive datasets of unlabeled images from the web to learn general-purpose image representations for tasks from image classification to face recognition. But do unsupervised computer vision models automatically learn implicit patterns and embed social biases that could have harmful downstream effects? We develop a novel method for quantifying biased associations between representations of social concepts and attributes in images. We find that state-of-the-art unsupervised models trained on ImageNet, a popular benchmark image dataset curated from internet images, automatically learn racial, gender, and intersectional biases. We replicate 8 previously documented human biases from social psychology, from the innocuous, as with insects and flowers, to the potentially harmful, as with race and gender. Our results closely match three hypotheses about intersectional bias from social psychology. For the first time in unsupervised computer vision, we also quantify implicit human biases about weight, disabilities, and several ethnicities. When compared with statistical patterns in online image datasets, our findings suggest that machine learning models can automatically learn bias from the way people are stereotypically portrayed on the web.
△ Less
Submitted 27 January, 2021; v1 submitted 28 October, 2020;
originally announced October 2020.
-
Heuristic-Based Weak Learning for Automated Decision-Making
Authors:
Ryan Steed,
Benjamin Williams
Abstract:
Machine learning systems impact many stakeholders and groups of users, often disparately. Prior studies have reconciled conflicting user preferences by aggregating a high volume of manually labeled pairwise comparisons, but this technique may be costly or impractical. How can we lower the barrier to participation in algorithm design? Instead of creating a simplified labeling task for a crowd, we s…
▽ More
Machine learning systems impact many stakeholders and groups of users, often disparately. Prior studies have reconciled conflicting user preferences by aggregating a high volume of manually labeled pairwise comparisons, but this technique may be costly or impractical. How can we lower the barrier to participation in algorithm design? Instead of creating a simplified labeling task for a crowd, we suggest collecting ranked decision-making heuristics from a focused sample of affected users. With empirical data from two use cases, we show that our weak learning approach, which requires little to no manual labeling, agrees with participants' pairwise choices nearly as often as fully supervised approaches.
△ Less
Submitted 2 December, 2020; v1 submitted 5 May, 2020;
originally announced May 2020.
-
A Set of Distinct Facial Traits Learned by Machines Is Not Predictive of Appearance Bias in the Wild
Authors:
Ryan Steed,
Aylin Caliskan
Abstract:
Research in social psychology has shown that people's biased, subjective judgments about another's personality based solely on their appearance are not predictive of their actual personality traits. But researchers and companies often utilize computer vision models to predict similarly subjective personality attributes such as "employability." We seek to determine whether state-of-the-art, black b…
▽ More
Research in social psychology has shown that people's biased, subjective judgments about another's personality based solely on their appearance are not predictive of their actual personality traits. But researchers and companies often utilize computer vision models to predict similarly subjective personality attributes such as "employability." We seek to determine whether state-of-the-art, black box face processing technology can learn human-like appearance biases. With features extracted with FaceNet, a widely used face recognition framework, we train a transfer learning model on human subjects' first impressions of personality traits in other faces as measured by social psychologists. We find that features extracted with FaceNet can be used to predict human appearance bias scores for deliberately manipulated faces but not for randomly generated faces scored by humans. Additionally, in contrast to work with human biases in social psychology, the model does not find a significant signal correlating politicians' vote shares with perceived competence bias. With Local Interpretable Model-Agnostic Explanations (LIME), we provide several explanations for this discrepancy. Our results suggest that some signals of appearance bias documented in social psychology are not embedded by the machine learning techniques we investigate. We shed light on the ways in which appearance bias could be embedded in face processing technology and cast further doubt on the practice of predicting subjective traits based on appearances.
△ Less
Submitted 13 January, 2021; v1 submitted 13 February, 2020;
originally announced February 2020.
