-
BLIP: Facilitating the Exploration of Undesirable Consequences of Digital Technologies
Authors:
Rock Yuren Pang,
Sebastin Santy,
René Just,
Katharina Reinecke
Abstract:
Digital technologies have positively transformed society, but they have also led to undesirable consequences not anticipated at the time of design or development. We posit that insights into past undesirable consequences can help researchers and practitioners gain awareness and anticipate potential adverse effects. To test this assumption, we introduce BLIP, a system that extracts real-world undes…
▽ More
Digital technologies have positively transformed society, but they have also led to undesirable consequences not anticipated at the time of design or development. We posit that insights into past undesirable consequences can help researchers and practitioners gain awareness and anticipate potential adverse effects. To test this assumption, we introduce BLIP, a system that extracts real-world undesirable consequences of technology from online articles, summarizes and categorizes them, and presents them in an interactive, web-based interface. In two user studies with 15 researchers in various computer science disciplines, we found that BLIP substantially increased the number and diversity of undesirable consequences they could list in comparison to relying on prior knowledge or searching online. Moreover, BLIP helped them identify undesirable consequences relevant to their ongoing projects, made them aware of undesirable consequences they "had never considered," and inspired them to reflect on their own experiences with technology.
△ Less
Submitted 10 May, 2024;
originally announced May 2024.
-
NormAd: A Benchmark for Measuring the Cultural Adaptability of Large Language Models
Authors:
Abhinav Rao,
Akhila Yerukola,
Vishwa Shah,
Katharina Reinecke,
Maarten Sap
Abstract:
The integration of Large Language Models (LLMs) into various global cultures fundamentally presents a cultural challenge: LLMs must navigate interactions, respect social norms, and avoid transgressing cultural boundaries. However, it is still unclear if LLMs can adapt their outputs to diverse cultural norms. Our study focuses on this aspect. We introduce NormAd, a novel dataset, which includes 2.6…
▽ More
The integration of Large Language Models (LLMs) into various global cultures fundamentally presents a cultural challenge: LLMs must navigate interactions, respect social norms, and avoid transgressing cultural boundaries. However, it is still unclear if LLMs can adapt their outputs to diverse cultural norms. Our study focuses on this aspect. We introduce NormAd, a novel dataset, which includes 2.6k stories that represent social and cultural norms from 75 countries, to assess the ability of LLMs to adapt to different granular levels of socio-cultural contexts such as the country of origin, its associated cultural values, and prevalent social norms. Our study reveals that LLMs struggle with cultural reasoning across all contextual granularities, showing stronger adaptability to English-centric cultures over those from the Global South. Even with explicit social norms, the top-performing model, Mistral-7b-Instruct, achieves only 81.8\% accuracy, lagging behind the 95.6\% achieved by humans. Evaluation on NormAd further reveals that LLMs struggle to adapt to stories involving gift-giving across cultures. Due to inherent agreement or sycophancy biases, LLMs find it considerably easier to assess the social acceptability of stories that adhere to cultural norms than those that deviate from them. Our benchmark measures the cultural adaptability (or lack thereof) of LLMs, emphasizing the potential to make these technologies more equitable and useful for global audiences. We release the NormAd dataset and its associated code on GitHub.
△ Less
Submitted 6 June, 2024; v1 submitted 18 April, 2024;
originally announced April 2024.
-
Imagine a dragon made of seaweed: How images enhance learning in Wikipedia
Authors:
Anita Silva,
Maria Tracy,
Katharina Reinecke,
Eytan Adar,
Miriam Redi
Abstract:
Though images are ubiquitous across Wikipedia, it is not obvious that the image choices optimally support learning. When well selected, images can enhance learning by dual coding, complementing, or supporting articles. When chosen poorly, images can mislead, distract, and confuse. We developed a large dataset containing 470 questions & answers to 94 Wikipedia articles with images on a wide range o…
▽ More
Though images are ubiquitous across Wikipedia, it is not obvious that the image choices optimally support learning. When well selected, images can enhance learning by dual coding, complementing, or supporting articles. When chosen poorly, images can mislead, distract, and confuse. We developed a large dataset containing 470 questions & answers to 94 Wikipedia articles with images on a wide range of topics. Through an online experiment (n=704), we determined whether the images displayed alongside the text of the article are effective in hel** readers understand and learn. For certain tasks, such as learning to identify targets visually (e.g., "which of these pictures is a gujia?"), article images significantly improve accuracy. Images did not significantly improve general knowledge questions (e.g., "where are gujia from?"). Most interestingly, only some images helped with visual knowledge questions (e.g., "what shape is a gujia?"). Using our findings, we reflect on the implications for editors and tools to support image selection.
△ Less
Submitted 12 March, 2024;
originally announced March 2024.
-
Know Your Audience: The benefits and pitfalls of generating plain language summaries beyond the "general" audience
Authors:
Tal August,
Kyle Lo,
Noah A. Smith,
Katharina Reinecke
Abstract:
Language models (LMs) show promise as tools for communicating science to the general public by simplifying and summarizing complex language. Because models can be prompted to generate text for a specific audience (e.g., college-educated adults), LMs might be used to create multiple versions of plain language summaries for people with different familiarities of scientific topics. However, it is not…
▽ More
Language models (LMs) show promise as tools for communicating science to the general public by simplifying and summarizing complex language. Because models can be prompted to generate text for a specific audience (e.g., college-educated adults), LMs might be used to create multiple versions of plain language summaries for people with different familiarities of scientific topics. However, it is not clear what the benefits and pitfalls of adaptive plain language are. When is simplifying necessary, what are the costs in doing so, and do these costs differ for readers with different background knowledge? Through three within-subjects studies in which we surface summaries for different envisioned audiences to participants of different backgrounds, we found that while simpler text led to the best reading experience for readers with little to no familiarity in a topic, high familiarity readers tended to ignore certain details in overly plain summaries (e.g., study limitations). Our work provides methods and guidance on ways of adapting plain language summaries beyond the single "general" audience.
△ Less
Submitted 7 March, 2024;
originally announced March 2024.
-
Culturally-Attuned Moral Machines: Implicit Learning of Human Value Systems by AI through Inverse Reinforcement Learning
Authors:
Nigini Oliveira,
Jasmine Li,
Koosha Khalvati,
Rodolfo Cortes Barragan,
Katharina Reinecke,
Andrew N. Meltzoff,
Rajesh P. N. Rao
Abstract:
Constructing a universal moral code for artificial intelligence (AI) is difficult or even impossible, given that different human cultures have different definitions of morality and different societal norms. We therefore argue that the value system of an AI should be culturally attuned: just as a child raised in a particular culture learns the specific values and norms of that culture, we propose t…
▽ More
Constructing a universal moral code for artificial intelligence (AI) is difficult or even impossible, given that different human cultures have different definitions of morality and different societal norms. We therefore argue that the value system of an AI should be culturally attuned: just as a child raised in a particular culture learns the specific values and norms of that culture, we propose that an AI agent operating in a particular human community should acquire that community's moral, ethical, and cultural codes. How AI systems might acquire such codes from human observation and interaction has remained an open question. Here, we propose using inverse reinforcement learning (IRL) as a method for AI agents to acquire a culturally-attuned value system implicitly. We test our approach using an experimental paradigm in which AI agents use IRL to learn different reward functions, which govern the agents' moral values, by observing the behavior of different cultural groups in an online virtual world requiring real-time decision making. We show that an AI agent learning from the average behavior of a particular cultural group can acquire altruistic characteristics reflective of that group's behavior, and this learned value system can generalize to new scenarios requiring altruistic judgments. Our results provide, to our knowledge, the first demonstration that AI agents could potentially be endowed with the ability to continually learn their values and norms from observing and interacting with humans, thereby becoming attuned to the culture they are operating in.
△ Less
Submitted 29 December, 2023;
originally announced December 2023.
-
The Case for Anticipating Undesirable Consequences of Computing Innovations Early, Often, and Across Computer Science
Authors:
Rock Yuren Pang,
Dan Grossman,
Tadayoshi Kohno,
Katharina Reinecke
Abstract:
From smart sensors that infringe on our privacy to neural nets that portray realistic imposter deepfakes, our society increasingly bears the burden of negative, if unintended, consequences of computing innovations. As the experts in the technology we create, Computer Science (CS) researchers must do better at anticipating and addressing these undesirable consequences proactively. Our prior work sh…
▽ More
From smart sensors that infringe on our privacy to neural nets that portray realistic imposter deepfakes, our society increasingly bears the burden of negative, if unintended, consequences of computing innovations. As the experts in the technology we create, Computer Science (CS) researchers must do better at anticipating and addressing these undesirable consequences proactively. Our prior work showed that many of us recognize the value of thinking preemptively about the perils our research can pose, yet we tend to address them only in hindsight. How can we change the culture in which considering undesirable consequences of digital technology is deemed as important, but is not commonly done?
△ Less
Submitted 8 September, 2023;
originally announced September 2023.
-
NLPositionality: Characterizing Design Biases of Datasets and Models
Authors:
Sebastin Santy,
Jenny T. Liang,
Ronan Le Bras,
Katharina Reinecke,
Maarten Sap
Abstract:
Design biases in NLP systems, such as performance differences for different populations, often stem from their creator's positionality, i.e., views and lived experiences shaped by identity and background. Despite the prevalence and risks of design biases, they are hard to quantify because researcher, system, and dataset positionality is often unobserved. We introduce NLPositionality, a framework f…
▽ More
Design biases in NLP systems, such as performance differences for different populations, often stem from their creator's positionality, i.e., views and lived experiences shaped by identity and background. Despite the prevalence and risks of design biases, they are hard to quantify because researcher, system, and dataset positionality is often unobserved. We introduce NLPositionality, a framework for characterizing design biases and quantifying the positionality of NLP datasets and models. Our framework continuously collects annotations from a diverse pool of volunteer participants on LabintheWild, and statistically quantifies alignment with dataset labels and model predictions. We apply NLPositionality to existing datasets and models for two tasks -- social acceptability and hate speech detection. To date, we have collected 16,299 annotations in over a year for 600 instances from 1,096 annotators across 87 countries. We find that datasets and models align predominantly with Western, White, college-educated, and younger populations. Additionally, certain groups, such as non-binary people and non-native English speakers, are further marginalized by datasets and models as they rank least in alignment across all tasks. Finally, we draw from prior literature to discuss how researchers can examine their own positionality and that of their datasets and models, opening the door for more inclusive NLP systems.
△ Less
Submitted 2 June, 2023;
originally announced June 2023.
-
Anticipating Unintended Consequences of Technology Using Insights from Creativity Support Tools
Authors:
Rock Yuren Pang,
Katharina Reinecke
Abstract:
Our society has been increasingly witnessing a number of negative, unintended consequences of digital technologies. While post-hoc policy regulation is crucial in addressing these issues, reasonably anticipating the consequences before deploying technology can help mitigate potential harm to society in the first place. Yet, the quest to anticipate potential harms can be difficult without seeing di…
▽ More
Our society has been increasingly witnessing a number of negative, unintended consequences of digital technologies. While post-hoc policy regulation is crucial in addressing these issues, reasonably anticipating the consequences before deploying technology can help mitigate potential harm to society in the first place. Yet, the quest to anticipate potential harms can be difficult without seeing digital technologies deployed in the real world. In this position paper, we argue that anticipating unintended consequences of technology can be facilitated through creativity-enhancing interventions, such as by building on existing knowledge and insights from diverse stakeholders. Using lessons learned from prior work on creativity-support tools, the HCI community is uniquely equipped to design novel systems that aid in anticipating negative unintended consequences of technology on society.
△ Less
Submitted 12 April, 2023;
originally announced April 2023.
-
"That's important, but...": How Computer Science Researchers Anticipate Unintended Consequences of Their Research Innovations
Authors:
Kimberly Do,
Rock Yuren Pang,
Jiachen Jiang,
Katharina Reinecke
Abstract:
Computer science research has led to many breakthrough innovations but has also been scrutinized for enabling technology that has negative, unintended consequences for society. Given the increasing discussions of ethics in the news and among researchers, we interviewed 20 researchers in various CS sub-disciplines to identify whether and how they consider potential unintended consequences of their…
▽ More
Computer science research has led to many breakthrough innovations but has also been scrutinized for enabling technology that has negative, unintended consequences for society. Given the increasing discussions of ethics in the news and among researchers, we interviewed 20 researchers in various CS sub-disciplines to identify whether and how they consider potential unintended consequences of their research innovations. We show that considering unintended consequences is generally seen as important but rarely practiced. Principal barriers are a lack of formal process and strategy as well as the academic practice that prioritizes fast progress and publications. Drawing on these findings, we discuss approaches to support researchers in routinely considering unintended consequences, from bringing diverse perspectives through community participation to increasing incentives to investigate potential consequences. We intend for our work to pave the way for routine explorations of the societal implications of technological innovations before, during, and after the research process.
△ Less
Submitted 27 March, 2023;
originally announced March 2023.
-
Gendered Mental Health Stigma in Masked Language Models
Authors:
Inna Wanyin Lin,
Lucille Njoo,
Anjalie Field,
Ashish Sharma,
Katharina Reinecke,
Tim Althoff,
Yulia Tsvetkov
Abstract:
Mental health stigma prevents many individuals from receiving the appropriate care, and social psychology studies have shown that mental health tends to be overlooked in men. In this work, we investigate gendered mental health stigma in masked language models. In doing so, we operationalize mental health stigma by develo** a framework grounded in psychology research: we use clinical psychology l…
▽ More
Mental health stigma prevents many individuals from receiving the appropriate care, and social psychology studies have shown that mental health tends to be overlooked in men. In this work, we investigate gendered mental health stigma in masked language models. In doing so, we operationalize mental health stigma by develo** a framework grounded in psychology research: we use clinical psychology literature to curate prompts, then evaluate the models' propensity to generate gendered words. We find that masked language models capture societal stigma about gender in mental health: models are consistently more likely to predict female subjects than male in sentences about having a mental health condition (32% vs. 19%), and this disparity is exacerbated for sentences that indicate treatment-seeking behavior. Furthermore, we find that different models capture dimensions of stigma differently for men and women, associating stereotypes like anger, blame, and pity more with women with mental health conditions than with men. In showing the complex nuances of models' gendered mental health stigma, we demonstrate that context and overlap** dimensions of identity are important considerations when assessing computational models' social biases.
△ Less
Submitted 11 April, 2023; v1 submitted 26 October, 2022;
originally announced October 2022.
-
Tea: A High-level Language and Runtime System for Automating Statistical Analysis
Authors:
Eunice Jun,
Maureen Daum,
Jared Roesch,
Sarah E. Chasins,
Emery D. Berger,
Rene Just,
Katharina Reinecke
Abstract:
Though statistical analyses are centered on research questions and hypotheses, current statistical analysis tools are not. Users must first translate their hypotheses into specific statistical tests and then perform API calls with functions and parameters. To do so accurately requires that users have statistical expertise. To lower this barrier to valid, replicable statistical analysis, we introdu…
▽ More
Though statistical analyses are centered on research questions and hypotheses, current statistical analysis tools are not. Users must first translate their hypotheses into specific statistical tests and then perform API calls with functions and parameters. To do so accurately requires that users have statistical expertise. To lower this barrier to valid, replicable statistical analysis, we introduce Tea, a high-level declarative language and runtime system. In Tea, users express their study design, any parametric assumptions, and their hypotheses. Tea compiles these high-level specifications into a constraint satisfaction problem that determines the set of valid statistical tests, and then executes them to test the hypothesis. We evaluate Tea using a suite of statistical analyses drawn from popular tutorials. We show that Tea generally matches the choices of experts while automatically switching to non-parametric tests when parametric assumptions are not met. We simulate the effect of mistakes made by non-expert users and show that Tea automatically avoids both false negatives and false positives that could be produced by the application of incorrect statistical tests.
△ Less
Submitted 10 April, 2019;
originally announced April 2019.
