-
Causal Inference in Natural Language Processing: Estimation, Prediction, Interpretation and Beyond
Authors:
Amir Feder,
Katherine A. Keith,
Emaad Manzoor,
Reid Pryzant,
Dhanya Sridhar,
Zach Wood-Doughty,
Jacob Eisenstein,
Justin Grimmer,
Roi Reichart,
Margaret E. Roberts,
Brandon M. Stewart,
Victor Veitch,
Diyi Yang
Abstract:
A fundamental goal of scientific research is to learn about causal relationships. However, despite its critical role in the life and social sciences, causality has not had the same importance in Natural Language Processing (NLP), which has traditionally placed more emphasis on predictive tasks. This distinction is beginning to fade, with an emerging area of interdisciplinary research at the conver…
▽ More
A fundamental goal of scientific research is to learn about causal relationships. However, despite its critical role in the life and social sciences, causality has not had the same importance in Natural Language Processing (NLP), which has traditionally placed more emphasis on predictive tasks. This distinction is beginning to fade, with an emerging area of interdisciplinary research at the convergence of causal inference and language processing. Still, research on causality in NLP remains scattered across domains without unified definitions, benchmark datasets and clear articulations of the challenges and opportunities in the application of causal inference to the textual domain, with its unique properties. In this survey, we consolidate research across academic areas and situate it in the broader NLP landscape. We introduce the statistical challenge of estimating causal effects with text, encompassing settings where text is used as an outcome, treatment, or to address confounding. In addition, we explore potential uses of causal inference to improve the robustness, fairness, and interpretability of NLP models. We thus provide a unified overview of causal inference for the NLP community.
△ Less
Submitted 30 July, 2022; v1 submitted 2 September, 2021;
originally announced September 2021.
-
Censorship of Online Encyclopedias: Implications for NLP Models
Authors:
Eddie Yang,
Margaret E. Roberts
Abstract:
While artificial intelligence provides the backbone for many tools people use around the world, recent work has brought to attention that the algorithms powering AI are not free of politics, stereotypes, and bias. While most work in this area has focused on the ways in which AI can exacerbate existing inequalities and discrimination, very little work has studied how governments actively shape trai…
▽ More
While artificial intelligence provides the backbone for many tools people use around the world, recent work has brought to attention that the algorithms powering AI are not free of politics, stereotypes, and bias. While most work in this area has focused on the ways in which AI can exacerbate existing inequalities and discrimination, very little work has studied how governments actively shape training data. We describe how censorship has affected the development of Wikipedia corpuses, text data which are regularly used for pre-trained inputs into NLP algorithms. We show that word embeddings trained on Baidu Baike, an online Chinese encyclopedia, have very different associations between adjectives and a range of concepts about democracy, freedom, collective action, equality, and people and historical events in China than its regularly blocked but uncensored counterpart - Chinese language Wikipedia. We examine the implications of these discrepancies by studying their use in downstream AI applications. Our paper shows how government repression, censorship, and self-censorship may impact training data and the applications that draw from them.
△ Less
Submitted 22 January, 2021;
originally announced January 2021.
-
On Constructing a Knowledge Base of Chinese Criminal Cases
Authors:
Xiaohan Wu,
Benjamin L. Liebman,
Rachel E. Stern,
Margaret E. Roberts,
Amarnath Gupta
Abstract:
We are develo** a knowledge base over Chinese judicial decision documents to facilitate landscape analyses of Chinese Criminal Cases. We view judicial decision documents as a mixed-granularity semi-structured text where different levels of the text carry different semantic constructs and entailments. We use a combination of context-sensitive grammar, dependency parsing and discourse analysis to…
▽ More
We are develo** a knowledge base over Chinese judicial decision documents to facilitate landscape analyses of Chinese Criminal Cases. We view judicial decision documents as a mixed-granularity semi-structured text where different levels of the text carry different semantic constructs and entailments. We use a combination of context-sensitive grammar, dependency parsing and discourse analysis to extract a formal and interpretable representation of these documents. Our knowledge base is developed by constructing associations between different elements of these documents. The interpretability is contributed in part by our formal representation of the Chinese criminal laws, also as semi-structured documents. The landscape analyses utilize these two representations and enable a law researcher to ask legal pattern analysis queries.
△ Less
Submitted 14 October, 2019;
originally announced October 2019.
-
How to Make Causal Inferences Using Texts
Authors:
Naoki Egami,
Christian J. Fong,
Justin Grimmer,
Margaret E. Roberts,
Brandon M. Stewart
Abstract:
New text as data techniques offer a great promise: the ability to inductively discover measures that are useful for testing social science theories of interest from large collections of text. We introduce a conceptual framework for making causal inferences with discovered measures as a treatment or outcome. Our framework enables researchers to discover high-dimensional textual interventions and es…
▽ More
New text as data techniques offer a great promise: the ability to inductively discover measures that are useful for testing social science theories of interest from large collections of text. We introduce a conceptual framework for making causal inferences with discovered measures as a treatment or outcome. Our framework enables researchers to discover high-dimensional textual interventions and estimate the ways that observed treatments affect text-based outcomes. We argue that nearly all text-based causal inferences depend upon a latent representation of the text and we provide a framework to learn the latent representation. But estimating this latent representation, we show, creates new risks: we may introduce an identification problem or overfit. To address these risks we describe a split-sample framework and apply it to estimate causal effects from an experiment on immigration attitudes and a study on bureaucratic response. Our work provides a rigorous foundation for text-based causal inferences.
△ Less
Submitted 6 February, 2018;
originally announced February 2018.
-
On the influence of social bots in online protests. Preliminary findings of a Mexican case study
Authors:
Pablo Suárez-Serrato,
Margaret E. Roberts,
Clayton A. Davis,
Filippo Menczer
Abstract:
Social bots can affect online communication among humans. We study this phenomenon by focusing on #YaMeCanse, the most active protest hashtag in the history of Twitter in Mexico. Accounts using the hashtag are classified using the BotOrNot bot detection tool. Our preliminary analysis suggests that bots played a critical role in disrupting online communication about the protest movement.
Social bots can affect online communication among humans. We study this phenomenon by focusing on #YaMeCanse, the most active protest hashtag in the history of Twitter in Mexico. Accounts using the hashtag are classified using the BotOrNot bot detection tool. Our preliminary analysis suggests that bots played a critical role in disrupting online communication about the protest movement.
△ Less
Submitted 26 September, 2016;
originally announced September 2016.
-
Group Foraging in Dynamic Environments
Authors:
Michael E. Roberts,
Sam Cheesman,
Patrick McMullen
Abstract:
Previous human foraging experiments have shown that human groups routinely undermatch environmental resources much like other animal species. In this experiment, we test whether humans also selectively rely on others as information sources when the environmental state is uncertain, and we also test whether overt signals of other foragers' success influences group matching behavior and group adapta…
▽ More
Previous human foraging experiments have shown that human groups routinely undermatch environmental resources much like other animal species. In this experiment, we test whether humans also selectively rely on others as information sources when the environmental state is uncertain, and we also test whether overt signals of other foragers' success influences group matching behavior and group adaptation to a changing environment. The results show evidence of reliance on social information in specific conditions, but participants were primarily influenced by their individual assessments of food location rather than the success of other foragers.
△ Less
Submitted 16 April, 2012;
originally announced April 2012.
-
Chandra Observations of G11.2-0.3: Implications for Pulsar Ages
Authors:
V. M. Kaspi,
M. E. Roberts,
G. Vasisht,
E. V. Gotthelf,
M. Pivovaroff,
N. Kawai
Abstract:
We present Chandra X-ray Observatory imaging observations of the young Galactic supernova remnant G11.2-0.3. The image shows that the previously known young 65-ms X-ray pulsar is at position (J2000) RA 18h 11m 29.22s, DEC -19o 25' 27.''6, with 1 sigma error radius 0.''6. This is within 8'' of the geometric center of the shell. This provides strong confirming evidence that the system is younger,…
▽ More
We present Chandra X-ray Observatory imaging observations of the young Galactic supernova remnant G11.2-0.3. The image shows that the previously known young 65-ms X-ray pulsar is at position (J2000) RA 18h 11m 29.22s, DEC -19o 25' 27.''6, with 1 sigma error radius 0.''6. This is within 8'' of the geometric center of the shell. This provides strong confirming evidence that the system is younger, by a factor of ~12, than the characteristic age of the pulsar. The age discrepancy suggests that pulsar characteristic ages can be poor age estimators for young pulsars. Assuming conventional spin down with constant magnetic field and braking index, the most likely explanation for the age discrepancy in G11.2-0.3 is that the pulsar was born with a spin period of ~62 ms. The Chandra image also reveals, for the first time, the morphology of the pulsar wind nebula. The elongated hard-X-ray structure can be interpreted as either a jet or a Crab-like torus seen edge on. This adds to the growing list of highly aspherical pulsar wind nebulae and argues that such structures are common around young pulsars.
△ Less
Submitted 16 July, 2001;
originally announced July 2001.
