-
Leveraging Machine-Generated Rationales to Facilitate Social Meaning Detection in Conversations
Authors:
Ritam Dutt,
Zhen Wu,
Kelly Shi,
Divyanshu Sheth,
Prakhar Gupta,
Carolyn Penstein Rose
Abstract:
We present a generalizable classification approach that leverages Large Language Models (LLMs) to facilitate the detection of implicitly encoded social meaning in conversations. We design a multi-faceted prompt to extract a textual explanation of the reasoning that connects visible cues to underlying social meanings. These extracted explanations or rationales serve as augmentations to the conversa…
▽ More
We present a generalizable classification approach that leverages Large Language Models (LLMs) to facilitate the detection of implicitly encoded social meaning in conversations. We design a multi-faceted prompt to extract a textual explanation of the reasoning that connects visible cues to underlying social meanings. These extracted explanations or rationales serve as augmentations to the conversational text to facilitate dialogue understanding and transfer. Our empirical results over 2,340 experimental settings demonstrate the significant positive impact of adding these rationales. Our findings hold true for in-domain classification, zero-shot, and few-shot domain transfer for two different social meaning detection tasks, each spanning two different corpora.
△ Less
Submitted 27 June, 2024;
originally announced June 2024.
-
Generating Situated Reflection Triggers about Alternative Solution Paths: A Case Study of Generative AI for Computer-Supported Collaborative Learning
Authors:
Atharva Naik,
Jessica Ruhan Yin,
Anusha Kamath,
Qianou Ma,
Sherry Tongshuang Wu,
Charles Murray,
Christopher Bogart,
Majd Sakr,
Carolyn P. Rose
Abstract:
An advantage of Large Language Models (LLMs) is their contextualization capability - providing different responses based on student inputs like solution strategy or prior discussion, to potentially better engage students than standard feedback. We present a design and evaluation of a proof-of-concept LLM application to offer students dynamic and contextualized feedback. Specifically, we augment an…
▽ More
An advantage of Large Language Models (LLMs) is their contextualization capability - providing different responses based on student inputs like solution strategy or prior discussion, to potentially better engage students than standard feedback. We present a design and evaluation of a proof-of-concept LLM application to offer students dynamic and contextualized feedback. Specifically, we augment an Online Programming Exercise bot for a college-level Cloud Computing course with ChatGPT, which offers students contextualized reflection triggers during a collaborative query optimization task in database design. We demonstrate that LLMs can be used to generate highly situated reflection triggers that incorporate details of the collaborative discussion happening in context. We discuss in depth the exploration of the design space of the triggers and their correspondence with the learning objectives as well as the impact on student learning in a pilot study with 34 students.
△ Less
Submitted 28 April, 2024;
originally announced April 2024.
-
Making Sense of Machine Learning: Integrating Youth's Conceptual, Creative, and Critical Understandings of AI
Authors:
Luis Morales-Navarro,
Yasmin B. Kafai,
Francisco Castro,
William Payne,
Kayla DesPortes,
Daniella DiPaola,
Randi Williams,
Safinah Ali,
Cynthia Breazeal,
Clifford Lee,
Elisabeth Soep,
Duri Long,
Brian Magerko,
Jaemarie Solyst,
Amy Ogan,
Cansu Tatar,
Shiyan Jiang,
Jie Chao,
Carolyn P. Rosé,
Sepehr Vakil
Abstract:
Understanding how youth make sense of machine learning and how learning about machine learning can be supported in and out of school is more relevant than ever before as young people interact with machine learning powered applications everyday; while connecting with friends, listening to music, playing games, or attending school. In this symposium, we present different perspectives on understandin…
▽ More
Understanding how youth make sense of machine learning and how learning about machine learning can be supported in and out of school is more relevant than ever before as young people interact with machine learning powered applications everyday; while connecting with friends, listening to music, playing games, or attending school. In this symposium, we present different perspectives on understanding how learners make sense of machine learning in their everyday lives, how sensemaking of machine learning can be supported in and out of school through the construction of applications, and how youth critically evaluate machine learning powered systems. We discuss how sensemaking of machine learning applications involves the development and integration of conceptual, creative, and critical understandings that are increasingly important to prepare youth to participate in the world.
△ Less
Submitted 4 May, 2023;
originally announced May 2023.
-
Localize, Group, and Select: Boosting Text-VQA by Scene Text Modeling
Authors:
Xiaopeng Lu,
Zhen Fan,
Yansen Wang,
Jean Oh,
Carolyn P. Rose
Abstract:
As an important task in multimodal context understanding, Text-VQA (Visual Question Answering) aims at question answering through reading text information in images. It differentiates from the original VQA task as Text-VQA requires large amounts of scene-text relationship understanding, in addition to the cross-modal grounding capability. In this paper, we propose Localize, Group, and Select (LOGO…
▽ More
As an important task in multimodal context understanding, Text-VQA (Visual Question Answering) aims at question answering through reading text information in images. It differentiates from the original VQA task as Text-VQA requires large amounts of scene-text relationship understanding, in addition to the cross-modal grounding capability. In this paper, we propose Localize, Group, and Select (LOGOS), a novel model which attempts to tackle this problem from multiple aspects. LOGOS leverages two grounding tasks to better localize the key information of the image, utilizes scene text clustering to group individual OCR tokens, and learns to select the best answer from different sources of OCR (Optical Character Recognition) texts. Experiments show that LOGOS outperforms previous state-of-the-art methods on two Text-VQA benchmarks without using additional OCR annotation data. Ablation studies and analysis demonstrate the capability of LOGOS to bridge different modalities and better understand scene text.
△ Less
Submitted 19 August, 2021;
originally announced August 2021.
-
Robust Knowledge Graph Completion with Stacked Convolutions and a Student Re-Ranking Network
Authors:
Justin Lovelace,
Denis Newman-Griffis,
Shikhar Vashishth,
Jill Fain Lehman,
Carolyn Penstein Rosé
Abstract:
Knowledge Graph (KG) completion research usually focuses on densely connected benchmark datasets that are not representative of real KGs. We curate two KG datasets that include biomedical and encyclopedic knowledge and use an existing commonsense KG dataset to explore KG completion in the more realistic setting where dense connectivity is not guaranteed. We develop a deep convolutional network tha…
▽ More
Knowledge Graph (KG) completion research usually focuses on densely connected benchmark datasets that are not representative of real KGs. We curate two KG datasets that include biomedical and encyclopedic knowledge and use an existing commonsense KG dataset to explore KG completion in the more realistic setting where dense connectivity is not guaranteed. We develop a deep convolutional network that utilizes textual entity representations and demonstrate that our model outperforms recent KG completion methods in this challenging setting. We find that our model's performance improvements stem primarily from its robustness to sparsity. We then distill the knowledge from the convolutional network into a student network that re-ranks promising candidate entities. This re-ranking stage leads to further improvements in performance and demonstrates the effectiveness of entity re-ranking for KG completion.
△ Less
Submitted 11 June, 2021;
originally announced June 2021.
-
RESPER: Computationally Modelling Resisting Strategies in Persuasive Conversations
Authors:
Ritam Dutt,
Sayan Sinha,
Rishabh Joshi,
Surya Shekhar Chakraborty,
Meredith Riggs,
Xinru Yan,
Haogang Bao,
Carolyn Penstein Rosé
Abstract:
Modelling persuasion strategies as predictors of task outcome has several real-world applications and has received considerable attention from the computational linguistics community. However, previous research has failed to account for the resisting strategies employed by an individual to foil such persuasion attempts. Grounded in prior literature in cognitive and social psychology, we propose a…
▽ More
Modelling persuasion strategies as predictors of task outcome has several real-world applications and has received considerable attention from the computational linguistics community. However, previous research has failed to account for the resisting strategies employed by an individual to foil such persuasion attempts. Grounded in prior literature in cognitive and social psychology, we propose a generalised framework for identifying resisting strategies in persuasive conversations. We instantiate our framework on two distinct datasets comprising persuasion and negotiation conversations. We also leverage a hierarchical sequence-labelling neural architecture to infer the aforementioned resisting strategies automatically. Our experiments reveal the asymmetry of power roles in non-collaborative goal-directed conversations and the benefits accrued from incorporating resisting strategies on the final conversation outcome. We also investigate the role of different resisting strategies on the conversation outcome and glean insights that corroborate with past findings. We also make the code and the dataset of this work publicly available at https://github.com/americast/resper.
△ Less
Submitted 25 January, 2021;
originally announced January 2021.
-
Kee** Up Appearances: Computational Modeling of Face Acts in Persuasion Oriented Discussions
Authors:
Ritam Dutt,
Rishabh Joshi,
Carolyn Penstein Rose
Abstract:
The notion of face refers to the public self-image of an individual that emerges both from the individual's own actions as well as from the interaction with others. Modeling face and understanding its state changes throughout a conversation is critical to the study of maintenance of basic human needs in and through interaction. Grounded in the politeness theory of Brown and Levinson (1978), we pro…
▽ More
The notion of face refers to the public self-image of an individual that emerges both from the individual's own actions as well as from the interaction with others. Modeling face and understanding its state changes throughout a conversation is critical to the study of maintenance of basic human needs in and through interaction. Grounded in the politeness theory of Brown and Levinson (1978), we propose a generalized framework for modeling face acts in persuasion conversations, resulting in a reliable coding manual, an annotated corpus, and computational models. The framework reveals insights about differences in face act utilization between asymmetric roles in persuasion conversations. Using computational models, we are able to successfully identify face acts as well as predict a key conversational outcome (e.g. donation success). Finally, we model a latent representation of the conversational state to analyze the impact of predicted face acts on the probability of a positive conversational outcome and observe several correlations that corroborate previous findings.
△ Less
Submitted 23 September, 2020; v1 submitted 22 September, 2020;
originally announced September 2020.
-
A Machine Learning Framework for Authorship Identification From Texts
Authors:
Rahul Radhakrishnan Iyer,
Carolyn Penstein Rose
Abstract:
Authorship identification is a process in which the author of a text is identified. Most known literary texts can easily be attributed to a certain author because they are, for example, signed. Yet sometimes we find unfinished pieces of work or a whole bunch of manuscripts with a wide variety of possible authors. In order to assess the importance of such a manuscript, it is vital to know who wrote…
▽ More
Authorship identification is a process in which the author of a text is identified. Most known literary texts can easily be attributed to a certain author because they are, for example, signed. Yet sometimes we find unfinished pieces of work or a whole bunch of manuscripts with a wide variety of possible authors. In order to assess the importance of such a manuscript, it is vital to know who wrote it. In this work, we aim to develop a machine learning framework to effectively determine authorship. We formulate the task as a single-label multi-class text categorization problem and propose a supervised machine learning framework incorporating stylometric features. This task is highly interdisciplinary in that it takes advantage of machine learning, information retrieval, and natural language processing. We present an approach and a model which learns the differences in writing style between $50$ different authors and is able to predict the author of a new text with high accuracy. The accuracy is seen to increase significantly after introducing certain linguistic stylometric features along with text features.
△ Less
Submitted 21 December, 2019;
originally announced December 2019.
-
Time-series Insights into the Process of Passing or Failing Online University Courses using Neural-Induced Interpretable Student States
Authors:
Byungsoo Jeon,
Eyal Shafran,
Luke Breitfeller,
Jason Levin,
Carolyn P. Rose
Abstract:
This paper addresses a key challenge in Educational Data Mining, namely to model student behavioral trajectories in order to provide a means for identifying students most at-risk, with the goal of providing supportive interventions. While many forms of data including clickstream data or data from sensors have been used extensively in time series models for such purposes, in this paper we explore t…
▽ More
This paper addresses a key challenge in Educational Data Mining, namely to model student behavioral trajectories in order to provide a means for identifying students most at-risk, with the goal of providing supportive interventions. While many forms of data including clickstream data or data from sensors have been used extensively in time series models for such purposes, in this paper we explore the use of textual data, which is sometimes available in the records of students at large, online universities. We propose a time series model that constructs an evolving student state representation using both clickstream data and a signal extracted from the textual notes recorded by human mentors assigned to each student. We explore how the addition of this textual data improves both the predictive power of student states for the purpose of identifying students at risk for course failure as well as for providing interpretable insights about student course engagement processes.
△ Less
Submitted 1 May, 2019;
originally announced May 2019.
-
Combining Model-Free Q-Ensembles and Model-Based Approaches for Informed Exploration
Authors:
Sreecharan Sankaranarayanan,
Raghuram Mandyam Annasamy,
Katia Sycara,
Carolyn Penstein Rosé
Abstract:
Q-Ensembles are a model-free approach where input images are fed into different Q-networks and exploration is driven by the assumption that uncertainty is proportional to the variance of the output Q-values obtained. They have been shown to perform relatively well compared to other exploration strategies. Further, model-based approaches, such as encoder-decoder models have been used successfully f…
▽ More
Q-Ensembles are a model-free approach where input images are fed into different Q-networks and exploration is driven by the assumption that uncertainty is proportional to the variance of the output Q-values obtained. They have been shown to perform relatively well compared to other exploration strategies. Further, model-based approaches, such as encoder-decoder models have been used successfully for next frame prediction given previous frames. This paper proposes to integrate the model-free Q-ensembles and model-based approaches with the hope of compounding the benefits of both and achieving superior exploration as a result. Results show that a model-based trajectory memory approach when combined with Q-ensembles produces superior performance when compared to only using Q-ensembles.
△ Less
Submitted 12 June, 2018;
originally announced June 2018.
-
Attentive Interaction Model: Modeling Changes in View in Argumentation
Authors:
Yohan Jo,
Shivani Poddar,
Byungsoo Jeon,
Qinlan Shen,
Carolyn P. Rose,
Graham Neubig
Abstract:
We present a neural architecture for modeling argumentative dialogue that explicitly models the interplay between an Opinion Holder's (OH's) reasoning and a challenger's argument, with the goal of predicting if the argument successfully changes the OH's view. The model has two components: (1) vulnerable region detection, an attention model that identifies parts of the OH's reasoning that are amena…
▽ More
We present a neural architecture for modeling argumentative dialogue that explicitly models the interplay between an Opinion Holder's (OH's) reasoning and a challenger's argument, with the goal of predicting if the argument successfully changes the OH's view. The model has two components: (1) vulnerable region detection, an attention model that identifies parts of the OH's reasoning that are amenable to change, and (2) interaction encoding, which identifies the relationship between the content of the OH's reasoning and that of the challenger's argument. Based on evaluation on discussions from the Change My View forum on Reddit, the two components work together to predict an OH's change in view, outperforming several baselines. A posthoc analysis suggests that sentences picked out by the attention model are addressed more frequently by successful arguments than by unsuccessful ones.
△ Less
Submitted 18 April, 2018; v1 submitted 30 March, 2018;
originally announced April 2018.
-
Linguistic Markers of Influence in Informal Interactions
Authors:
Shrimai Prabhumoye,
Samridhi Choudhary,
Evangelia Spiliopoulou,
Christopher Bogart,
Carolyn Penstein Rose,
Alan W Black
Abstract:
There has been a long standing interest in understanding `Social Influence' both in Social Sciences and in Computational Linguistics. In this paper, we present a novel approach to study and measure interpersonal influence in daily interactions. Motivated by the basic principles of influence, we attempt to identify indicative linguistic features of the posts in an online knitting community. We pres…
▽ More
There has been a long standing interest in understanding `Social Influence' both in Social Sciences and in Computational Linguistics. In this paper, we present a novel approach to study and measure interpersonal influence in daily interactions. Motivated by the basic principles of influence, we attempt to identify indicative linguistic features of the posts in an online knitting community. We present the scheme used to operationalize and label the posts with indicator features. Experiments with the identified features show an improvement in the classification accuracy of influence by 3.15%. Our results illustrate the important correlation between the characteristics of the language and its potential to influence others.
△ Less
Submitted 14 July, 2017;
originally announced July 2017.
-
Coordinating Collaborative Chat in Massive Open Online Courses
Authors:
Gaurav Singh Tomar,
Sreecharan Sankaranarayanan,
Xu Wang,
Carolyn Penstein Rosé
Abstract:
An earlier study of a collaborative chat intervention in a Massive Open Online Course (MOOC) identified negative effects on attrition stemming from a requirement for students to be matched with exactly one partner prior to beginning the activity. That study raised questions about how to orchestrate a collaborative chat intervention in a MOOC context in order to provide the benefit of synchronous s…
▽ More
An earlier study of a collaborative chat intervention in a Massive Open Online Course (MOOC) identified negative effects on attrition stemming from a requirement for students to be matched with exactly one partner prior to beginning the activity. That study raised questions about how to orchestrate a collaborative chat intervention in a MOOC context in order to provide the benefit of synchronous social engagement without the coordination difficulties. In this paper we present a careful analysis of an intervention designed to overcome coordination difficulties by welcoming students into the chat on a rolling basis as they arrive rather than requiring them to be matched with a partner before beginning. The results suggest the most positive impact when experiencing a chat with exactly one partner rather than more or less. A qualitative analysis of the chat data reveals differential experiences between these configurations that suggests a potential explanation for the effect and raises questions for future research.
△ Less
Submitted 18 April, 2017;
originally announced April 2017.
-
Expediting Support for Social Learning with Behavior Modeling
Authors:
Yohan Jo,
Gaurav Tomar,
Oliver Ferschke,
Carolyn P. Rose,
Dragan Gasevic
Abstract:
An important research problem for Educational Data Mining is to expedite the cycle of data leading to the analysis of student learning processes and the improvement of support for those processes. For this goal in the context of social interaction in learning, we propose a three-part pipeline that includes data infrastructure, learning process analysis with behavior modeling, and intervention for…
▽ More
An important research problem for Educational Data Mining is to expedite the cycle of data leading to the analysis of student learning processes and the improvement of support for those processes. For this goal in the context of social interaction in learning, we propose a three-part pipeline that includes data infrastructure, learning process analysis with behavior modeling, and intervention for support. We also describe an application of the pipeline to data from a social learning platform to investigate appropriate goal-setting behavior as a qualification of role models. Students following appropriate goal setters persisted longer in the course, showed increased engagement in hands-on course activities, and were more likely to review previously covered materials as they continued through the course. To foster this beneficial social interaction among students, we propose a social recommender system and show potential for assisting students in interacting with qualified goal setters as role models. We discuss how this generalizable pipeline can be adapted for other support needs in online learning settings.
△ Less
Submitted 13 July, 2016; v1 submitted 9 May, 2016;
originally announced May 2016.
-
Scalable Modeling of Conversational-role based Self-presentation Characteristics in Large Online Forums
Authors:
Abhimanu Kumar,
Shriphani Palakodety,
Chong Wang,
Carolyn P. Rose,
Eric P. Xing,
Miaomiao Wen
Abstract:
Online discussion forums are complex webs of overlap** subcommunities (macrolevel structure, across threads) in which users enact different roles depending on which subcommunity they are participating in within a particular time point (microlevel structure, within threads). This sub-network structure is implicit in massive collections of threads. To uncover this structure, we develop a scalable…
▽ More
Online discussion forums are complex webs of overlap** subcommunities (macrolevel structure, across threads) in which users enact different roles depending on which subcommunity they are participating in within a particular time point (microlevel structure, within threads). This sub-network structure is implicit in massive collections of threads. To uncover this structure, we develop a scalable algorithm based on stochastic variational inference and leverage topic models (LDA) along with mixed membership stochastic block (MMSB) models. We evaluate our model on three large-scale datasets, Cancer-ThreadStarter (22K users and 14.4K threads), Cancer-NameMention(15.1K users and 12.4K threads) and StackOverFlow (1.19 million users and 4.55 million threads). Qualitatively, we demonstrate that our model can provide useful explanations of microlevel and macrolevel user presentation characteristics in different communities using the topics discovered from posts. Quantitatively, we show that our model does better than MMSB and LDA in predicting user reply structure within threads. In addition, we demonstrate via synthetic data experiments that the proposed active sub-network discovery model is stable and recovers the original parameters of the experimental setup with high probability.
△ Less
Submitted 10 December, 2015;
originally announced December 2015.
-
Computational Sociolinguistics: A Survey
Authors:
Dong Nguyen,
A. Seza Doğruöz,
Carolyn P. Rosé,
Franciska de Jong
Abstract:
Language is a social phenomenon and variation is inherent to its social nature. Recently, there has been a surge of interest within the computational linguistics (CL) community in the social dimension of language. In this article we present a survey of the emerging field of "Computational Sociolinguistics" that reflects this increased interest. We aim to provide a comprehensive overview of CL rese…
▽ More
Language is a social phenomenon and variation is inherent to its social nature. Recently, there has been a surge of interest within the computational linguistics (CL) community in the social dimension of language. In this article we present a survey of the emerging field of "Computational Sociolinguistics" that reflects this increased interest. We aim to provide a comprehensive overview of CL research on sociolinguistic themes, featuring topics such as the relation between language and social identity, language use in social interaction and multilingual communication. Moreover, we demonstrate the potential for synergy between the research communities involved, by showing how the large-scale data-driven methods that are widely used in CL can complement existing sociolinguistic studies, and how sociolinguistics can inform and challenge the methods and assumptions employed in CL studies. We hope to convey the possible benefits of a closer collaboration between the two communities and conclude with a discussion of open challenges.
△ Less
Submitted 6 April, 2016; v1 submitted 30 August, 2015;
originally announced August 2015.
