-
Conversational Grounding: Annotation and Analysis of Grounding Acts and Grounding Units
Authors:
Biswesh Mohapatra,
Seemab Hassan,
Laurent Romary,
Justine Cassell
Abstract:
Successful conversations often rest on common understanding, where all parties are on the same page about the information being shared. This process, known as conversational grounding, is crucial for building trustworthy dialog systems that can accurately keep track of and recall the shared information. The proficiencies of an agent in grounding the conveyed information significantly contribute to…
▽ More
Successful conversations often rest on common understanding, where all parties are on the same page about the information being shared. This process, known as conversational grounding, is crucial for building trustworthy dialog systems that can accurately keep track of and recall the shared information. The proficiencies of an agent in grounding the conveyed information significantly contribute to building a reliable dialog system. Despite recent advancements in dialog systems, there exists a noticeable deficit in their grounding capabilities. Traum provided a framework for conversational grounding introducing Grounding Acts and Grounding Units, but substantial progress, especially in the realm of Large Language Models, remains lacking. To bridge this gap, we present the annotation of two dialog corpora employing Grounding Acts, Grounding Units, and a measure of their degree of grounding. We discuss our key findings during the annotation and also provide a baseline model to test the performance of current Language Models in categorizing the grounding acts of the dialogs. Our work aims to provide a useful resource for further research in making conversations with machines better understood and more reliable in natural day-to-day collaborative dialogs.
△ Less
Submitted 25 March, 2024;
originally announced March 2024.
-
When to generate hedges in peer-tutoring interactions
Authors:
Alafate Abulimiti,
Chloé Clavel,
Justine Cassell
Abstract:
This paper explores the application of machine learning techniques to predict where hedging occurs in peer-tutoring interactions. The study uses a naturalistic face-to-face dataset annotated for natural language turns, conversational strategies, tutoring strategies, and nonverbal behaviours. These elements are processed into a vector representation of the previous turns, which serves as input to s…
▽ More
This paper explores the application of machine learning techniques to predict where hedging occurs in peer-tutoring interactions. The study uses a naturalistic face-to-face dataset annotated for natural language turns, conversational strategies, tutoring strategies, and nonverbal behaviours. These elements are processed into a vector representation of the previous turns, which serves as input to several machine learning models. Results show that embedding layers, that capture the semantic information of the previous turns, significantly improves the model's performance. Additionally, the study provides insights into the importance of various features, such as interpersonal rapport and nonverbal behaviours, in predicting hedges by using Shapley values for feature explanation. We discover that the eye gaze of both the tutor and the tutee has a significant impact on hedge prediction. We further validate this observation through a follow-up ablation study.
△ Less
Submitted 28 July, 2023;
originally announced July 2023.
-
"You might think about slightly revising the title": identifying hedges in peer-tutoring interactions
Authors:
Yann Raphalen,
Chloé Clavel,
Justine Cassell
Abstract:
Hedges play an important role in the management of conversational interaction. In peer tutoring, they are notably used by tutors in dyads (pairs of interlocutors) experiencing low rapport to tone down the impact of instructions and negative feedback. Pursuing the objective of building a tutoring agent that manages rapport with students in order to improve learning, we used a multimodal peer-tutori…
▽ More
Hedges play an important role in the management of conversational interaction. In peer tutoring, they are notably used by tutors in dyads (pairs of interlocutors) experiencing low rapport to tone down the impact of instructions and negative feedback. Pursuing the objective of building a tutoring agent that manages rapport with students in order to improve learning, we used a multimodal peer-tutoring dataset to construct a computational framework for identifying hedges. We compared approaches relying on pre-trained resources with others that integrate insights from the social science literature. Our best performance involved a hybrid approach that outperforms the existing baseline while being easier to interpret. We employ a model explainability tool to explore the features that characterize hedges in peer-tutoring conversations, and we identify some novel features, and the benefits of such a hybrid model approach.
△ Less
Submitted 18 June, 2023;
originally announced June 2023.
-
How About Kind of Generating Hedges using End-to-End Neural Models?
Authors:
Alafate Abulimiti,
Chloé Clavel,
Justine Cassell
Abstract:
Hedging is a strategy for softening the impact of a statement in conversation. In reducing the strength of an expression, it may help to avoid embarrassment (more technically, ``face threat'') to one's listener. For this reason, it is often found in contexts of instruction, such as tutoring. In this work, we develop a model of hedge generation based on i) fine-tuning state-of-the-art language mode…
▽ More
Hedging is a strategy for softening the impact of a statement in conversation. In reducing the strength of an expression, it may help to avoid embarrassment (more technically, ``face threat'') to one's listener. For this reason, it is often found in contexts of instruction, such as tutoring. In this work, we develop a model of hedge generation based on i) fine-tuning state-of-the-art language models trained on human-human tutoring data, followed by ii) reranking to select the candidate that best matches the expected hedging strategy within a candidate pool using a hedge classifier. We apply this method to a natural peer-tutoring corpus containing a significant number of disfluencies, repetitions, and repairs. The results show that generation in this noisy environment is feasible with reranking. By conducting an error analysis for both approaches, we reveal the challenges faced by systems attempting to accomplish both social and task-oriented goals in conversation.
△ Less
Submitted 26 June, 2023;
originally announced June 2023.
-
A Novel Multimodal Approach for Studying the Dynamics of Curiosity in Small Group Learning
Authors:
Tanmay Sinha,
Zhen Bai,
Justine Cassell
Abstract:
Curiosity is a vital metacognitive skill in educational contexts, leading to creativity, and a love of learning. And while many school systems increasingly undercut curiosity by teaching to the test, teachers are increasingly interested in how to evoke curiosity in their students to prepare them for a world in which lifelong learning and reskilling will be more and more important. One aspect of cu…
▽ More
Curiosity is a vital metacognitive skill in educational contexts, leading to creativity, and a love of learning. And while many school systems increasingly undercut curiosity by teaching to the test, teachers are increasingly interested in how to evoke curiosity in their students to prepare them for a world in which lifelong learning and reskilling will be more and more important. One aspect of curiosity that has received little attention, however, is the role of peers in eliciting curiosity. We present what we believe to be the first theoretical framework that articulates an integrated socio-cognitive account of curiosity that ties observable behaviors in peers to underlying curiosity states. We make a bipartite distinction between individual and interpersonal functions that contribute to curiosity, and multimodal behaviors that fulfill these functions. We validate the proposed framework by leveraging a longitudinal latent variable modeling approach. Findings confirm a positive predictive relationship between the latent variables of individual and interpersonal functions and curiosity, with the interpersonal functions exercising a comparatively stronger influence. Prominent behavioral realizations of these functions are also discovered in a data-driven manner. We instantiate the proposed theoretical framework in a set of strategies and tactics that can be incorporated into learning technologies to indicate, evoke, and scaffold curiosity. This work is a step towards designing learning technologies that can recognize and evoke moment-by-moment curiosity during learning in social contexts and towards a more complete multimodal learning analytics. The underlying rationale is applicable more generally for develo** computer support for other metacognitive and socio-emotional skills.
△ Less
Submitted 1 April, 2022;
originally announced April 2022.
-
A survey of neural models for the automatic analysis of conversation: Towards a better integration of the social sciences
Authors:
Chloé Clavel,
Matthieu Labeau,
Justine Cassell
Abstract:
Some exciting new approaches to neural architectures for the analysis of conversation have been introduced over the past couple of years. These include neural architectures for detecting emotion, dialogue acts, and sentiment polarity. They take advantage of some of the key attributes of contemporary machine learning, such as recurrent neural networks with attention mechanisms and transformer-based…
▽ More
Some exciting new approaches to neural architectures for the analysis of conversation have been introduced over the past couple of years. These include neural architectures for detecting emotion, dialogue acts, and sentiment polarity. They take advantage of some of the key attributes of contemporary machine learning, such as recurrent neural networks with attention mechanisms and transformer-based approaches. However, while the architectures themselves are extremely promising, the phenomena they have been applied to to date are but a small part of what makes conversation engaging. In this paper we survey these neural architectures and what they have been applied to. On the basis of the social science literature, we then describe what we believe to be the most fundamental and definitional feature of conversation, which is its co-construction over time by two or more interlocutors. We discuss how neural architectures of the sort surveyed could profitably be applied to these more fundamental aspects of conversation, and what this buys us in terms of a better analysis of conversation and even, in the longer term, a better way of generating conversation for a conversational system.
△ Less
Submitted 31 March, 2022;
originally announced March 2022.
-
C-band single photons from a trapped ion via two-stage frequency conversion
Authors:
John Hannegan,
Uday Saha,
James D. Siverns,
Jake Cassell,
Edo Waks,
Qudsia Quraishi
Abstract:
Fiber-based quantum networks require photons at telecommunications wavelengths to interconnect qubits separated by long distances. Trapped ions are leading candidates for quantum networking with high-fidelity two-qubit gates, long coherence times, and the ability to readily emit photons entangled with the ion's internal qubit states. However, trapped ions typically emit photons at wavelengths inco…
▽ More
Fiber-based quantum networks require photons at telecommunications wavelengths to interconnect qubits separated by long distances. Trapped ions are leading candidates for quantum networking with high-fidelity two-qubit gates, long coherence times, and the ability to readily emit photons entangled with the ion's internal qubit states. However, trapped ions typically emit photons at wavelengths incompatible with telecommunications fiber. Here, we demonstrate frequency conversion of visible photons, emitted from the S-P dipole transition of a trapped Ba+ ion, into the telecommunications C-band. These results are an important step towards enabling a long-distance trapped ion quantum internet.
△ Less
Submitted 28 February, 2022; v1 submitted 30 March, 2021;
originally announced March 2021.
-
Improving entanglement generation rates in trapped ion quantum networks using nondestructive photon measurement and storage
Authors:
John Hannegan,
James D. Siverns,
Jake Cassell,
Qudsia Quraishi
Abstract:
Long range quantum information processing will require the integration of different technologies to form hybrid architectures combining the strengths of multiple quantum systems. In this work, we propose a hybrid networking architecture designed to improve entanglement rates in quantum networks based on trapped ions. Trapped ions are excellent candidates as network nodes but photon losses make lon…
▽ More
Long range quantum information processing will require the integration of different technologies to form hybrid architectures combining the strengths of multiple quantum systems. In this work, we propose a hybrid networking architecture designed to improve entanglement rates in quantum networks based on trapped ions. Trapped ions are excellent candidates as network nodes but photon losses make long-distance networking difficult. To overcome some losses and extend the range of trapped-ion-based networks, we propose including neutral-atom-based non-destructive single-photon detection and single photon storage in between networking nodes, forming a hybrid network. This work builds on recently demonstrated optical frequency conversion of single photons emitted by trapped ions. We derive the average two-node entanglement rate for this proposed network architecture as a function of distance. Using reasonable experimental parameters, we show this proposed quantum network can generate remote entanglement rates up to a factor of 100 larger than that of an equivalent homogeneous network at both near-IR and C-band wavelengths for distances up to 50 km.
△ Less
Submitted 24 March, 2021; v1 submitted 11 January, 2021;
originally announced January 2021.
-
Curious Minds Wonder Alike: Studying Multimodal Behavioral Dynamics to Design Social Scaffolding of Curiosity
Authors:
Tanmay Sinha,
Zhen Bai,
Justine Cassell
Abstract:
Curiosity is the strong desire to learn or know more about something or someone. Since learning is often a social endeavor, social dynamics in collaborative learning may inevitably influence curiosity. There is a scarcity of research, however, focusing on how curiosity can be evoked in group learning contexts. Inspired by a recently proposed theoretical framework that articulates an integrated soc…
▽ More
Curiosity is the strong desire to learn or know more about something or someone. Since learning is often a social endeavor, social dynamics in collaborative learning may inevitably influence curiosity. There is a scarcity of research, however, focusing on how curiosity can be evoked in group learning contexts. Inspired by a recently proposed theoretical framework that articulates an integrated socio-cognitive infrastructure of curiosity, in this work, we use data-driven approaches to identify fine-grained social scaffolding of curiosity in child-child interaction, and propose how they can be used to elicit and maintain curiosity in technology-enhanced learning environments. For example, we discovered sequential patterns of multimodal behaviors across group members and we describe those that maximize an individual's utility, or likelihood, of demonstrating curiosity during open-ended problem-solving in group work. We also discovered, and describe here, behaviors that directly or in a mediated manner cause curiosity related conversational behaviors in the interaction, with twice as many interpersonal causal influences compared to intrapersonal ones. We explain how these findings form a solid foundation for develo** curiosity-increasing learning technologies or even assisting a human coach to induce curiosity among learners.
△ Less
Submitted 21 June, 2017; v1 submitted 29 April, 2017;
originally announced May 2017.
-
A New Theoretical Framework for Curiosity for Learning in Social Contexts
Authors:
Tanmay Sinha,
Zhen Bai,
Justine Cassell
Abstract:
Curiosity is a vital metacognitive skill in educational contexts. Yet, little is known about how social factors influence curiosity in group work. We argue that curiosity is evoked not only through individual, but also interpersonal activities, and present what we believe to be the first theoretical framework that articulates an integrated socio-cognitive account of curiosity based on literature s…
▽ More
Curiosity is a vital metacognitive skill in educational contexts. Yet, little is known about how social factors influence curiosity in group work. We argue that curiosity is evoked not only through individual, but also interpersonal activities, and present what we believe to be the first theoretical framework that articulates an integrated socio-cognitive account of curiosity based on literature spanning psychology, learning sciences and group dynamics, along with empirical observation of small-group science activity in an informal learning environment. We make a bipartite distinction between individual and interpersonal functions that contribute to curiosity, and multimodal behaviors that fulfill these functions. We validate the proposed framework by leveraging a longitudinal latent variable modeling approach. Findings confirm positive predictive relationship of the latent variables of individual and interpersonal functions on curiosity, with the interpersonal functions exercising a comparatively stronger influence. Prominent behavioral realizations of these functions are also discovered in a data-driven way. This framework is a step towards designing learning technologies that can recognize and evoke curiosity during learning in social contexts.
△ Less
Submitted 25 June, 2017; v1 submitted 24 April, 2017;
originally announced April 2017.
-
Statistical Verification of Computational Rapport Model
Authors:
Xuhai Xu,
Justine Cassell
Abstract:
Rapport plays an important role during communication because it can help people understand each other's feelings or ideas and leads to a smooth communication. Computational rapport model has been proposed based on theory in previous work. But there lacks solid verification. In this paper, we apply structural equation model (SEM) to the theoretical model on both dyads of friend and stranger. The re…
▽ More
Rapport plays an important role during communication because it can help people understand each other's feelings or ideas and leads to a smooth communication. Computational rapport model has been proposed based on theory in previous work. But there lacks solid verification. In this paper, we apply structural equation model (SEM) to the theoretical model on both dyads of friend and stranger. The results indicate some unfavorable paths. Based on the results and more literature, we modify the original model to integrate more nonverbal behaviors, including gaze and smile. Fit indices and other examination show the goodness of our new models, which can give us more insight into rapport management during conversation.
△ Less
Submitted 18 June, 2018; v1 submitted 1 March, 2017;
originally announced March 2017.
-
Leveraging Recurrent Neural Networks for Multimodal Recognition of Social Norm Violation in Dialog
Authors:
Tiancheng Zhao,
Ran Zhao,
Zhao Meng,
Justine Cassell
Abstract:
Social norms are shared rules that govern and facilitate social interaction. Violating such social norms via teasing and insults may serve to upend power imbalances or, on the contrary reinforce solidarity and rapport in conversation, rapport which is highly situated and context-dependent. In this work, we investigate the task of automatically identifying the phenomena of social norm violation in…
▽ More
Social norms are shared rules that govern and facilitate social interaction. Violating such social norms via teasing and insults may serve to upend power imbalances or, on the contrary reinforce solidarity and rapport in conversation, rapport which is highly situated and context-dependent. In this work, we investigate the task of automatically identifying the phenomena of social norm violation in discourse. Towards this goal, we leverage the power of recurrent neural networks and multimodal information present in the interaction, and propose a predictive model to recognize social norm violation. Using long-term temporal and contextual information, our model achieves an F1 score of 0.705. Implications of our work regarding develo** a social-aware agent are discussed.
△ Less
Submitted 10 October, 2016;
originally announced October 2016.
