DBATES: DataBase of Audio features, Text, and visual Expressions in competitive debate Speeches
Authors:
Taylan K. Sen,
Gazi Naven,
Luke Gerstner,
Daryl Bagley,
Raiyan Abdul Baten,
Wasifur Rahman,
Kamrul Hasan,
Kurtis G. Haut,
Abdullah Mamun,
Samiha Samrose,
Anne Solbu,
R. Eric Barnes,
Mark G. Frank,
Ehsan Hoque
Abstract:
In this work, we present a database of multimodal communication features extracted from debate speeches in the 2019 North American Universities Debate Championships (NAUDC). Feature sets were extracted from the visual (facial expression, gaze, and head pose), audio (PRAAT), and textual (word sentiment and linguistic category) modalities of raw video recordings of competitive collegiate debaters (N…
▽ More
In this work, we present a database of multimodal communication features extracted from debate speeches in the 2019 North American Universities Debate Championships (NAUDC). Feature sets were extracted from the visual (facial expression, gaze, and head pose), audio (PRAAT), and textual (word sentiment and linguistic category) modalities of raw video recordings of competitive collegiate debaters (N=717 6-minute recordings from 140 unique debaters). Each speech has an associated competition debate score (range: 67-96) from expert judges as well as competitor demographic and per-round reflection surveys. We observe the fully multimodal model performs best in comparison to models trained on various compositions of modalities. We also find that the weights of some features (such as the expression of joy and the use of the word we) change in direction between the aforementioned models. We use these results to highlight the value of a multimodal dataset for studying competitive, collegiate debate.
△ Less
Submitted 25 March, 2021;
originally announced March 2021.
Immediate or Reflective?: Effects of Real-timeFeedback on Group Discussions over Videochat
Authors:
Samiha Samrose,
Reza Rawassizadeh,
Ehsan Hoque
Abstract:
Having a group discussion with the members holding conflicting viewpoints is difficult. It is especially challenging for machine-mediated discussions in which the subtle social cues are hard to notice. We present a fully automated videochat framework that can automatically analyze audio-video data of the participants and provide real-time feedback on participation, interruption, volume, and facial…
▽ More
Having a group discussion with the members holding conflicting viewpoints is difficult. It is especially challenging for machine-mediated discussions in which the subtle social cues are hard to notice. We present a fully automated videochat framework that can automatically analyze audio-video data of the participants and provide real-time feedback on participation, interruption, volume, and facial emotion. In a heated discourse, these features are especially aligned with the undesired characteristics of dominating the conversation without taking turns, interrupting constantly, raising voice, and expressing negative emotion. We conduct a treatment-control user study with 40 participants having 20 sessions in total. We analyze the immediate and the reflective effects of real-time feedback on participants. Our findings show that while real-time feedback can make the ongoing discussion significantly less spontaneous, its effects propagate to successive sessions bringing significantly more expressiveness to the team. Our explorations with instant and propagated impacts of real-time feedback can be useful for develo** design strategies for various collaborative environments.
△ Less
Submitted 12 November, 2020;
originally announced November 2020.
