Automated Assessment of Encouragement and Warmth in Classrooms Leveraging Multimodal Emotional Features and ChatGPT
Authors:
Ruikun Hou,
Tim Fütterer,
Babette Bühler,
Efe Bozkir,
Peter Gerjets,
Ulrich Trautwein,
Enkelejda Kasneci
Abstract:
Classroom observation protocols standardize the assessment of teaching effectiveness and facilitate comprehension of classroom interactions. Whereas these protocols offer teachers specific feedback on their teaching practices, the manual coding by human raters is resource-intensive and often unreliable. This has sparked interest in develo** AI-driven, cost-effective methods for automating such h…
▽ More
Classroom observation protocols standardize the assessment of teaching effectiveness and facilitate comprehension of classroom interactions. Whereas these protocols offer teachers specific feedback on their teaching practices, the manual coding by human raters is resource-intensive and often unreliable. This has sparked interest in develo** AI-driven, cost-effective methods for automating such holistic coding. Our work explores a multimodal approach to automatically estimating encouragement and warmth in classrooms, a key component of the Global Teaching Insights (GTI) study's observation protocol. To this end, we employed facial and speech emotion recognition with sentiment analysis to extract interpretable features from video, audio, and transcript data. The prediction task involved both classification and regression methods. Additionally, in light of recent large language models' remarkable text annotation capabilities, we evaluated ChatGPT's zero-shot performance on this scoring task based on transcripts. We demonstrated our approach on the GTI dataset, comprising 367 16-minute video segments from 92 authentic lesson recordings. The inferences of GPT-4 and the best-trained model yielded correlations of r = .341 and r = .441 with human ratings, respectively. Combining estimates from both models through averaging, an ensemble approach achieved a correlation of r = .513, comparable to human inter-rater reliability. Our model explanation analysis indicated that text sentiment features were the primary contributors to the trained model's decisions. Moreover, GPT-4 could deliver logical and concrete reasoning as potential teacher guidelines. Our findings provide insights into using advanced, multimodal techniques for automated classroom observation, aiming to foster teacher training through frequent and valuable feedback.
△ Less
Submitted 1 April, 2024;
originally announced April 2024.
WikiPulse - A News-Portal Based on Wikipedia
Authors:
Tobias Futterer,
Peter A. Gloor,
Tushar Malhotra,
Harrison Mfula,
Karsten Packmohr,
Stefan Schultheiss
Abstract:
More and more user-generated content is complementing conventional journalism. While we don't think that CNN or New York Times and its professional journalists will disappear anytime soon, formidable competition is emerging through humble Wikipedia editors. In earlier work (Becker 2012), we found that entertainment and sports news appeared on average about two hours earlier on Wikipedia than on CN…
▽ More
More and more user-generated content is complementing conventional journalism. While we don't think that CNN or New York Times and its professional journalists will disappear anytime soon, formidable competition is emerging through humble Wikipedia editors. In earlier work (Becker 2012), we found that entertainment and sports news appeared on average about two hours earlier on Wikipedia than on CNN and Reuters online. In this project we build a news-reader that automatically identifies late-breaking news among the most recent Wikipedia articles and then displays it on a dedicated Web site.
△ Less
Submitted 5 August, 2013;
originally announced August 2013.
