-
A Graphical Model of Hurricane Evacuation Behaviors
Authors:
Hui Sophie Wang,
Nutchanon Yongsatianchot,
Stacy Marsella
Abstract:
Natural disasters such as hurricanes are increasing and causing widespread devastation. People's decisions and actions regarding whether to evacuate or not are critical and have a large impact on emergency planning and response. Our interest lies in computationally modeling complex relationships among various factors influencing evacuation decisions. We conducted a study on the evacuation of Hurri…
▽ More
Natural disasters such as hurricanes are increasing and causing widespread devastation. People's decisions and actions regarding whether to evacuate or not are critical and have a large impact on emergency planning and response. Our interest lies in computationally modeling complex relationships among various factors influencing evacuation decisions. We conducted a study on the evacuation of Hurricane Irma of the 2017 Atlantic hurricane season. The study was guided by the Protection motivation theory (PMT), a widely-used framework to understand people's responses to potential threats. Graphical models were constructed to represent the complex relationships among the factors involved and the evacuation decision. We evaluated different graphical structures based on conditional independence tests using Irma data. The final model largely aligns with PMT. It shows that both risk perception (threat appraisal) and difficulties in evacuation (co** appraisal) influence evacuation decisions directly and independently. Certain information received from media was found to influence risk perception, and through it influence evacuation behaviors indirectly. In addition, several variables were found to influence both risk perception and evacuation behaviors directly, including family and friends' suggestions, neighbors' evacuation behaviors, and evacuation notices from officials.
△ Less
Submitted 16 November, 2023;
originally announced November 2023.
-
A Unified Dynamic Model for the Decomposition of Skin Conductance and the Inference of Sudomotor Nerve Activities
Authors:
Hui Sophie Wang,
Stacy Marsella,
Misha Pavel
Abstract:
In behavioral health informatics, inferring an individual's psychological state from physiological and behavioral data is fundamental. A key physiological signal in this endeavor is electrodermal activity (EDA), often quantified as skin conductance (SC), known for its sensitivity to a variety of psychological stimuli. Traditional methods to analyze skin conductance, such as the trough-to-peak meth…
▽ More
In behavioral health informatics, inferring an individual's psychological state from physiological and behavioral data is fundamental. A key physiological signal in this endeavor is electrodermal activity (EDA), often quantified as skin conductance (SC), known for its sensitivity to a variety of psychological stimuli. Traditional methods to analyze skin conductance, such as the trough-to-peak method, often result in imprecise estimations due to overlap** skin conductance responses. While various mathematical models have been proposed to improve the analysis, many of them do not incorporate the tonic level in the dynamic system. This paper introduces a novel fourth order dynamic system to model the temporal dynamics of skin conductance, unifying both the tonic level and phasic response. Applied to a large dataset with over 200 participants, majority of the models achieved an $R^2$ value above 0.99. Furthermore, this work offers a unique three-component decomposition of skin conductance, shedding light on its temporal dynamics. Comparative evaluations highlight the model's capability to differentiate arousal levels and maintain an appropriate sparsity level for the estimated sudomotor nerve activities signal. The code of the proposed model and algorithm are available for open access.
△ Less
Submitted 5 November, 2023;
originally announced November 2023.
-
What's Next in Affective Modeling? Large Language Models
Authors:
Nutchanon Yongsatianchot,
Tobias Thejll-Madsen,
Stacy Marsella
Abstract:
Large Language Models (LLM) have recently been shown to perform well at various tasks from language understanding, reasoning, storytelling, and information search to theory of mind. In an extension of this work, we explore the ability of GPT-4 to solve tasks related to emotion prediction. GPT-4 performs well across multiple emotion tasks; it can distinguish emotion theories and come up with emotio…
▽ More
Large Language Models (LLM) have recently been shown to perform well at various tasks from language understanding, reasoning, storytelling, and information search to theory of mind. In an extension of this work, we explore the ability of GPT-4 to solve tasks related to emotion prediction. GPT-4 performs well across multiple emotion tasks; it can distinguish emotion theories and come up with emotional stories. We show that by prompting GPT-4 to identify key factors of an emotional experience, it is able to manipulate the emotional intensity of its own stories. Furthermore, we explore GPT-4's ability on reverse appraisals by asking it to predict either the goal, belief, or emotion of a person using the other two. In general, GPT-4 can make the correct inferences. We suggest that LLMs could play an important role in affective modeling; however, they will not fully replace works that attempt to model the mechanisms underlying emotion-related processes.
△ Less
Submitted 3 October, 2023;
originally announced October 2023.
-
Large language models in textual analysis for gesture selection
Authors:
Laura B. Hensel,
Nutchanon Yongsatianchot,
Parisa Torshizi,
Elena Minucci,
Stacy Marsella
Abstract:
Gestures perform a variety of communicative functions that powerfully influence human face-to-face interaction. How this communicative function is achieved varies greatly between individuals and depends on the role of the speaker and the context of the interaction. Approaches to automatic gesture generation vary not only in the degree to which they rely on data-driven techniques but also the degre…
▽ More
Gestures perform a variety of communicative functions that powerfully influence human face-to-face interaction. How this communicative function is achieved varies greatly between individuals and depends on the role of the speaker and the context of the interaction. Approaches to automatic gesture generation vary not only in the degree to which they rely on data-driven techniques but also the degree to which they can produce context and speaker specific gestures. However, these approaches face two major challenges: The first is obtaining sufficient training data that is appropriate for the context and the goal of the application. The second is related to designer control to realize their specific intent for the application. Here, we approach these challenges by using large language models (LLMs) to show that these powerful models of large amounts of data can be adapted for gesture analysis and generation. Specifically, we used ChatGPT as a tool for suggesting context-specific gestures that can realize designer intent based on minimal prompts. We also find that ChatGPT can suggests novel yet appropriate gestures not present in the minimal training data. The use of LLMs is a promising avenue for gesture generation that reduce the need for laborious annotations and has the potential to flexibly and quickly adapt to different designer intents.
△ Less
Submitted 4 October, 2023;
originally announced October 2023.
-
Investigating Large Language Models' Perception of Emotion Using Appraisal Theory
Authors:
Nutchanon Yongsatianchot,
Parisa Ghanad Torshizi,
Stacy Marsella
Abstract:
Large Language Models (LLM) like ChatGPT have significantly advanced in recent years and are now being used by the general public. As more people interact with these systems, improving our understanding of these black box models is crucial, especially regarding their understanding of human psychological aspects. In this work, we investigate their emotion perception through the lens of appraisal an…
▽ More
Large Language Models (LLM) like ChatGPT have significantly advanced in recent years and are now being used by the general public. As more people interact with these systems, improving our understanding of these black box models is crucial, especially regarding their understanding of human psychological aspects. In this work, we investigate their emotion perception through the lens of appraisal and co** theory using the Stress and Co** Process Questionaire (SCPQ). SCPQ is a validated clinical instrument consisting of multiple stories that evolve over time and differ in key appraisal variables such as controllability and changeability. We applied SCPQ to three recent LLMs from OpenAI, davinci-003, ChatGPT, and GPT-4 and compared the results with predictions from the appraisal theory and human data. The results show that LLMs' responses are similar to humans in terms of dynamics of appraisal and co**, but their responses did not differ along key appraisal dimensions as predicted by the theory and data. The magnitude of their responses is also quite different from humans in several variables. We also found that GPTs can be quite sensitive to instruction and how questions are asked. This work adds to the growing literature evaluating the psychological aspects of LLMs and helps enrich our understanding of the current models.
△ Less
Submitted 3 October, 2023;
originally announced October 2023.
-
Thought Bubbles: A Proxy into Players' Mental Model Development
Authors:
Omid Mohaddesi,
Noah Chicoine,
Min Gong,
Ozlem Ergun,
Jacqueline Griffin,
David Kaeli,
Stacy Marsella,
Casper Harteveld
Abstract:
Studying mental models has recently received more attention, aiming to understand the cognitive aspects of human-computer interaction. However, there is not enough research on the elicitation of mental models in complex dynamic systems. We present Thought Bubbles as an approach for eliciting mental models and an avenue for understanding players' mental model development in interactive virtual envi…
▽ More
Studying mental models has recently received more attention, aiming to understand the cognitive aspects of human-computer interaction. However, there is not enough research on the elicitation of mental models in complex dynamic systems. We present Thought Bubbles as an approach for eliciting mental models and an avenue for understanding players' mental model development in interactive virtual environments. We demonstrate the use of Thought Bubbles in two experimental studies involving 250 participants playing a supply chain game. In our analyses, we rely on Situation Awareness (SA) levels, including perception, comprehension, and projection, and show how experimental manipulations such as disruptions and information sharing shape players' mental models and drive their decisions depending on their behavioral profile. Our results provide evidence for the use of thought bubbles in uncovering cognitive aspects of behavior by indicating how disruption location and availability of information affect people's mental model development and influence their decisions.
△ Less
Submitted 30 January, 2023;
originally announced January 2023.
-
EvolvingBehavior: Towards Co-Creative Evolution of Behavior Trees for Game NPCs
Authors:
Nathan Partlan,
Luis Soto,
Jim Howe,
Sarthak Shrivastava,
Magy Seif El-Nasr,
Stacy Marsella
Abstract:
To assist game developers in crafting game NPCs, we present EvolvingBehavior, a novel tool for genetic programming to evolve behavior trees in Unreal Engine 4. In an initial evaluation, we compare evolved behavior to hand-crafted trees designed by our researchers, and to randomly-grown trees, in a 3D survival game. We find that EvolvingBehavior is capable of producing behavior approaching the desi…
▽ More
To assist game developers in crafting game NPCs, we present EvolvingBehavior, a novel tool for genetic programming to evolve behavior trees in Unreal Engine 4. In an initial evaluation, we compare evolved behavior to hand-crafted trees designed by our researchers, and to randomly-grown trees, in a 3D survival game. We find that EvolvingBehavior is capable of producing behavior approaching the designer's goals in this context. Finally, we discuss implications and future avenues of exploration for co-creative game AI design tools, as well as challenges and difficulties in behavior tree evolution.
△ Less
Submitted 1 September, 2022;
originally announced September 2022.
-
Study of detecting behavioral signatures within DeepFake videos
Authors:
Qiaomu Miao,
Sinhwa Kang,
Stacy Marsella,
Steve DiPaola,
Chao Wang,
Ari Shapiro
Abstract:
There is strong interest in the generation of synthetic video imagery of people talking for various purposes, including entertainment, communication, training, and advertisement. With the development of deep fake generation models, synthetic video imagery will soon be visually indistinguishable to the naked eye from a naturally capture video. In addition, many methods are continuing to improve to…
▽ More
There is strong interest in the generation of synthetic video imagery of people talking for various purposes, including entertainment, communication, training, and advertisement. With the development of deep fake generation models, synthetic video imagery will soon be visually indistinguishable to the naked eye from a naturally capture video. In addition, many methods are continuing to improve to avoid more careful, forensic visual analysis. Some deep fake videos are produced through the use of facial puppetry, which directly controls the head and face of the synthetic image through the movements of the actor, allow the actor to 'puppet' the image of another. In this paper, we address the question of whether one person's movements can be distinguished from the original speaker by controlling the visual appearance of the speaker but transferring the behavior signals from another source. We conduct a study by comparing synthetic imagery that: 1) originates from a different person speaking a different utterance, 2) originates from the same person speaking a different utterance, and 3) originates from a different person speaking the same utterance. Our study shows that synthetic videos in all three cases are seen as less real and less engaging than the original source video. Our results indicate that there could be a behavioral signature that is detectable from a person's movements that is separate from their visual appearance, and that this behavioral signature could be used to distinguish a deep fake from a properly captured video.
△ Less
Submitted 6 August, 2022;
originally announced August 2022.
-
To Trust or to Stockpile: Modeling Human-Simulation Interaction in Supply Chain Shortages
Authors:
Omid Mohaddesi,
Jacqueline Griffin,
Ozlem Ergun,
David Kaeli,
Stacy Marsella,
Casper Harteveld
Abstract:
Understanding decision-making in dynamic and complex settings is a challenge yet essential for preventing, mitigating, and responding to adverse events (e.g., disasters, financial crises). Simulation games have shown promise to advance our understanding of decision-making in such settings. However, an open question remains on how we extract useful information from these games. We contribute an app…
▽ More
Understanding decision-making in dynamic and complex settings is a challenge yet essential for preventing, mitigating, and responding to adverse events (e.g., disasters, financial crises). Simulation games have shown promise to advance our understanding of decision-making in such settings. However, an open question remains on how we extract useful information from these games. We contribute an approach to model human-simulation interaction by leveraging existing methods to characterize: (1) system states of dynamic simulation environments (with Principal Component Analysis), (2) behavioral responses from human interaction with simulation (with Hidden Markov Models), and (3) behavioral responses across system states (with Sequence Analysis). We demonstrate this approach with our game simulating drug shortages in a supply chain context. Results from our experimental study with 135 participants show different player types (hoarders, reactors, followers), how behavior changes in different system states, and how sharing information impacts behavior. We discuss how our findings challenge existing literature.
△ Less
Submitted 7 January, 2022;
originally announced January 2022.
-
Examining Older Adults' Information Exposure, Wellbeing, and Adherence to Protective Measures During the COVID-19 Pandemic
Authors:
Nurul Suhaimi,
Nutchanon Yongsatianchot,
Yixuan Zhang,
Anisa Amiji,
Shivani A. Patel,
Stacy Marsella,
Miso Kim,
Jacqueline Griffin,
Andrea Parker
Abstract:
Older adults are at greater risk of experiencing negative physical and psychological impacts of the novel coronavirus 2019 (COVID-19) pandemic. Our ongoing study is assessing COVID-19 information exposure in adults aged 55 and above compared to other age groups living in Massachusetts and Georgia. This work investigates the potential association between information exposure and wellbeing as well a…
▽ More
Older adults are at greater risk of experiencing negative physical and psychological impacts of the novel coronavirus 2019 (COVID-19) pandemic. Our ongoing study is assessing COVID-19 information exposure in adults aged 55 and above compared to other age groups living in Massachusetts and Georgia. This work investigates the potential association between information exposure and wellbeing as well as adherence to COVID-19 protective measures. Our initial results show that older adults received information related to COVID-19 less frequently than the middle-aged group, yet they feel more content and less stressed than the other age groups. Further analysis to identify other potential confounding variables is addressed.
△ Less
Submitted 17 December, 2021;
originally announced December 2021.
-
Design-Driven Requirements for Computationally Co-Creative Game AI Design Tools
Authors:
Nathan Partlan,
Erica Kleinman,
Jim Howe,
Sabbir Ahmad,
Stacy Marsella,
Magy Seif El-Nasr
Abstract:
Game AI designers must manage complex interactions between the AI character, the game world, and the player, while achieving their design visions. Computational co-creativity tools can aid them, but first, AI and HCI researchers must gather requirements and determine design heuristics to build effective co-creative tools. In this work, we present a participatory design study that categorizes and a…
▽ More
Game AI designers must manage complex interactions between the AI character, the game world, and the player, while achieving their design visions. Computational co-creativity tools can aid them, but first, AI and HCI researchers must gather requirements and determine design heuristics to build effective co-creative tools. In this work, we present a participatory design study that categorizes and analyzes game AI designers' workflows, goals, and expectations for such tools. We evince deep connections between game AI design and the design of co-creative tools, and present implications for future co-creativity tool research and development.
△ Less
Submitted 29 July, 2021;
originally announced July 2021.
-
Using Virtual Humans to Understand Real Ones
Authors:
Katie Hoemann,
Behnaz Rezaei,
Stacy C. Marsella,
Sarah Ostadabbas
Abstract:
Human interactions are characterized by explicit as well as implicit channels of communication. While the explicit channel transmits overt messages, the implicit ones transmit hidden messages about the communicator (e.g., his/her intentions and attitudes). There is a growing consensus that providing a computer with the ability to manipulate implicit affective cues should allow for a more meaningfu…
▽ More
Human interactions are characterized by explicit as well as implicit channels of communication. While the explicit channel transmits overt messages, the implicit ones transmit hidden messages about the communicator (e.g., his/her intentions and attitudes). There is a growing consensus that providing a computer with the ability to manipulate implicit affective cues should allow for a more meaningful and natural way of studying particular non-verbal signals of human-human communications by human-computer interactions. In this pilot study, we created a non-dynamic human-computer interaction while manipulating three specific non-verbal channels of communication: gaze pattern, facial expression, and gesture. Participants rated the virtual agent on affective dimensional scales (pleasure, arousal, and dominance) while their physiological signal (electrodermal activity, EDA) was captured during the interaction. Assessment of the behavioral data revealed a significant and complex three-way interaction between gaze, gesture, and facial configuration on the dimension of pleasure, as well as a main effect of gesture on the dimension of dominance. These results suggest a complex relationship between different non-verbal cues and the social context in which they are interpreted. Qualifying considerations as well as possible next steps are further discussed in light of these exploratory findings.
△ Less
Submitted 13 June, 2016;
originally announced June 2016.
