-
Cross-Cultural Validation of Partner Models for Voice User Interfaces
Authors:
Katie Seaborn,
Iona Gessinger,
Suzuka Yoshida,
Benjamin R. Cowan,
Philip R. Doyle
Abstract:
Recent research has begun to assess people's perceptions of voice user interfaces (VUIs) as dialogue partners, termed partner models. Current self-report measures are only available in English, limiting research to English-speaking users. To improve the diversity of user samples and contexts that inform partner modelling research, we translated, localized, and evaluated the Partner Modelling Quest…
▽ More
Recent research has begun to assess people's perceptions of voice user interfaces (VUIs) as dialogue partners, termed partner models. Current self-report measures are only available in English, limiting research to English-speaking users. To improve the diversity of user samples and contexts that inform partner modelling research, we translated, localized, and evaluated the Partner Modelling Questionnaire (PMQ) for non-English speaking Western (German, n=185) and East Asian (Japanese, n=198) cohorts where VUI use is popular. Through confirmatory factor analysis (CFA), we find that the scale produces equivalent levels of goodness-to-fit for both our German and Japanese translations, confirming its cross-cultural validity. Still, the structure of the communicative flexibility factor did not replicate directly across Western and East Asian cohorts. We discuss how our translations can open up critical research on cultural similarities and differences in partner model use and design, whilst highlighting the challenges for ensuring accurate translation across cultural contexts.
△ Less
Submitted 14 May, 2024;
originally announced May 2024.
-
Theorizing Deception: A Sco** Review of Theory in Research on Dark Patterns and Deceptive Design
Authors:
Weichen Joe Chang,
Katie Seaborn,
Andrew A. Adams
Abstract:
The issue of dark patterns and deceptive designs (DPs) in everyday interfaces and interactions continues to grow. DPs are manipulative and malicious elements within user interfaces that deceive users into making unintended choices. In parallel, research on DPs has significantly increased over the past two decades. As the field has matured, epistemological gaps have also become a salient and pressi…
▽ More
The issue of dark patterns and deceptive designs (DPs) in everyday interfaces and interactions continues to grow. DPs are manipulative and malicious elements within user interfaces that deceive users into making unintended choices. In parallel, research on DPs has significantly increased over the past two decades. As the field has matured, epistemological gaps have also become a salient and pressing concern. In this sco** review, we assessed the academic work so far -- 51 papers between 2014 to 2023 -- to identify the state of theory in DP research. We identified the key theories employed, examined how these theories have been referenced, and call for enhancing the incorporation of theory into DP research. We also propose broad theoretical foundations to establish a comprehensive and solid base for contextualizing and informing future DP research from a variety of theoretical scopes and lenses.
△ Less
Submitted 13 May, 2024;
originally announced May 2024.
-
Deceptive, Disruptive, No Big Deal: Japanese People React to Simulated Dark Commercial Patterns
Authors:
Katie Seaborn,
Tatsuya Itagaki,
Mizuki Watanabe,
Yijia Wang,
** Geng,
Takao Fujii,
Yuto Mandai,
Miu Kojima,
Suzuka Yoshida
Abstract:
Dark patterns and deceptive designs (DPs) are user interface elements that trick people into taking actions that benefit the purveyor. Such designs are widely deployed, with special varieties found in certain nations like Japan that can be traced to global power hierarchies and the local socio-linguistic context of use. In this breaking work, we report on the first user study involving Japanese pe…
▽ More
Dark patterns and deceptive designs (DPs) are user interface elements that trick people into taking actions that benefit the purveyor. Such designs are widely deployed, with special varieties found in certain nations like Japan that can be traced to global power hierarchies and the local socio-linguistic context of use. In this breaking work, we report on the first user study involving Japanese people (n=30) experiencing a mock shop** website injected with simulated DPs. We found that Alphabet Soup and Misleading Reference Pricing were the most deceptive and least noticeable. Social Proofs, Sneaking in Items, and Untranslation were the least deceptive but Untranslation prevented most from cancelling their account. Mood significantly worsened after experiencing the website. We contribute the first empirical findings on a Japanese consumer base alongside a scalable approach to evaluating user attitudes, perceptions, and behaviours towards DPs in an interactive context. We urge for more human participant research and ideally collaborations with industry to assess real designs in the wild.
△ Less
Submitted 13 May, 2024;
originally announced May 2024.
-
Kawaii Computing: Sco** Out the Japanese Notion of Cute in User Experiences with Interactive Systems
Authors:
Yijia Wang,
Katie Seaborn
Abstract:
Kawaii computing is a new term for a steadily growing body of work on the Japanese notion of "cute" in human-computer interaction (HCI) research and practice. Kawaii is distinguished from general notions of cute by its experiential and culturally-sensitive nature. While it can be designed into the appearance and behaviour of interactive agents, interfaces, and systems, kawaii also refers to certai…
▽ More
Kawaii computing is a new term for a steadily growing body of work on the Japanese notion of "cute" in human-computer interaction (HCI) research and practice. Kawaii is distinguished from general notions of cute by its experiential and culturally-sensitive nature. While it can be designed into the appearance and behaviour of interactive agents, interfaces, and systems, kawaii also refers to certain affective and cultural dimensions experienced by culturally Japanese users, i.e., kawaii user experiences (UX) and mental models of kawaii elicited by the socio-cultural context of Japan. In this sco** review, we map out the ways in which kawaii has been explored within HCI research and related fields as a factor of design and experience. We illuminate theoretical and methodological gaps and opportunities for future work on kawaii computing.
△ Less
Submitted 13 May, 2024;
originally announced May 2024.
-
No Joke: An Embodied Conversational Agent Greeting Older Adults with Humour or a Smile Unrelated to Initial Acceptance
Authors:
Ge "Rikaku" Li,
Katie Seaborn
Abstract:
Embodied conversation agents (ECAs) are increasingly being developed for older adults as assistants or companions. Older adults may not be familiar with ECAs, influencing uptake and acceptability. First impressions can correlate strongly with subsequent judgments, even of computer agents, and could influence acceptance. Using the circumplex model of affect, we developed three versions of an ECA --…
▽ More
Embodied conversation agents (ECAs) are increasingly being developed for older adults as assistants or companions. Older adults may not be familiar with ECAs, influencing uptake and acceptability. First impressions can correlate strongly with subsequent judgments, even of computer agents, and could influence acceptance. Using the circumplex model of affect, we developed three versions of an ECA -- laughing, smiling, and neutral in expression -- to evaluate how positive first impressions affect acceptance. Results from 249 older adults indicated no statistically significant effects except for general attitudes towards technology and intelligent agents. This questions the potential of laughter, jokes, puns, and smiles as a method of initial engagement for older adults.
△ Less
Submitted 13 May, 2024;
originally announced May 2024.
-
Play Across Boundaries: Exploring Cross-Cultural Maldaimonic Game Experiences
Authors:
Katie Seaborn,
Satoru Iseya,
Shun Hidaka,
Sota Kobuki,
Shruti Chandra
Abstract:
Maldaimonic game experiences occur when people engage in personally fulfilling play through egocentric, destructive, and/or exploitative acts. Initial qualitative work verified this orientation and experiential construct for English-speaking Westerners. In this comparative mixed methods study, we explored whether and how maldaimonic game experiences and orientations play out in Japan, an Eastern g…
▽ More
Maldaimonic game experiences occur when people engage in personally fulfilling play through egocentric, destructive, and/or exploitative acts. Initial qualitative work verified this orientation and experiential construct for English-speaking Westerners. In this comparative mixed methods study, we explored whether and how maldaimonic game experiences and orientations play out in Japan, an Eastern gaming capital that may have cultural values incongruous with the Western philosophical basis underlying maldaimonia. We present findings anchored to the initial frameworks on maldaimonia in game experiences that show little divergence between the Japanese and US cohorts. We also extend the qualitative findings with quantitative measures on affect, player experience, and the related constructs of hedonia and eudaimonia. We confirm this novel construct for Japan and set the stage for scale development.
△ Less
Submitted 13 May, 2024;
originally announced May 2024.
-
Silver-Tongued and Sundry: Exploring Intersectional Pronouns with ChatGPT
Authors:
Takao Fujii,
Katie Seaborn,
Madeleine Steeds
Abstract:
ChatGPT is a conversational agent built on a large language model. Trained on a significant portion of human output, ChatGPT can mimic people to a degree. As such, we need to consider what social identities ChatGPT simulates (or can be designed to simulate). In this study, we explored the case of identity simulation through Japanese first-person pronouns, which are tightly connected to social iden…
▽ More
ChatGPT is a conversational agent built on a large language model. Trained on a significant portion of human output, ChatGPT can mimic people to a degree. As such, we need to consider what social identities ChatGPT simulates (or can be designed to simulate). In this study, we explored the case of identity simulation through Japanese first-person pronouns, which are tightly connected to social identities in intersectional ways, i.e., intersectional pronouns. We conducted a controlled online experiment where people from two regions in Japan (Kanto and Kinki) witnessed interactions with ChatGPT using ten sets of first-person pronouns. We discovered that pronouns alone can evoke perceptions of social identities in ChatGPT at the intersections of gender, age, region, and formality, with caveats. This work highlights the importance of pronoun use for social identity simulation, provides a language-based methodology for culturally-sensitive persona development, and advances the potential of intersectional identities in intelligent agents.
△ Less
Submitted 13 May, 2024;
originally announced May 2024.
-
Qualitative Approaches to Voice UX
Authors:
Katie Seaborn,
Jacqueline Urakami,
Peter Pennefather,
Norihisa P. Miyake
Abstract:
Voice is a natural mode of expression offered by modern computer-based systems. Qualitative perspectives on voice-based user experiences (voice UX) offer rich descriptions of complex interactions that numbers alone cannot fully represent. We conducted a systematic review of the literature on qualitative approaches to voice UX, capturing the nature of this body of work in a systematic map and offer…
▽ More
Voice is a natural mode of expression offered by modern computer-based systems. Qualitative perspectives on voice-based user experiences (voice UX) offer rich descriptions of complex interactions that numbers alone cannot fully represent. We conducted a systematic review of the literature on qualitative approaches to voice UX, capturing the nature of this body of work in a systematic map and offering a qualitative synthesis of findings. We highlight the benefits of qualitative methods for voice UX research, identify opportunities for increasing rigour in methods and outcomes, and distill patterns of experience across a diversity of devices and modes of qualitative praxis.
△ Less
Submitted 23 April, 2024;
originally announced April 2024.
-
Coimagining the Future of Voice Assistants with Cultural Sensitivity
Authors:
Katie Seaborn,
Yuto Sawa,
Mizuki Watanabe
Abstract:
Voice assistants (VAs) are becoming a feature of our everyday life. Yet, the user experience (UX) is often limited, leading to underuse, disengagement, and abandonment. Co-designing interactions for VAs with potential end-users can be useful. Crowdsourcing this process online and anonymously may add value. However, most work has been done in the English-speaking West on dialogue data sets. We must…
▽ More
Voice assistants (VAs) are becoming a feature of our everyday life. Yet, the user experience (UX) is often limited, leading to underuse, disengagement, and abandonment. Co-designing interactions for VAs with potential end-users can be useful. Crowdsourcing this process online and anonymously may add value. However, most work has been done in the English-speaking West on dialogue data sets. We must be sensitive to cultural differences in language, social interactions, and attitudes towards technology. Our aims were to explore the value of co-designing VAs in the non-Western context of Japan and demonstrate the necessity of cultural sensitivity. We conducted an online elicitation study (N = 135) where Americans (n = 64) and Japanese people (n = 71) imagined dialogues (N = 282) and activities (N = 73) with future VAs. We discuss the implications for coimagining interactions with future VAs, offer design guidelines for the Japanese and English-speaking US contexts, and suggest opportunities for cultural plurality in VA design and scholarship.
△ Less
Submitted 26 March, 2024;
originally announced March 2024.
-
Link, user-centred designer: Game characters as transcendent models
Authors:
Katie Seaborn
Abstract:
Games allow us to construct and explore identities and offer us role models, good and bad. Game characters are a reflection of us -- players and creators alike -- or could be. But do games also encode identities, values, and orientations that transcend diegetic categories and player self-insertion? I explore the notion of game characters as conduits of transcendent models through the case study of…
▽ More
Games allow us to construct and explore identities and offer us role models, good and bad. Game characters are a reflection of us -- players and creators alike -- or could be. But do games also encode identities, values, and orientations that transcend diegetic categories and player self-insertion? I explore the notion of game characters as conduits of transcendent models through the case study of Link from the Legend of Zelda series. I propose that designers embed tacit, nondiegetic patterns of praxis and complex value models, such as user-centred design, when crafting the embodiment of characters in gameplay, even unawares.
△ Less
Submitted 7 October, 2023;
originally announced October 2023.
-
Super Synthesis Pros., or why CHI PLAY needs research synthesis
Authors:
Katie Seaborn
Abstract:
Games user research is a-booming -- or maybe a-goomba-ing -- with a boundless parade of papers pop** up from every nook and pipe. We may need a super power -- or super method -- from another world. I outline three motivations for jump-starting research synthesis in games user research. I argue that: research synthesis will validate this field of study and enrich primary research (meta-scholarshi…
▽ More
Games user research is a-booming -- or maybe a-goomba-ing -- with a boundless parade of papers pop** up from every nook and pipe. We may need a super power -- or super method -- from another world. I outline three motivations for jump-starting research synthesis in games user research. I argue that: research synthesis will validate this field of study and enrich primary research (meta-scholarship); we must level up both primary and secondary research (education); and we should reflect this epistemological stance in community structures and adopt established tools and protocols (standardization). I offer power-ups to get the toads rolling.
△ Less
Submitted 7 October, 2023;
originally announced October 2023.
-
Meaningful play and malicious delight: Exploring maldaimonic game UX
Authors:
Katie Seaborn,
Satoru Iseya
Abstract:
Maldaimonia is a new experiential concept that refers to self-actualization and self-expression through egocentric, destructive, and/or exploitative activities. Still, it is unclear whether maldaimonia is an actual facet of real experience. As a subversive orientation, it may be rare or socially challenging to discuss openly. However, video games provide a space in which people can be expressive i…
▽ More
Maldaimonia is a new experiential concept that refers to self-actualization and self-expression through egocentric, destructive, and/or exploitative activities. Still, it is unclear whether maldaimonia is an actual facet of real experience. As a subversive orientation, it may be rare or socially challenging to discuss openly. However, video games provide a space in which people can be expressive in different ways without the same repercussions as in real life. Indeed, game spaces may be one of the few contexts in which to study maldaimonic experiences. In this study, we examined whether and how maldaimonia exists as a feature of game user experiences by analyzing critical self-reports of gaming activities, confirming its existence. We contribute this new construct to work on "dark play" in games research.
△ Less
Submitted 7 October, 2023;
originally announced October 2023.
-
Kawaii Game Vocalics: A Preliminary Model
Authors:
Katie Seaborn,
Katja Rogers,
Somang Name,
Miu Kojima
Abstract:
Kawaii is the Japanese concept of cute++, a global export with local characteristics. Recent work has explored kawaii as a feature of user experience (UX) with social robots, virtual characters, and voice assistants, i.e., kawaii vocalics. Games have a long history of incorporating characters that use voice as a means of expressing kawaii. Nevertheless, no work to date has evaluated kawaii game vo…
▽ More
Kawaii is the Japanese concept of cute++, a global export with local characteristics. Recent work has explored kawaii as a feature of user experience (UX) with social robots, virtual characters, and voice assistants, i.e., kawaii vocalics. Games have a long history of incorporating characters that use voice as a means of expressing kawaii. Nevertheless, no work to date has evaluated kawaii game voices or mapped out a model of kawaii game vocalics. In this work, we explored whether and how a model of kawaii vocalics maps onto game character voices. We conducted an online perceptions study (N=157) using 18 voices from kawaii characters in Japanese games. We replicated the results for computer voice and discovered nuanced relationships between gender and age, especially youthfulness, agelessness, gender ambiguity, and gender neutrality. We provide our initial model and advocate for future work on character visuals and within play contexts.
△ Less
Submitted 7 October, 2023;
originally announced October 2023.
-
Interacting with Masculinities: A Sco** Review
Authors:
Katie Seaborn
Abstract:
Gender is a hot topic in the field of human-computer interaction (HCI). Work has run the gamut, from assessing how we embed gender in our computational creations to correcting systemic sexism, online and off. While gender is often framed around women and femininities, we must recognize the genderful nature of humanity, acknowledge the evasiveness of men and masculinities, and avoid burdening women…
▽ More
Gender is a hot topic in the field of human-computer interaction (HCI). Work has run the gamut, from assessing how we embed gender in our computational creations to correcting systemic sexism, online and off. While gender is often framed around women and femininities, we must recognize the genderful nature of humanity, acknowledge the evasiveness of men and masculinities, and avoid burdening women and genderful folk as the central actors and targets of change. Indeed, critical voices have called for a shift in focus to masculinities, not only in terms of privilege, power, and patriarchal harms, but also participation, plurality, and transformation. To this end, I present a 30-year history of masculinities in HCI work through a sco** review of 126 papers published to the ACM Human Factors in Computing Systems (CHI) conference proceedings. I offer a primer and agenda grounded in the CHI and extant literatures to direct future work.
△ Less
Submitted 22 April, 2023;
originally announced April 2023.
-
"I'm" Lost in Translation: Pronoun Missteps in Crowdsourced Data Sets
Authors:
Katie Seaborn,
Yeongdae Kim
Abstract:
As virtual assistants continue to be taken up globally, there is an ever-greater need for these speech-based systems to communicate naturally in a variety of languages. Crowdsourcing initiatives have focused on multilingual translation of big, open data sets for use in natural language processing (NLP). Yet, language translation is often not one-to-one, and biases can trickle in. In this late-brea…
▽ More
As virtual assistants continue to be taken up globally, there is an ever-greater need for these speech-based systems to communicate naturally in a variety of languages. Crowdsourcing initiatives have focused on multilingual translation of big, open data sets for use in natural language processing (NLP). Yet, language translation is often not one-to-one, and biases can trickle in. In this late-breaking work, we focus on the case of pronouns translated between English and Japanese in the crowdsourced Tatoeba database. We found that masculine pronoun biases were present overall, even though plurality in language was accounted for in other ways. Importantly, we detected biases in the translation process that reflect nuanced reactions to the presence of feminine, neutral, and/or non-binary pronouns. We raise the issue of translation bias for pronouns and offer a practical solution to embed plurality in NLP data sets.
△ Less
Submitted 22 April, 2023;
originally announced April 2023.
-
The Systematic Review-lution: A Manifesto to Promote Rigour and Inclusivity in Research Synthesis
Authors:
Katja Rogers,
Katie Seaborn
Abstract:
The field of human-computer interaction (HCI) is maturing. Systematic reviews, a staple of many disciplines, play an important and often essential role in how each field contributes to human knowledge. On this prospect, we argue that our meta-level approach to research within HCI needs a revolution. First, we echo previous calls for greater rigour in primary research reporting with a view towards…
▽ More
The field of human-computer interaction (HCI) is maturing. Systematic reviews, a staple of many disciplines, play an important and often essential role in how each field contributes to human knowledge. On this prospect, we argue that our meta-level approach to research within HCI needs a revolution. First, we echo previous calls for greater rigour in primary research reporting with a view towards supporting knowledge synthesis in secondary research. Second, we must decide as a community how to carry out systematic review work in light of the many ways that knowledge is produced within HCI (rigour in secondary research methods and epistemological inclusivity). In short, our manifesto is this: we need to develop and make space for an inclusive but rigorous set of standards that supports systematic review work in HCI, through careful consideration of both primary and secondary research methods, expectations, and infrastructure. We call for any and all fellow systematic review-lutionaries to join us.
△ Less
Submitted 22 April, 2023;
originally announced April 2023.
-
Linguistic Dead-Ends and Alphabet Soup: Finding Dark Patterns in Japanese Apps
Authors:
Shun Hidaka,
Sota Kobuki,
Mizuki Watanabe,
Katie Seaborn
Abstract:
Dark patterns are deceptive and malicious properties of user interfaces that lead the end-user to do something different from intended or expected. While now a key topic in critical computing, most work has been conducted in Western contexts. Japan, with its booming app market, is a relatively uncharted context that offers culturally- and linguistically-sensitive differences in design standards, c…
▽ More
Dark patterns are deceptive and malicious properties of user interfaces that lead the end-user to do something different from intended or expected. While now a key topic in critical computing, most work has been conducted in Western contexts. Japan, with its booming app market, is a relatively uncharted context that offers culturally- and linguistically-sensitive differences in design standards, contexts of use, values, and language, all of which could influence the presence and expression of dark patterns. In this work, we analyzed 200 popular mobile apps in the Japanese market. We found that most apps had dark patterns, with an average of 3.9 per app. We also identified a new class of dark pattern: "Linguistic Dead-Ends" in the forms of "Untranslation" and "Alphabet Soup." We outline the implications for design and research practice, especially for future cross-cultural research on dark patterns.
△ Less
Submitted 22 April, 2023;
originally announced April 2023.
-
Transcending the "Male Code": Implicit Masculine Biases in NLP Contexts
Authors:
Katie Seaborn,
Shruti Chandra,
Thibault Fabre
Abstract:
Critical scholarship has elevated the problem of gender bias in data sets used to train virtual assistants (VAs). Most work has focused on explicit biases in language, especially against women, girls, femme-identifying people, and genderqueer folk; implicit associations through word embeddings; and limited models of gender and masculinities, especially toxic masculinities, conflation of sex and ge…
▽ More
Critical scholarship has elevated the problem of gender bias in data sets used to train virtual assistants (VAs). Most work has focused on explicit biases in language, especially against women, girls, femme-identifying people, and genderqueer folk; implicit associations through word embeddings; and limited models of gender and masculinities, especially toxic masculinities, conflation of sex and gender, and a sex/gender binary framing of the masculine as diametric to the feminine. Yet, we must also interrogate how masculinities are "coded" into language and the assumption of "male" as the linguistic default: implicit masculine biases. To this end, we examined two natural language processing (NLP) data sets. We found that when gendered language was present, so were gender biases and especially masculine biases. Moreover, these biases related in nuanced ways to the NLP context. We offer a new dictionary called AVA that covers ambiguous associations between gendered language and the language of VAs.
△ Less
Submitted 21 April, 2023;
originally announced April 2023.
-
Can Voice Assistants Sound Cute? Towards a Model of Kawaii Vocalics
Authors:
Katie Seaborn,
Somang Nam,
Julia Keckeis,
Tatsuya Itagaki
Abstract:
The Japanese notion of "kawaii" or expressions of cuteness, vulnerability, and/or charm is a global cultural export. Work has explored kawaii-ness as a design feature and factor of user experience in the visual appearance, nonverbal behaviour, and sound of robots and virtual characters. In this initial work, we consider whether voices can be kawaii by exploring the vocal qualities of voice assista…
▽ More
The Japanese notion of "kawaii" or expressions of cuteness, vulnerability, and/or charm is a global cultural export. Work has explored kawaii-ness as a design feature and factor of user experience in the visual appearance, nonverbal behaviour, and sound of robots and virtual characters. In this initial work, we consider whether voices can be kawaii by exploring the vocal qualities of voice assistant speech, i.e., kawaii vocalics. Drawing from an age-inclusive model of kawaii, we ran a user perceptions study on the kawaii-ness of younger- and older-sounding Japanese computer voices. We found that kawaii-ness intersected with perceptions of gender and age, i.e., gender ambiguous and girlish, as well as VA features, i.e., fluency and artificiality. We propose an initial model of kawaii vocalics to be validated through the identification and study of vocal qualities, cognitive appraisals, behavioural responses, and affective reports.
△ Less
Submitted 21 April, 2023;
originally announced April 2023.
-
Nonverbal Cues in Human-Robot Interaction: A Communication Studies Perspective
Authors:
Jacqueline Urakami,
Katie Seaborn
Abstract:
Communication between people is characterized by a broad range of nonverbal cues. Transferring these cues into the design of robots and other artificial agents that interact with people may foster more natural, inviting, and accessible experiences. In this position paper, we offer a series of definitive nonverbal codes for human-robot interaction (HRI) that address the five human sensory systems (…
▽ More
Communication between people is characterized by a broad range of nonverbal cues. Transferring these cues into the design of robots and other artificial agents that interact with people may foster more natural, inviting, and accessible experiences. In this position paper, we offer a series of definitive nonverbal codes for human-robot interaction (HRI) that address the five human sensory systems (visual, auditory, haptic, olfactory, gustatory) drawn from the field of communication studies. We discuss how these codes can be translated into design patterns for HRI using a curated sample of the communication studies and HRI literatures. As nonverbal codes are an essential mode in human communication, we argue that integrating robotic nonverbal codes in HRI will afford robots a feeling of "aliveness" or "social agency" that would otherwise be missing. We end with suggestions for research directions to stimulate work on nonverbal communication within the field of HRI and improve communication between human and robots.
△ Less
Submitted 21 April, 2023;
originally announced April 2023.
-
Dis/Immersion in Mindfulness Meditation with a Wandering Voice Assistant
Authors:
Bonhee Ku,
Tatsuya Itagaki,
Katie Seaborn
Abstract:
Mindfulness meditation is a validated means of hel** people manage stress. Voice-based virtual assistants (VAs) in smart speakers, smartphones, and smart environments can assist people in carrying out mindfulness meditation through guided experiences. However, the common fixed location embodiment of VAs makes it difficult to provide intuitive support. In this work, we explored the novel embodime…
▽ More
Mindfulness meditation is a validated means of hel** people manage stress. Voice-based virtual assistants (VAs) in smart speakers, smartphones, and smart environments can assist people in carrying out mindfulness meditation through guided experiences. However, the common fixed location embodiment of VAs makes it difficult to provide intuitive support. In this work, we explored the novel embodiment of a "wandering voice" that is co-located with the user and "moves" with the task. We developed a multi-speaker VA embedded in a yoga mat that changes location along the body according to the meditation experience. We conducted a qualitative user study in two sessions, comparing a typical fixed smart speaker to the wandering VA embodiment. Thick descriptions from interviews with twelve people revealed sometimes simultaneous experiences of immersion and dis-immersion. We offer design implications for "wandering voices" and a new paradigm for VA embodiment that may extend to guidance tasks in other contexts.
△ Less
Submitted 21 April, 2023;
originally announced April 2023.
-
Trust and Reliance in Consensus-Based Explanations from an Anti-Misinformation Agent
Authors:
Takane Ueno,
Yeongdae Kim,
Hiroki Oura,
Katie Seaborn
Abstract:
The illusion of consensus occurs when people believe there is consensus across multiple sources, but the sources are the same and thus there is no "true" consensus. We explore this phenomenon in the context of an AI-based intelligent agent designed to augment metacognition on social media. Misinformation, especially on platforms like Twitter, is a global problem for which there is currently no goo…
▽ More
The illusion of consensus occurs when people believe there is consensus across multiple sources, but the sources are the same and thus there is no "true" consensus. We explore this phenomenon in the context of an AI-based intelligent agent designed to augment metacognition on social media. Misinformation, especially on platforms like Twitter, is a global problem for which there is currently no good solution. As an explainable AI (XAI) system, the agent provides explanations for its decisions on the misinformed nature of social media content. In this late-breaking study, we explored the roles of trust (attitude) and reliance (behaviour) as key elements of XAI user experience (UX) and whether these influenced the illusion of consensus. Findings show no effect of trust, but an effect of reliance on consensus-based explanations. This work may guide the design of anti-misinformation systems that use XAI, especially the user-centred design of explanations.
△ Less
Submitted 21 April, 2023;
originally announced April 2023.
-
Exoskeleton for the Mind: Exploring Strategies Against Misinformation with a Metacognitive Agent
Authors:
Yeongdae Kim,
Takane Ueno,
Katie Seaborn,
Hiroki Oura,
Jacqueline Urakami,
Yuto Sawa
Abstract:
Misinformation is a global problem in modern social media platforms with few solutions known to be effective. Social media platforms have offered tools to raise awareness of information, but these are closed systems that have not been empirically evaluated. Others have developed novel tools and strategies, but most have been studied out of context using static stimuli, researcher prompts, or low f…
▽ More
Misinformation is a global problem in modern social media platforms with few solutions known to be effective. Social media platforms have offered tools to raise awareness of information, but these are closed systems that have not been empirically evaluated. Others have developed novel tools and strategies, but most have been studied out of context using static stimuli, researcher prompts, or low fidelity prototypes. We offer a new anti-misinformation agent grounded in theories of metacognition that was evaluated within Twitter. We report on a pilot study (n=17) and multi-part experimental study (n=57, n=49) where participants experienced three versions of the agent, each deploying a different strategy. We found that no single strategy was superior over the control. We also confirmed the necessity of transparency and clarity about the agent's underlying logic, as well as concerns about repeated exposure to misinformation and lack of user engagement.
△ Less
Submitted 18 April, 2023;
originally announced April 2023.
-
Not Only WEIRD but "Uncanny"? A Systematic Review of Diversity in Human-Robot Interaction Research
Authors:
Katie Seaborn,
Giulia Barbareschi,
Shruti Chandra
Abstract:
Critical voices within and beyond the scientific community have pointed to a grave matter of concern regarding who is included in research and who is not. Subsequent investigations have revealed an extensive form of sampling bias across a broad range of disciplines that conduct human subjects research called "WEIRD": Western, Educated, Industrial, Rich, and Democratic. Recent work has indicated th…
▽ More
Critical voices within and beyond the scientific community have pointed to a grave matter of concern regarding who is included in research and who is not. Subsequent investigations have revealed an extensive form of sampling bias across a broad range of disciplines that conduct human subjects research called "WEIRD": Western, Educated, Industrial, Rich, and Democratic. Recent work has indicated that this pattern exists within human-computer interaction (HCI) research, as well. How then does human-robot interaction (HRI) fare? And could there be other patterns of sampling bias at play, perhaps those especially relevant to this field of study? We conducted a systematic review of the premier ACM/IEEE International Conference on Human-Robot Interaction (2006-2022) to discover whether and how WEIRD HRI research is. Importantly, we expanded our purview to other factors of representation highlighted by critical work on inclusion and intersectionality as potentially underreported, overlooked, and even marginalized factors of human diversity. Findings from 827 studies across 749 papers confirm that participants in HRI research also tend to be drawn from WEIRD populations. Moreover, we find evidence of limited, obscured, and possible misrepresentation in participant sampling and reporting along key axes of diversity: sex and gender, race and ethnicity, age, sexuality and family configuration, disability, body type, ideology, and domain expertise. We discuss methodological and ethical implications for recruitment, analysis, and reporting, as well as the significance for HRI as a base of knowledge.
△ Less
Submitted 18 April, 2023;
originally announced April 2023.
-
Wizundry: A Cooperative Wizard of Oz Platform for Simulating Future Speech-based Interfaces with Multiple Wizards
Authors:
Siying Hu,
Hen Chen Yen,
Ziwei Yu,
Mingjian Zhao,
Katie Seaborn,
Can Liu
Abstract:
Wizard of Oz (WoZ) as a prototy** method has been used to simulate intelligent user interfaces, particularly for speech-based systems. However, as our societies' expectations on artificial intelligence (AI) grows, the question remains whether a single Wizard is sufficient for it to simulate smarter systems and more complex interactions. Optimistic visions of 'what artificial intelligence (AI) ca…
▽ More
Wizard of Oz (WoZ) as a prototy** method has been used to simulate intelligent user interfaces, particularly for speech-based systems. However, as our societies' expectations on artificial intelligence (AI) grows, the question remains whether a single Wizard is sufficient for it to simulate smarter systems and more complex interactions. Optimistic visions of 'what artificial intelligence (AI) can do' places demands on WoZ platforms to simulate smarter systems and more complex interactions. This raises the question of whether the typical approach of employing a single Wizard is sufficient. Moreover, while existing work has employed multiple Wizards in WoZ studies, a multi-Wizard approach has not been systematically studied in terms of feasibility, effectiveness, and challenges. We offer Wizundry, a real-time, web-based WoZ platform that allows multiple Wizards to collaboratively operate a speech-to-text based system remotely. We outline the design and technical specifications of our open-source platform, which we iterated over two design phases. We report on two studies in which participant-Wizards were tasked with negotiating how to cooperatively simulate an interface that can handle natural speech for dictation and text editing as well as other intelligent text processing tasks. We offer qualitative findings on the Multi-Wizard experience for Dyads and Triads of Wizards. Our findings reveal the promises and challenges of the multi-Wizard approach and open up new research questions.
△ Less
Submitted 17 April, 2023;
originally announced April 2023.
-
Voice Over Body? Older Adults' Reactions to Robot and Voice Assistant Facilitators of Group Conversation
Authors:
Katie Seaborn,
Takuya Sekiguchi,
Seiki Tokunaga,
Norihisa P. Miyake,
Mihoko Otake-Matsuura
Abstract:
Intelligent agents have great potential as facilitators of group conversation among older adults. However, little is known about how to design agents for this purpose and user group, especially in terms of agent embodiment. To this end, we conducted a mixed methods study of older adults' reactions to voice and body in a group conversation facilitation agent. Two agent forms with the same underlyin…
▽ More
Intelligent agents have great potential as facilitators of group conversation among older adults. However, little is known about how to design agents for this purpose and user group, especially in terms of agent embodiment. To this end, we conducted a mixed methods study of older adults' reactions to voice and body in a group conversation facilitation agent. Two agent forms with the same underlying artificial intelligence (AI) and voice system were compared: a humanoid robot and a voice assistant. One preliminary study (total n=24) and one experimental study comparing voice and body morphologies (n=36) were conducted with older adults and an experienced human facilitator. Findings revealed that the artificiality of the agent, regardless of its form, was beneficial for the socially uncomfortable task of conversation facilitation. Even so, talkative personality types had a poorer experience with the "bodied" robot version. Design implications and supplementary reactions, especially to agent voice, are also discussed.
△ Less
Submitted 8 December, 2022;
originally announced December 2022.
-
What Pronouns for Pepper? A Critical Review of Gender/ing in Research
Authors:
Katie Seaborn,
Alexa Frank
Abstract:
Gender/ing guides how we view ourselves, the world around us, and each other--including non-humans. Critical voices have raised the alarm about stereotyped gendering in the design of socially embodied artificial agents like voice assistants, conversational agents, and robots. Yet, little is known about how this plays out in research and to what extent. As a first step, we critically reviewed the c…
▽ More
Gender/ing guides how we view ourselves, the world around us, and each other--including non-humans. Critical voices have raised the alarm about stereotyped gendering in the design of socially embodied artificial agents like voice assistants, conversational agents, and robots. Yet, little is known about how this plays out in research and to what extent. As a first step, we critically reviewed the case of Pepper, a gender-ambiguous humanoid robot. We conducted a systematic review (n=75) involving meta-synthesis and content analysis, examining how participants and researchers gendered Pepper through stated and unstated signifiers and pronoun usage. We found that ascriptions of Pepper's gender were inconsistent, limited, and at times discordant, with little evidence of conscious gendering and some indication of researcher influence on participant gendering. We offer six challenges driving the state of affairs and a practical framework coupled with a critical checklist for centering gender in research on artificial agents.
△ Less
Submitted 8 December, 2022;
originally announced December 2022.
-
Exploring Gender-Expansive Categorization Options for Robots
Authors:
Katie Seaborn,
Peter Pennefather,
Haruki Kotani
Abstract:
Gender is increasingly being explored as a social characteristic ascribed to robots by people. Yet, research involving social robots that may be gendered tends not to address gender perceptions, such as through pilot studies or manipulation checks. Moreover, research that does address gender perceptions has been limited by a reliance on the human gender binary model of feminine and masculine, pres…
▽ More
Gender is increasingly being explored as a social characteristic ascribed to robots by people. Yet, research involving social robots that may be gendered tends not to address gender perceptions, such as through pilot studies or manipulation checks. Moreover, research that does address gender perceptions has been limited by a reliance on the human gender binary model of feminine and masculine, prescriptive response options, and/or researcher assumptions and/or ascriptions of participant gendering. In response, we conducted an online pilot categorization study (n=55) wherein we provided gender-expansive response options for rating four robots ranging across four levels of anthropomorphism. Findings indicate that people gender robots in diverse ways, and not necessarily in relation to the gender binary. Additionally, less anthropomorphic robots and the childlike humanoid robot were deemed masculine, while the iconic robot was deemed gender neutral, fluid, and/or ambiguous. We discuss implications for future work on all humanoid robots.
△ Less
Submitted 30 April, 2022;
originally announced May 2022.
-
Trust in Human-AI Interaction: Sco** Out Models, Measures, and Methods
Authors:
Takane Ueno,
Yuto Sawa,
Yeongdae Kim,
Jacqueline Urakami,
Hiroki Oura,
Katie Seaborn
Abstract:
Trust has emerged as a key factor in people's interactions with AI-infused systems. Yet, little is known about what models of trust have been used and for what systems: robots, virtual characters, smart vehicles, decision aids, or others. Moreover, there is yet no known standard approach to measuring trust in AI. This sco** review maps out the state of affairs on trust in human-AI interaction (H…
▽ More
Trust has emerged as a key factor in people's interactions with AI-infused systems. Yet, little is known about what models of trust have been used and for what systems: robots, virtual characters, smart vehicles, decision aids, or others. Moreover, there is yet no known standard approach to measuring trust in AI. This sco** review maps out the state of affairs on trust in human-AI interaction (HAII) from the perspectives of models, measures, and methods. Findings suggest that trust is an important and multi-faceted topic of study within HAII contexts. However, most work is under-theorized and under-reported, generally not using established trust models and missing details about methods, especially Wizard of Oz. We offer several targets for systematic review work as well as a research agenda for combining the strengths and addressing the weaknesses of the current literature.
△ Less
Submitted 30 April, 2022;
originally announced May 2022.
-
Finding Strategies Against Misinformation in Social Media: A Qualitative Study
Authors:
Jacqueline Urakami,
Yeongdae Kim,
Hiroki Oura,
Katie Seaborn
Abstract:
Misinformation spread through social media has become a fundamental challenge in modern society. Recent studies have evaluated various strategies for addressing this problem, such as by modifying social media platforms or educating people about misinformation, to varying degrees of success. Our goal is to develop a new strategy for countering misinformation: intelligent tools that encourage social…
▽ More
Misinformation spread through social media has become a fundamental challenge in modern society. Recent studies have evaluated various strategies for addressing this problem, such as by modifying social media platforms or educating people about misinformation, to varying degrees of success. Our goal is to develop a new strategy for countering misinformation: intelligent tools that encourage social media users to foster metacognitive skills "in the wild." As a first step, we conducted focus groups with social media users to discover how they can be best supported in combating misinformation. Qualitative analyses of the discussions revealed that people find it difficult to detect misinformation. Findings also indicated a need for but lack of resources to support cross-validation of information. Moreover, misinformation had a nuanced emotional impact on people. Suggestions for the design of intelligent tools that support social media users in information selection, information engagement, and emotional response management are presented.
△ Less
Submitted 30 April, 2022;
originally announced May 2022.
-
Neither "Hear" Nor "Their": Interrogating Gender Neutrality in Robots
Authors:
Katie Seaborn,
Peter Pennefather
Abstract:
Gender is a social framework through which people organize themselves-and non-human subjects, including robots. Research stretching back decades has found evidence that people tend to gender artificial agents unwittingly, even with the slightest cue of humanlike features in voice, body, role, and other social features. This has led to the notion of gender neutrality in robots: ways in which we can…
▽ More
Gender is a social framework through which people organize themselves-and non-human subjects, including robots. Research stretching back decades has found evidence that people tend to gender artificial agents unwittingly, even with the slightest cue of humanlike features in voice, body, role, and other social features. This has led to the notion of gender neutrality in robots: ways in which we can avoid gendering robots in line with human models, as well explorations of extra-human genders. This rapid review critically surveyed the literature to capture the state of art on gender neutrality in robots that interact with people. We present findings on theory, methods, results, and reflexivity. We interrogate the very idea that robot gender/ing can be neutral and explore alternate ways of approaching gender/ing through the design and study of robots interacting with people.
△ Less
Submitted 30 April, 2022;
originally announced May 2022.
-
Gender neutrality in robots: An open living review framework
Authors:
Katie Seaborn,
Peter Pennefather
Abstract:
Gender is a primary characteristic by which people organize themselves. Previous research has shown that people tend to unknowingly ascribe gender to robots based on features of their embodiment. Yet, robots are not necessarily ascribed the same, or any, gender by different people. Indeed, robots may be ascribed non-human genders or used as "genderless" alternatives. This underlies the notion of g…
▽ More
Gender is a primary characteristic by which people organize themselves. Previous research has shown that people tend to unknowingly ascribe gender to robots based on features of their embodiment. Yet, robots are not necessarily ascribed the same, or any, gender by different people. Indeed, robots may be ascribed non-human genders or used as "genderless" alternatives. This underlies the notion of gender neutrality in robots: neither masculine nor feminine but somewhere in between or even beyond gender. Responding to calls for gender as a locus of study within robotics, we offer a framework for conducting an open living review to be updated periodically as work emerges. Significantly, we provide an open, formalized submission process and open access dataset of research on gender neutrality in robots. This novel and timely approach to consensus-building is expected to pave the way for similar endeavours on other key topics within human-robot interaction research.
△ Less
Submitted 30 April, 2022;
originally announced May 2022.
-
Crossing the Tepper Line: An Emerging Ontology for Describing the Dynamic Sociality of Embodied AI
Authors:
Katie Seaborn,
Peter Pennefather,
Norihisa P. Miyake,
Mihoko Otake-Matsuura
Abstract:
Artificial intelligences (AI) are increasingly being embodied and embedded in the world to carry out tasks and support decision-making with and for people. Robots, recommender systems, voice assistants, virtual humans - do these disparate types of embodied AI have something in common? Here we show how they can manifest as "socially embodied AI." We define this as the state that embodied AI "circum…
▽ More
Artificial intelligences (AI) are increasingly being embodied and embedded in the world to carry out tasks and support decision-making with and for people. Robots, recommender systems, voice assistants, virtual humans - do these disparate types of embodied AI have something in common? Here we show how they can manifest as "socially embodied AI." We define this as the state that embodied AI "circumstantially" take on within interactive contexts when perceived as both social and agentic by people. We offer a working ontology that describes how embodied AI can dynamically transition into socially embodied AI. We propose an ontological heuristic for describing the threshold: the Tepper line. We reinforce our theoretical work with expert insights from a card sort workshop. We end with two case studies to illustrate the dynamic and contextual nature of this heuristic.
△ Less
Submitted 14 March, 2021;
originally announced March 2021.
-
Measuring Voice UX Quantitatively: A Rapid Review
Authors:
Katie Seaborn,
Jacqueline Urakami
Abstract:
Computer voice is experiencing a renaissance through the growing popularity of voice-based interfaces, agents, and environments. Yet, how to measure the user experience (UX) of voice-based systems remains an open and urgent question, especially given that their form factors and interaction styles tend to be non-visual, intangible, and often considered disembodied or "body-less." As a first step, w…
▽ More
Computer voice is experiencing a renaissance through the growing popularity of voice-based interfaces, agents, and environments. Yet, how to measure the user experience (UX) of voice-based systems remains an open and urgent question, especially given that their form factors and interaction styles tend to be non-visual, intangible, and often considered disembodied or "body-less." As a first step, we surveyed the ACM and IEEE literatures to determine which quantitative measures and measurements have been deemed important for voice UX. Our findings show that there is little consensus, even with similar situations and systems, as well as an overreliance on lab work and unvalidated scales. In response, we offer two high-level descriptive frameworks for guiding future research, develo** standardized instruments, and informing ongoing review work. Our work highlights the current strengths and weaknesses of voice UX research and charts a path towards measuring voice UX in a more comprehensive way.
△ Less
Submitted 12 March, 2021;
originally announced March 2021.
-
Removing Gamification: A Research Agenda
Authors:
Katie Seaborn
Abstract:
The effect of removing gamification elements from interactive systems has been a long-standing question in gamification research. Early work and foundational theories raised concerns about the endurance of positive effects and the emergence of negative ones. Yet, nearly a decade later, no work to date has sought consensus on these matters. Here, I offer a rapid review on the state of the art and w…
▽ More
The effect of removing gamification elements from interactive systems has been a long-standing question in gamification research. Early work and foundational theories raised concerns about the endurance of positive effects and the emergence of negative ones. Yet, nearly a decade later, no work to date has sought consensus on these matters. Here, I offer a rapid review on the state of the art and what is known about the impact of removing gamification. A small corpus of 8 papers published between 2012 and 2020 were found. Findings suggest a mix of positive and negative effects related to removing gamification. Significantly, insufficient reporting, methodological weaknesses, limited measures, and superficial interpretations of "negative" results prevent firm conclusions. I offer a research agenda towards better understanding the nature of gamification removal. I end with a call for empirical and theoretical work on illuminating the effects that may linger after systems are un-gamified.
△ Less
Submitted 14 March, 2021; v1 submitted 9 March, 2021;
originally announced March 2021.
