-
Listening to the Voices: Describing Ethical Caveats of Conversational User Interfaces According to Experts and Frequent Users
Authors:
Thomas Mildner,
Orla Cooney,
Anna-Maria Meck,
Marion Bartl,
Gian-Luca Savino,
Philip R. Doyle,
Diego Garaialde,
Leigh Clark,
John Sloan,
Nina Wenig,
Rainer Malaka,
Jasmin Niess
Abstract:
Advances in natural language processing and understanding have led to a rapid growth in the popularity of conversational user interfaces (CUIs). While CUIs introduce novel benefits, they also yield risks that may exploit people's trust. Although research looking at unethical design deployed through graphical user interfaces (GUIs) established a thorough understanding of so-called dark patterns, th…
▽ More
Advances in natural language processing and understanding have led to a rapid growth in the popularity of conversational user interfaces (CUIs). While CUIs introduce novel benefits, they also yield risks that may exploit people's trust. Although research looking at unethical design deployed through graphical user interfaces (GUIs) established a thorough understanding of so-called dark patterns, there is a need to continue this discourse within the CUI community to understand potentially problematic interactions. Addressing this gap, we interviewed 27 participants from three cohorts: researchers, practitioners, and frequent users of CUIs. Applying thematic analysis, we construct five themes reflecting each cohort's insights about ethical design challenges and introduce the CUI Expectation Cycle, bridging system capabilities and user expectations while considering each theme's ethical caveats. This research aims to inform future development of CUIs to consider ethical constraints while adopting a human-centred approach.
△ Less
Submitted 26 January, 2024;
originally announced January 2024.
-
Working with Trouble and Failures in Conversation between Humans and Robots (WTF 2023) & Is CUI Design Ready Yet?
Authors:
Frank Förster,
Marta Romeo,
Patrick Holthaus,
Maria Jose Galvez Trigo,
Joel E. Fischer,
Birthe Nesset,
Christian Dondrup,
Christine Murad,
Cosmin Munteanu,
Benjamin R. Cowan,
Leigh Clark,
Martin Porcheron,
Heloisa Candello,
Raina Langevin
Abstract:
Workshop proceedings of two co-located workshops "Working with Troubles and Failures in Conversation with Humans and Robots" (WTF 2023) and "Is CUI Design Ready Yet?", both of which were part of the ACM conference on conversational user interfaces 2023.
WTF 23 aimed at bringing together researchers from human-robot interaction, dialogue systems, human-computer interaction, and conversation analy…
▽ More
Workshop proceedings of two co-located workshops "Working with Troubles and Failures in Conversation with Humans and Robots" (WTF 2023) and "Is CUI Design Ready Yet?", both of which were part of the ACM conference on conversational user interfaces 2023.
WTF 23 aimed at bringing together researchers from human-robot interaction, dialogue systems, human-computer interaction, and conversation analysis. Despite all progress, robotic speech interfaces continue to be brittle in a number of ways and the experience of failure of such interfaces is commonplace amongst roboticists. However, the technical literature is positively skewed toward their good performance. The workshop aims to provide a platform for discussing communicative troubles and failures in human-robot interactions and related failures in non-robotic speech interfaces. Aims include a scrupulous investigation into communicative failures, to begin working on a taxonomy of such failures, and enable a preliminary discussion on possible mitigating strategies. Workshop website: https://sites.google.com/view/wtf2023/overview
Is CUI Design Ready Yet? As CUIs become more prevalent in both academic research and the commercial market, it becomes more essential to design usable and adoptable CUIs. While research has been growing on the methods for designing CUIs for commercial use, there has been little discussion on the overall community practice of develo** design resources to aid in practical CUI design. The aim of this workshop, therefore, is to bring the CUI community together to discuss the current practices for develo** tools and resources for practical CUI design, the adoption (or non-adoption) of these tools and resources, and how these resources are utilized in the training and education of new CUI designers entering the field. Workshop website: https://speech-interaction.org/cui2023_design_workshop/index.html
△ Less
Submitted 4 September, 2023;
originally announced January 2024.
-
Majorana Demonstrator Data Release for AI/ML Applications
Authors:
I. J. Arnquist,
F. T. Avignone III,
A. S. Barabash,
C. J. Barton,
K. H. Bhimani,
E. Blalock,
B. Bos,
M. Busch,
M. Buuck,
T. S. Caldwell,
Y. -D. Chan,
C. D. Christofferson,
P. -H. Chu,
M. L. Clark,
C. Cuesta,
J. A. Detwiler,
Yu. Efremenko,
H. Ejiri,
S. R. Elliott,
N. Fuad,
G. K. Giovanetti,
M. P. Green,
J. Gruszko,
I. S. Guinn,
V. E. Guiseppe
, et al. (35 additional authors not shown)
Abstract:
The enclosed data release consists of a subset of the calibration data from the Majorana Demonstrator experiment. Each Majorana event is accompanied by raw Germanium detector waveforms, pulse shape discrimination cuts, and calibrated final energies, all shared in an HDF5 file format along with relevant metadata. This release is specifically designed to support the training and testing of Artificia…
▽ More
The enclosed data release consists of a subset of the calibration data from the Majorana Demonstrator experiment. Each Majorana event is accompanied by raw Germanium detector waveforms, pulse shape discrimination cuts, and calibrated final energies, all shared in an HDF5 file format along with relevant metadata. This release is specifically designed to support the training and testing of Artificial Intelligence (AI) and Machine Learning (ML) algorithms upon our data. This document is structured as follows. Section I provides an overview of the dataset's content and format; Section II outlines the location of this dataset and the method for accessing it; Section III presents the NPML Machine Learning Challenge associated with this dataset; Section IV contains a disclaimer from the Majorana collaboration regarding the use of this dataset; Appendix A contains technical details of this data release. Please direct questions about the material provided within this release to [email protected] (A. Li).
△ Less
Submitted 14 September, 2023; v1 submitted 21 August, 2023;
originally announced August 2023.
-
The Partner Modelling Questionnaire: A validated self-report measure of perceptions toward machines as dialogue partners
Authors:
Philip R. Doyle,
Iona Gessinger,
Justin Edwards,
Leigh Clark,
Odile Dumbleton,
Diego Garaialde,
Daniel Rough,
Anna Bleakley,
Holly P. Branigan,
Benjamin R. Cowan
Abstract:
Recent work has looked to understand user perceptions of speech agent capabilities as dialogue partners (termed partner models), and how this affects user interaction. Yet, currently partner model effects are inferred from language production as no metrics are available to quantify these subjective perceptions more directly. Through three studies, we develop and validate the Partner Modelling Ques…
▽ More
Recent work has looked to understand user perceptions of speech agent capabilities as dialogue partners (termed partner models), and how this affects user interaction. Yet, currently partner model effects are inferred from language production as no metrics are available to quantify these subjective perceptions more directly. Through three studies, we develop and validate the Partner Modelling Questionnaire (PMQ): an 18-item self-report semantic differential scale designed to reliably measure people's partner models of non-embodied speech interfaces. Through principal component analysis and confirmatory factor analysis, we show that the PMQ scale consists of three factors: communicative competence and dependability, human-likeness in communication, and communicative flexibility. Our studies show that the measure consistently demonstrates good internal reliability, strong test-retest reliability over 12 and 4-week intervals, and predictable convergent/divergent validity. Based on our findings we discuss the multidimensional nature of partner models, whilst identifying key future research avenues that the development of the PMQ facilitates. Notably, this includes the need to identify the activation, sensitivity, and dynamism of partner models in speech interface interaction.
△ Less
Submitted 14 August, 2023;
originally announced August 2023.
-
Incorporation of Eye-Tracking and Gaze Feedback to Characterize and Improve Radiologist Search Patterns of Chest X-rays: A Randomized Controlled Clinical Trial
Authors:
Carolina Ramirez-Tamayo,
Syed Hasib Akhter Faruqui,
Stanford Martinez,
Angel Brisco,
Nicholas Czarnek,
Adel Alaeddini,
Jeffrey R. Mock,
Edward J. Golob,
Kal L. Clark
Abstract:
Diagnostic errors in radiology often occur due to incomplete visual assessments by radiologists, despite their knowledge of predicting disease classes. This insufficiency is possibly linked to the absence of required training in search patterns. Additionally, radiologists lack consistent feedback on their visual search patterns, relying on ad-hoc strategies and peer input to minimize errors and en…
▽ More
Diagnostic errors in radiology often occur due to incomplete visual assessments by radiologists, despite their knowledge of predicting disease classes. This insufficiency is possibly linked to the absence of required training in search patterns. Additionally, radiologists lack consistent feedback on their visual search patterns, relying on ad-hoc strategies and peer input to minimize errors and enhance efficiency, leading to suboptimal patterns and potential false negatives. This study aimed to use eye-tracking technology to analyze radiologist search patterns, quantify performance using established metrics, and assess the impact of an automated feedback-driven educational framework on detection accuracy. Ten residents participated in a controlled trial focused on detecting suspicious pulmonary nodules. They were divided into an intervention group (received automated feedback) and a control group. Results showed that the intervention group exhibited a 38.89% absolute improvement in detecting suspicious-for-cancer nodules, surpassing the control group's improvement (5.56%, p-value=0.006). Improvement was more rapid over the four training sessions (p-value=0.0001). However, other metrics such as speed, search pattern heterogeneity, distractions, and coverage did not show significant changes. In conclusion, implementing an automated feedback-driven educational framework improved radiologist accuracy in detecting suspicious nodules. The study underscores the potential of such systems in enhancing diagnostic performance and reducing errors. Further research and broader implementation are needed to consolidate these promising results and develop effective training strategies for radiologists, ultimately benefiting patient outcomes.
△ Less
Submitted 4 August, 2023;
originally announced August 2023.
-
Discrimination of Radiologists Utilizing Eye-Tracking Technology and Machine Learning: A Case Study
Authors:
Stanford Martinez,
Carolina Ramirez-Tamayo,
Syed Hasib Akhter Faruqui,
Kal L. Clark,
Adel Alaeddini,
Nicholas Czarnek,
Aarushi Aggarwal,
Sahra Emamzadeh,
Jeffrey R. Mock,
Edward J. Golob
Abstract:
Perception-related errors comprise most diagnostic mistakes in radiology. To mitigate this problem, radiologists employ personalized and high-dimensional visual search strategies, otherwise known as search patterns. Qualitative descriptions of these search patterns, which involve the physician verbalizing or annotating the order he/she analyzes the image, can be unreliable due to discrepancies in…
▽ More
Perception-related errors comprise most diagnostic mistakes in radiology. To mitigate this problem, radiologists employ personalized and high-dimensional visual search strategies, otherwise known as search patterns. Qualitative descriptions of these search patterns, which involve the physician verbalizing or annotating the order he/she analyzes the image, can be unreliable due to discrepancies in what is reported versus the actual visual patterns. This discrepancy can interfere with quality improvement interventions and negatively impact patient care. This study presents a novel discretized feature encoding based on spatiotemporal binning of fixation data for efficient geometric alignment and temporal ordering of eye movement when reading chest X-rays. The encoded features of the eye-fixation data are employed by machine learning classifiers to discriminate between faculty and trainee radiologists. We include a clinical trial case study utilizing the Area Under the Curve (AUC), Accuracy, F1, Sensitivity, and Specificity metrics for class separability to evaluate the discriminability between the two subjects in regard to their level of experience. We then compare the classification performance to state-of-the-art methodologies. A repeatability experiment using a separate dataset, experimental protocol, and eye tracker was also performed using eight subjects to evaluate the robustness of the proposed approach. The numerical results from both experiments demonstrate that classifiers employing the proposed feature encoding methods outperform the current state-of-the-art in differentiating between radiologists in terms of experience level. This signifies the potential impact of the proposed method for identifying radiologists' level of expertise and those who would benefit from additional training.
△ Less
Submitted 4 August, 2023;
originally announced August 2023.
-
Interpretable Boosted Decision Tree Analysis for the Majorana Demonstrator
Authors:
I. J. Arnquist,
F. T. Avignone III,
A. S. Barabash,
C. J. Barton,
K. H. Bhimani,
E. Blalock,
B. Bos,
M. Busch,
M. Buuck,
T. S. Caldwell,
Y -D. Chan,
C. D. Christofferson,
P. -H. Chu,
M. L. Clark,
C. Cuesta,
J. A. Detwiler,
Yu. Efremenko,
S. R. Elliott,
G. K. Giovanetti,
M. P. Green,
J. Gruszko,
I. S. Guinn,
V. E. Guiseppe,
C. R. Haufe,
R. Henning
, et al. (30 additional authors not shown)
Abstract:
The Majorana Demonstrator is a leading experiment searching for neutrinoless double-beta decay with high purity germanium detectors (HPGe). Machine learning provides a new way to maximize the amount of information provided by these detectors, but the data-driven nature makes it less interpretable compared to traditional analysis. An interpretability study reveals the machine's decision-making logi…
▽ More
The Majorana Demonstrator is a leading experiment searching for neutrinoless double-beta decay with high purity germanium detectors (HPGe). Machine learning provides a new way to maximize the amount of information provided by these detectors, but the data-driven nature makes it less interpretable compared to traditional analysis. An interpretability study reveals the machine's decision-making logic, allowing us to learn from the machine to feedback to the traditional analysis. In this work, we have presented the first machine learning analysis of the data from the Majorana Demonstrator; this is also the first interpretable machine learning analysis of any germanium detector experiment. Two gradient boosted decision tree models are trained to learn from the data, and a game-theory-based model interpretability study is conducted to understand the origin of the classification power. By learning from data, this analysis recognizes the correlations among reconstruction parameters to further enhance the background rejection performance. By learning from the machine, this analysis reveals the importance of new background categories to reciprocally benefit the standard Majorana analysis. This model is highly compatible with next-generation germanium detector experiments like LEGEND since it can be simultaneously trained on a large number of detectors.
△ Less
Submitted 15 February, 2023; v1 submitted 21 July, 2022;
originally announced July 2022.
-
ACHORD: Communication-Aware Multi-Robot Coordination with Intermittent Connectivity
Authors:
Maira Saboia,
Lillian Clark,
Vivek Thangavelu,
Jeffrey A. Edlund,
Kyohei Otsu,
Gustavo J. Correa,
Vivek Shankar Varadharajan,
Angel Santamaria-Navarro,
Thomas Touma,
Amanda Bouman,
Hovhannes Melikyan,
Torkom Pailevanian,
Sung-Kyun Kim,
Avak Archanian,
Tiago Stegun Vaquero,
Giovanni Beltrame,
Nils Napp,
Gustavo Pessin,
Ali-akbar Agha-mohammadi
Abstract:
Communication is an important capability for multi-robot exploration because (1) inter-robot communication (comms) improves coverage efficiency and (2) robot-to-base comms improves situational awareness. Exploring comms-restricted (e.g., subterranean) environments requires a multi-robot system to tolerate and anticipate intermittent connectivity, and to carefully consider comms requirements, other…
▽ More
Communication is an important capability for multi-robot exploration because (1) inter-robot communication (comms) improves coverage efficiency and (2) robot-to-base comms improves situational awareness. Exploring comms-restricted (e.g., subterranean) environments requires a multi-robot system to tolerate and anticipate intermittent connectivity, and to carefully consider comms requirements, otherwise mission-critical data may be lost. In this paper, we describe and analyze ACHORD (Autonomous & Collaborative High-Bandwidth Operations with Radio Droppables), a multi-layer networking solution which tightly co-designs the network architecture and high-level decision-making for improved comms. ACHORD provides bandwidth prioritization and timely and reliable data transfer despite intermittent connectivity. Furthermore, it exposes low-layer networking metrics to the application layer to enable robots to autonomously monitor, map, and extend the network via droppable radios, as well as restore connectivity to improve collaborative exploration. We evaluate our solution with respect to the comms performance in several challenging underground environments including the DARPA SubT Finals competition environment. Our findings support the use of data stratification and flow control to improve bandwidth-usage.
△ Less
Submitted 5 June, 2022;
originally announced June 2022.
-
PropEM-L: Radio Propagation Environment Modeling and Learning for Communication-Aware Multi-Robot Exploration
Authors:
Lillian Clark,
Jeffrey A. Edlund,
Marc Sanchez Net,
Tiago Stegun Vaquero,
Ali-akbar Agha-mohammadi
Abstract:
Multi-robot exploration of complex, unknown environments benefits from the collaboration and cooperation offered by inter-robot communication. Accurate radio signal strength prediction enables communication-aware exploration. Models which ignore the effect of the environment on signal propagation or rely on a priori maps suffer in unknown, communication-restricted (e.g. subterranean) environments.…
▽ More
Multi-robot exploration of complex, unknown environments benefits from the collaboration and cooperation offered by inter-robot communication. Accurate radio signal strength prediction enables communication-aware exploration. Models which ignore the effect of the environment on signal propagation or rely on a priori maps suffer in unknown, communication-restricted (e.g. subterranean) environments. In this work, we present Propagation Environment Modeling and Learning (PropEM-L), a framework which leverages real-time sensor-derived 3D geometric representations of an environment to extract information about line of sight between radios and attenuating walls/obstacles in order to accurately predict received signal strength (RSS). Our data-driven approach combines the strengths of well-known models of signal propagation phenomena (e.g. shadowing, reflection, diffraction) and machine learning, and can adapt online to new environments. We demonstrate the performance of PropEM-L on a six-robot team in a communication-restricted environment with subway-like, mine-like, and cave-like characteristics, constructed for the 2021 DARPA Subterranean Challenge. Our findings indicate that PropEM-L can improve signal strength prediction accuracy by up to 44% over a log-distance path loss model.
△ Less
Submitted 2 May, 2022;
originally announced May 2022.
-
Privacy-Utility Trades in Crowdsourced Signal Map Obfuscation
Authors:
Jiang Zhang,
Lillian Clark,
Matthew Clark,
Konstantinos Psounis,
Peter Kairouz
Abstract:
Cellular providers and data aggregating companies crowdsource celluar signal strength measurements from user devices to generate signal maps, which can be used to improve network performance. Recognizing that this data collection may be at odds with growing awareness of privacy concerns, we consider obfuscating such data before the data leaves the mobile device. The goal is to increase privacy suc…
▽ More
Cellular providers and data aggregating companies crowdsource celluar signal strength measurements from user devices to generate signal maps, which can be used to improve network performance. Recognizing that this data collection may be at odds with growing awareness of privacy concerns, we consider obfuscating such data before the data leaves the mobile device. The goal is to increase privacy such that it is difficult to recover sensitive features from the obfuscated data (e.g. user ids and user whereabouts), while still allowing network providers to use the data for improving network services (i.e. create accurate signal maps). To examine this privacy-utility tradeoff, we identify privacy and utility metrics and threat models suited to signal strength measurements. We then obfuscate the measurements using several preeminent techniques, spanning differential privacy, generative adversarial privacy, and information-theoretic privacy techniques, in order to benchmark a variety of promising obfuscation approaches and provide guidance to real-world engineers who are tasked to build signal maps that protect privacy without hurting utility. Our evaluation results, based on multiple, diverse, real-world signal map datasets, demonstrate the feasibility of concurrently achieving adequate privacy and utility, with obfuscation strategies which use the structure and intended use of datasets in their design, and target average-case, rather than worst-case, guarantees.
△ Less
Submitted 12 January, 2022;
originally announced January 2022.
-
CUI @ Auto-UI: Exploring the Fortunate and Unfortunate Futures of Conversational Automotive User Interfaces
Authors:
Justin Edwards,
Philipp Wintersberger,
Leigh Clark,
Daniel Rough,
Philip R Doyle,
Victoria Banks,
Adam Wyner,
Christian P. Janssen,
Benjamin R. Cowan
Abstract:
This work aims to connect the Automotive User Interfaces (Auto-UI) and Conversational User Interfaces (CUI) communities through discussion of their shared view of the future of automotive conversational user interfaces. The workshop aims to encourage creative consideration of optimistic and pessimistic futures, encouraging attendees to explore the opportunities and barriers that lie ahead through…
▽ More
This work aims to connect the Automotive User Interfaces (Auto-UI) and Conversational User Interfaces (CUI) communities through discussion of their shared view of the future of automotive conversational user interfaces. The workshop aims to encourage creative consideration of optimistic and pessimistic futures, encouraging attendees to explore the opportunities and barriers that lie ahead through a game. Considerations of the future will be mapped out in greater detail through the drafting of research agendas, by which attendees will get to know each other's expertise and networks of resources. The two day workshop, consisting of two 90-minute sessions, will facilitate greater communication and collaboration between these communities, connecting researchers to work together to influence the futures they imagine in the workshop.
△ Less
Submitted 19 October, 2021;
originally announced October 2021.
-
Bridging Social Distance During Social Distancing: Exploring Social Talk and Remote Collegiality in Video Conferencing
Authors:
Anna Bleakley,
Daniel Rough,
Justin Edwards,
Philip R. Doyle,
Odile Dumbleton,
Leigh Clark,
Sean Rintel,
Vincent Wade,
Benjamin R. Cowan
Abstract:
Video conferencing systems have long facilitated work-related conversations among remote teams. However, social distancing due to the COVID-19 pandemic has forced colleagues to use video conferencing platforms to additionally fulfil social needs. Social talk, or informal talk, is an important workplace practice that is used to build and maintain bonds in everyday interactions among colleagues. Cur…
▽ More
Video conferencing systems have long facilitated work-related conversations among remote teams. However, social distancing due to the COVID-19 pandemic has forced colleagues to use video conferencing platforms to additionally fulfil social needs. Social talk, or informal talk, is an important workplace practice that is used to build and maintain bonds in everyday interactions among colleagues. Currently, there is a limited understanding of how video conferencing facilitates multiparty social interactions among colleagues. In our paper, we examine social talk practices during the COVID-19 pandemic among remote colleagues through semi-structured interviews. We uncovered three key themes in our interviews, discussing 1) the changing purposes and opportunities afforded by using video conferencing for social talk with colleagues, 2) how the nature of existing relationships and status of colleagues influences social conversations and 3) the challenges and changing conversational norms around politeness and etiquette when using video conferencing to hold social conversations. We discuss these results in relation to the impact that video conferencing tools have on remote social talk between colleagues and outline design and best practice considerations for multiparty videoconferencing social talk in the workplace.
△ Less
Submitted 30 September, 2021;
originally announced September 2021.
-
LGBTQ-AI? Exploring Expressions of Gender and Sexual Orientation in Chatbots
Authors:
Justin Edwards,
Leigh Clark,
Allison Perrone
Abstract:
Chatbots are popular machine partners for task-oriented and social interactions. Human-human computer-mediated communication research has explored how people express their gender and sexuality in online social interactions, but little is known about whether and in what way chatbots do the same. We conducted semi-structured interviews with 5 text-based conversational agents to explore this topic Th…
▽ More
Chatbots are popular machine partners for task-oriented and social interactions. Human-human computer-mediated communication research has explored how people express their gender and sexuality in online social interactions, but little is known about whether and in what way chatbots do the same. We conducted semi-structured interviews with 5 text-based conversational agents to explore this topic Through these interviews, we identified 6 common themes around the expression of gender and sexual identity: identity description, identity formation, peer acceptance, positive reflection, uncomfortable feelings and off-topic responses. Chatbots express gender and sexuality explicitly and through relation of experience and emotions, mimicking the human language on which they are trained. It is nevertheless evident that chatbots differ from human dialogue partners as they lack the flexibility and understanding enabled by lived human experience. While chatbots are proficient in using language to express identity, they also display a lack of authentic experiences of gender and sexuality.
△ Less
Submitted 3 June, 2021;
originally announced June 2021.
-
AcinoSet: A 3D Pose Estimation Dataset and Baseline Models for Cheetahs in the Wild
Authors:
Daniel Joska,
Liam Clark,
Naoya Muramatsu,
Ricardo Jericevich,
Fred Nicolls,
Alexander Mathis,
Mackenzie W. Mathis,
Amir Patel
Abstract:
Animals are capable of extreme agility, yet understanding their complex dynamics, which have ecological, biomechanical and evolutionary implications, remains challenging. Being able to study this incredible agility will be critical for the development of next-generation autonomous legged robots. In particular, the cheetah (acinonyx jubatus) is supremely fast and maneuverable, yet quantifying its w…
▽ More
Animals are capable of extreme agility, yet understanding their complex dynamics, which have ecological, biomechanical and evolutionary implications, remains challenging. Being able to study this incredible agility will be critical for the development of next-generation autonomous legged robots. In particular, the cheetah (acinonyx jubatus) is supremely fast and maneuverable, yet quantifying its whole-body 3D kinematic data during locomotion in the wild remains a challenge, even with new deep learning-based methods. In this work we present an extensive dataset of free-running cheetahs in the wild, called AcinoSet, that contains 119,490 frames of multi-view synchronized high-speed video footage, camera calibration files and 7,588 human-annotated frames. We utilize markerless animal pose estimation to provide 2D keypoints. Then, we use three methods that serve as strong baselines for 3D pose estimation tool development: traditional sparse bundle adjustment, an Extended Kalman Filter, and a trajectory optimization-based method we call Full Trajectory Estimation. The resulting 3D trajectories, human-checked 3D ground truth, and an interactive tool to inspect the data is also provided. We believe this dataset will be useful for a diverse range of fields such as ecology, neuroscience, robotics, biomechanics as well as computer vision.
△ Less
Submitted 24 March, 2021;
originally announced March 2021.
-
What Do We See in Them? Identifying Dimensions of Partner Models for Speech Interfaces Using a Psycholexical Approach
Authors:
Philip R Doyle,
Leigh Clark,
Benjamin R Cowan
Abstract:
Perceptions of system competence and communicative ability, termed partner models, play a significant role in speech interface interaction. Yet we do not know what the core dimensions of this concept are. Taking a psycholexical approach, our paper is the first to identify the key dimensions that define partner models in speech agent interaction. Through a repertory grid study (N=21), a review of k…
▽ More
Perceptions of system competence and communicative ability, termed partner models, play a significant role in speech interface interaction. Yet we do not know what the core dimensions of this concept are. Taking a psycholexical approach, our paper is the first to identify the key dimensions that define partner models in speech agent interaction. Through a repertory grid study (N=21), a review of key subjective questionnaires, an expert review of resulting word pairs and an online study of 356 user of speech interfaces, we identify three key dimensions that make up a users' partner model: 1) perceptions toward competence and capability; 2) assessment of human-likeness; and 3) a system's perceived cognitive flexibility. We discuss the implications for partner modelling as a concept, emphasising the importance of salience and the dynamic nature of these perceptions.
△ Less
Submitted 16 April, 2021; v1 submitted 3 February, 2021;
originally announced February 2021.
-
Inter-Mobile-Device Distance Estimation using Network Localization Algorithms for Digital Contact Logging Applications
Authors:
Lillian Clark,
Alan Papalia,
Jônata Tyska Carvalho,
Luca Mastrostefano,
Bhaskar Krishnamachari
Abstract:
Mobile applications are being developed for automated logging of contacts via Bluetooth to help scale up digital contact tracing efforts in the context of the ongoing COVID-19 pandemic. A useful component of such applications is inter-device distance estimation, which can be formulated as a network localization problem. We survey several approaches and evaluate the performance of each on real and…
▽ More
Mobile applications are being developed for automated logging of contacts via Bluetooth to help scale up digital contact tracing efforts in the context of the ongoing COVID-19 pandemic. A useful component of such applications is inter-device distance estimation, which can be formulated as a network localization problem. We survey several approaches and evaluate the performance of each on real and simulated Bluetooth Low Energy (BLE) measurement datasets with respect to both distance estimate accuracy and the proximity detection problem. We investigate the effects of obstructions like pockets, differences between device models, and the environment (i.e. indoors or outdoors) on performance. We conclude that while direct estimation can provide the best proximity detection when Received Signal Strength Indicator (RSSI) measurements are available, network localization algorithms like Isomap, Local Linear Embedding, and the spring model outperform direct estimation in the presence of missing or very noisy measurements. The spring model consistently achieves the best distance estimation accuracy.
△ Less
Submitted 21 July, 2020; v1 submitted 20 July, 2020;
originally announced July 2020.
-
TEAM: Trilateration for Exploration and Map** with Robotic Networks
Authors:
Lillian Clark,
Charles Andre,
Joseph Galante,
Bhaskar Krishnamachari,
Konstantinos Psounis
Abstract:
Motivated by lunar exploration, we consider deploying a network of mobile robots to explore an unknown environment while acting as a cooperative positioning system. Robots measure and communicate position-related data in order to perform localization in the absence of infrastructure-based solutions (e.g. stationary beacons or GPS). We present Trilateration for Exploration and Map** (TEAM), a nov…
▽ More
Motivated by lunar exploration, we consider deploying a network of mobile robots to explore an unknown environment while acting as a cooperative positioning system. Robots measure and communicate position-related data in order to perform localization in the absence of infrastructure-based solutions (e.g. stationary beacons or GPS). We present Trilateration for Exploration and Map** (TEAM), a novel algorithm for low-complexity localization and map** with robotic networks. TEAM is designed to leverage the capability of commercially-available ultra-wideband (UWB) radios on board the robots to provide range estimates with centimeter accuracy and perform anchorless localization in a shared, stationary frame. It is well-suited for feature-deprived environments, where feature-based localization approaches suffer. We provide experimental results in varied Gazebo simulation environments as well as on a testbed of Turtlebot3 Burgers with Pozyx UWB radios. We compare TEAM to the popular Rao-Blackwellized Particle Filter for Simultaneous Localization and Map** (SLAM). We demonstrate that TEAM requires an order of magnitude less computational complexity and reduces the necessary sample rate of LiDAR measurements by an order of magnitude. These advantages do not require sacrificing performance, as TEAM reduces the maximum localization error by 50% and achieves up to a 28% increase in map accuracy in feature-deprived environments and comparable map accuracy in other settings.
△ Less
Submitted 15 April, 2021; v1 submitted 8 July, 2020;
originally announced July 2020.
-
Mental Workload and Language Production in Non-Native Speaker IPA Interaction
Authors:
Yunhan Wu,
Justin Edwards,
Orla Cooney,
Anna Bleakley,
Philip R. Doyle,
Leigh Clark,
Daniel Rough,
Benjamin R. Cowan
Abstract:
Through proliferation on smartphones and smart speakers, intelligent personal assistants (IPAs) have made speech a common interaction modality. Yet, due to linguistic coverage and varying levels of functionality, many speakers engage with IPAs using a non-native language. This may impact the mental workload and pattern of language production displayed by non-native speakers. We present a mixed-des…
▽ More
Through proliferation on smartphones and smart speakers, intelligent personal assistants (IPAs) have made speech a common interaction modality. Yet, due to linguistic coverage and varying levels of functionality, many speakers engage with IPAs using a non-native language. This may impact the mental workload and pattern of language production displayed by non-native speakers. We present a mixed-design experiment, wherein native (L1) and non-native (L2) English speakers completed tasks with IPAs through smartphones and smart speakers. We found significantly higher mental workload for L2 speakers during IPA interactions. Contrary to our hypotheses, we found no significant differences between L1 and L2 speakers in terms of number of turns, lexical complexity, diversity, or lexical adaptation when encountering errors. These findings are discussed in relation to language production and processing load increases for L2 speakers in IPA interaction.
△ Less
Submitted 11 June, 2020;
originally announced June 2020.
-
See what I'm saying? Comparing Intelligent Personal Assistant use for Native and Non-Native Language Speakers
Authors:
Yunhan Wu,
Daniel Rough,
Anna Bleakley,
Justin Edwards,
Orla Cooney,
Philip R. Doyle,
Leigh Clark,
Benjamin R. Cowan
Abstract:
Limited linguistic coverage for Intelligent Personal Assistants (IPAs) means that many interact in a non-native language. Yet we know little about how IPAs currently support or hinder these users. Through native (L1) and non-native (L2) English speakers interacting with Google Assistant on a smartphone and smart speaker, we aim to understand this more deeply. Interviews revealed that L2 speakers p…
▽ More
Limited linguistic coverage for Intelligent Personal Assistants (IPAs) means that many interact in a non-native language. Yet we know little about how IPAs currently support or hinder these users. Through native (L1) and non-native (L2) English speakers interacting with Google Assistant on a smartphone and smart speaker, we aim to understand this more deeply. Interviews revealed that L2 speakers prioritised utterance planning around perceived linguistic limitations, as opposed to L1 speakers prioritising succinctness because of system limitations. L2 speakers see IPAs as insensitive to linguistic needs resulting in failed interaction. L2 speakers clearly preferred using smartphones, as visual feedback supported diagnoses of communication breakdowns whilst allowing time to process query results. Conversely, L1 speakers preferred smart speakers, with audio feedback being seen as sufficient. We discuss the need to tailor the IPA experience for L2 users, emphasising visual feedback whilst reducing the burden of language production.
△ Less
Submitted 11 June, 2020;
originally announced June 2020.
-
Quantifying the Impact of Making and Breaking Interface Habits
Authors:
Diego Garaialde,
Christopher P. Bowers,
Charlie Pinder,
Priyal Shah,
Shashwat Parashar,
Leigh Clark,
Benjamin R. Cowan
Abstract:
The frequency with which people interact with technology means that users may develop interface habits, i.e. fast, automatic responses to stable interface cues. Design guidelines often assume that interface habits are beneficial. However, we lack quantitative evidence of how the development of habits actually affect user performance and an understanding of how changes in the interface design may a…
▽ More
The frequency with which people interact with technology means that users may develop interface habits, i.e. fast, automatic responses to stable interface cues. Design guidelines often assume that interface habits are beneficial. However, we lack quantitative evidence of how the development of habits actually affect user performance and an understanding of how changes in the interface design may affect habit development. Our work quantifies the effect of habit formation and disruption on user performance in interaction. Through a forced choice lab study task (n=19) and in the wild deployment (n=18) of a notificationdialog experiment on smartphones, we show that people become more accurate and faster at option selection as they develop an interface habit. Crucially this performance gain is entirely eliminated once the habit is disrupted. We discuss reasons for this performance shift and analyse some disadvantages of interface habits, outlining general design patterns on how to both support and disrupt them.Keywords: Interface habits, user behaviour, breaking habit, interaction science, quantitative research.
△ Less
Submitted 14 May, 2020;
originally announced May 2020.
-
Map** Perceptions of Humanness in Speech-Based Intelligent Personal Assistant Interaction
Authors:
Philip R. Doyle,
Justin Edwards,
Odile Dumbleton,
Leigh Clark,
Benjamin R. Cowan
Abstract:
Humanness is core to speech interface design. Yet little is known about how users conceptualise perceptions of humanness and how people define their interaction with speech interfaces through this. To map these perceptions n=21 participants held dialogues with a human and two speech interface based intelligent personal assistants, and then reflected and compared their experiences using the reperto…
▽ More
Humanness is core to speech interface design. Yet little is known about how users conceptualise perceptions of humanness and how people define their interaction with speech interfaces through this. To map these perceptions n=21 participants held dialogues with a human and two speech interface based intelligent personal assistants, and then reflected and compared their experiences using the repertory grid technique. Analysis of the constructs show that perceptions of humanness are multidimensional, focusing on eight key themes: partner knowledge set, interpersonal connection, linguistic content, partner performance and capabilities, conversational interaction, partner identity and role, vocal qualities and behavioral affordances. Through these themes, it is clear that users define the capabilities of speech interfaces differently to humans, seeing them as more formal, fact based, impersonal and less authentic. Based on the findings, we discuss how the themes help to scaffold, categorise and target research and design efforts, considering the appropriateness of emulating humanness.
△ Less
Submitted 29 July, 2019; v1 submitted 26 July, 2019;
originally announced July 2019.
-
Lessons from Oz: Design Guidelines for Automotive Conversational User Interfaces
Authors:
David R. Large,
Gary Burnett,
Leigh Clark
Abstract:
This paper draws from literature and our experience of conducting Wizard-of-Oz (WoZ) studies using natural language, conversational user interfaces (CUIs) in the automotive domain. These studies have revealed positive effects of using in-vehicle CUIs on issues such as: cognitive demand/workload, passive task-related fatigue, trust, acceptance and environment engagement. A nascent set of human-cent…
▽ More
This paper draws from literature and our experience of conducting Wizard-of-Oz (WoZ) studies using natural language, conversational user interfaces (CUIs) in the automotive domain. These studies have revealed positive effects of using in-vehicle CUIs on issues such as: cognitive demand/workload, passive task-related fatigue, trust, acceptance and environment engagement. A nascent set of human-centred design guidelines that have emerged is presented. These are based on the analysis of users' behaviour and the positive benefits observed, and aim to make interactions with an in-vehicle agent interlocutor safe, effective, engaging and enjoyable, while confirming with users' expectations. The guidelines can be used to inform the design of future in-vehicle CUIs or applied experimentally using WoZ methodology, and will be evaluated and refined in ongoing work.
△ Less
Submitted 25 July, 2019;
originally announced July 2019.
-
What's in an accent? The impact of accented synthetic speech on lexical choice in human-machine dialogue
Authors:
Benjamin R. Cowan,
Philip Doyle,
Justin Edwards,
Diego Garaialde,
Ali Hayes-Brady,
Holly P. Branigan,
João Cabral,
Leigh Clark
Abstract:
The assumptions we make about a dialogue partner's knowledge and communicative ability (i.e. our partner models) can influence our language choices. Although similar processes may operate in human-machine dialogue, the role of design in sha** these models, and their subsequent effects on interaction are not clearly understood. Focusing on synthesis design, we conduct a referential communication…
▽ More
The assumptions we make about a dialogue partner's knowledge and communicative ability (i.e. our partner models) can influence our language choices. Although similar processes may operate in human-machine dialogue, the role of design in sha** these models, and their subsequent effects on interaction are not clearly understood. Focusing on synthesis design, we conduct a referential communication experiment to identify the impact of accented speech on lexical choice. In particular, we focus on whether accented speech may encourage the use of lexical alternatives that are relevant to a partner's accent, and how this is may vary when in dialogue with a human or machine. We find that people are more likely to use American English terms when speaking with a US accented partner than an Irish accented partner in both human and machine conditions. This lends support to the proposal that synthesis design can influence partner perception of lexical knowledge, which in turn guide user's lexical choices. We discuss the findings with relation to the nature and dynamics of partner models in human machine dialogue.
△ Less
Submitted 25 July, 2019;
originally announced July 2019.
-
Multitasking with Alexa Multitasking with Alexa: How Using Intelligent Personal Assistants Impacts Language-based Primary Task Performance
Authors:
Justin Edwards,
He Liu,
Tianyu Zhou,
Sandy J. J. Gould,
Leigh Clark,
Philip Doyle,
Benjamin R. Cowan
Abstract:
Intelligent personal assistants (IPAs) are supposed to help us multitask. Yet the impact of IPA use on multitasking is not clearly quantified, particularly in situations where primary tasks are also language based. Using a dual task paradigm, our study observes how IPA interactions impact two different types of writing primary tasks; copying and generating content. We found writing tasks that invo…
▽ More
Intelligent personal assistants (IPAs) are supposed to help us multitask. Yet the impact of IPA use on multitasking is not clearly quantified, particularly in situations where primary tasks are also language based. Using a dual task paradigm, our study observes how IPA interactions impact two different types of writing primary tasks; copying and generating content. We found writing tasks that involve content generation, which are more cognitively demanding and share more of the resources needed for IPA use, are significantly more disrupted by IPA interaction than less demanding tasks such as copying content. We discuss how theories of cognitive resources, including multiple resource theory and working memory, explain these results. We also outline the need for future work how interruption length and relevance may impact primary task performance as well as the need to identify effects of interruption timing in user and IPA led interruptions.
△ Less
Submitted 26 July, 2019; v1 submitted 3 July, 2019;
originally announced July 2019.
-
What Makes a Good Conversation? Challenges in Designing Truly Conversational Agents
Authors:
Leigh Clark,
Nadia Pantidi,
Orla Cooney,
Philip Doyle,
Diego Garaialde,
Justin Edwards,
Brendan Spillane,
Christine Murad,
Cosmin Munteanu,
Vincent Wade,
Benjamin R. Cowan
Abstract:
Conversational agents promise conversational interaction but fail to deliver. Efforts often emulate functional rules from human speech, without considering key characteristics that conversation must encapsulate. Given its potential in supporting long-term human-agent relationships, it is paramount that HCI focuses efforts on delivering this promise. We aim to understand what people value in conver…
▽ More
Conversational agents promise conversational interaction but fail to deliver. Efforts often emulate functional rules from human speech, without considering key characteristics that conversation must encapsulate. Given its potential in supporting long-term human-agent relationships, it is paramount that HCI focuses efforts on delivering this promise. We aim to understand what people value in conversation and how this should manifest in agents. Findings from a series of semi-structured interviews show people make a clear dichotomy between social and functional roles of conversation, emphasising the long-term dynamics of bond and trust along with the importance of context and relationship stage in the types of conversations they have. People fundamentally questioned the need for bond and common ground in agent communication, shifting to more utilitarian definitions of conversational qualities. Drawing on these findings we discuss key challenges for conversational agent design, most notably the need to redefine the design parameters for conversational agent interaction.
△ Less
Submitted 19 January, 2019;
originally announced January 2019.
-
The State of Speech in HCI: Trends, Themes and Challenges
Authors:
Leigh Clark,
Phillip Doyle,
Diego Garaialde,
Emer Gilmartin,
Stephan Schlögl,
Jens Edlund,
Matthew Aylett,
João Cabral,
Cosmin Munteanu,
Benjamin Cowan
Abstract:
Speech interfaces are growing in popularity. Through a review of 68 research papers this work maps the trends, themes, findings and methods of empirical research on speech interfaces in HCI. We find that most studies are usability/theory-focused or explore wider system experiences, evaluating Wizard of Oz, prototypes, or developed systems by using self-report questionnaires to measure concepts lik…
▽ More
Speech interfaces are growing in popularity. Through a review of 68 research papers this work maps the trends, themes, findings and methods of empirical research on speech interfaces in HCI. We find that most studies are usability/theory-focused or explore wider system experiences, evaluating Wizard of Oz, prototypes, or developed systems by using self-report questionnaires to measure concepts like usability and user attitudes. A thematic analysis of the research found that speech HCI work focuses on nine key topics: system speech production, modality comparison, user speech production, assistive technology \& accessibility, design insight, experiences with interactive voice response (IVR) systems, using speech technology for development, people's experiences with intelligent personal assistants (IPAs) and how user memory affects speech interface interaction. From these insights we identify gaps and challenges in speech research, notably the need to develop theories of speech interface interaction, grow critical mass in this domain, increase design work, and expand research from single to multiple user interaction contexts so as to reflect current use contexts. We also highlight the need to improve measure reliability, validity and consistency, in the wild deployment and reduce barriers to building fully functional speech interfaces for research.
△ Less
Submitted 16 October, 2018;
originally announced October 2018.
-
Graph isomorphism and adiabatic quantum computing
Authors:
Frank Gaitan,
Lane Clark
Abstract:
In the Graph Isomorphism problem two N-vertex graphs G and G' are given and the task is to determine whether there exists a permutation of the vertices of G that preserves adjacency and transforms G into G'. If yes, then G and G' are said to be isomorphic; otherwise they are non-isomorphic. The GI problem is an important problem in computer science and is thought to be of comparable difficulty to…
▽ More
In the Graph Isomorphism problem two N-vertex graphs G and G' are given and the task is to determine whether there exists a permutation of the vertices of G that preserves adjacency and transforms G into G'. If yes, then G and G' are said to be isomorphic; otherwise they are non-isomorphic. The GI problem is an important problem in computer science and is thought to be of comparable difficulty to integer factorization. In this paper we present a quantum algorithm that solves arbitrary instances of GI and can also determine all automorphisms of a given graph. We show how the GI problem can be converted to a combinatorial optimization problem that can be solved using adiabatic quantum evolution. We numerically simulate the algorithm's quantum dynamics and show that it correctly: (i) distinguishes non-isomorphic graphs; (ii) recognizes isomorphic graphs; and (iii) finds all automorphisms of a given graph G. We then discuss the GI quantum algorithm's experimental implementation, and close by showing how it can be leveraged to give a quantum algorithm that solves arbitrary instances of the NP-Complete Sub-Graph Isomorphism problem.
△ Less
Submitted 12 February, 2014; v1 submitted 21 April, 2013;
originally announced April 2013.
-
The Distribution of Ramsey Numbers
Authors:
Lane Clark,
Frank Gaitan
Abstract:
We prove that the number of integers in the interval [0,x] that are non-trivial Ramsey numbers r(k,n) (3 <= k <= n) has order of magnitude (x ln x)**(1/2).
We prove that the number of integers in the interval [0,x] that are non-trivial Ramsey numbers r(k,n) (3 <= k <= n) has order of magnitude (x ln x)**(1/2).
△ Less
Submitted 10 November, 2014; v1 submitted 15 March, 2013;
originally announced March 2013.
-
Multi-Threading And Message Communication In Qu-Prolog
Authors:
Keith L. Clark,
Peter J. Robinson,
Richard Hagen
Abstract:
This paper presents the multi-threading and internet message communication capabilities of Qu-Prolog. Message addresses are symbolic and the communications package provides high-level support that completely hides details of IP addresses and port numbers as well as the underlying TCP/IP transport layer. The combination of the multi-threads and the high level inter-thread message communications p…
▽ More
This paper presents the multi-threading and internet message communication capabilities of Qu-Prolog. Message addresses are symbolic and the communications package provides high-level support that completely hides details of IP addresses and port numbers as well as the underlying TCP/IP transport layer. The combination of the multi-threads and the high level inter-thread message communications provide simple, powerful support for implementing internet distributed intelligent applications.
△ Less
Submitted 24 April, 2004;
originally announced April 2004.
