-
Personalizing Smart Home Privacy Protection With Individuals' Regulatory Focus: Would You Preserve or Enhance Your Information Privacy?
Authors:
Reza Ghaiumy Anaraky,
Yao Li,
Hichang Cho,
Danny Yuxing Huang,
Kaileigh A. Byrne,
Bart Knijnenburg,
Oded Nov
Abstract:
In this study, we explore the effectiveness of persuasive messages endorsing the adoption of a privacy protection technology (IoT Inspector) tailored to individuals' regulatory focus (promotion or prevention). We explore if and how regulatory fit (i.e., tuning the goal-pursuit mechanism to individuals' internal regulatory focus) can increase persuasion and adoption. We conducted a between-subject…
▽ More
In this study, we explore the effectiveness of persuasive messages endorsing the adoption of a privacy protection technology (IoT Inspector) tailored to individuals' regulatory focus (promotion or prevention). We explore if and how regulatory fit (i.e., tuning the goal-pursuit mechanism to individuals' internal regulatory focus) can increase persuasion and adoption. We conducted a between-subject experiment (N = 236) presenting participants with the IoT Inspector in gain ("Privacy Enhancing Technology" -- PET) or loss ("Privacy Preserving Technology" -- PPT) framing. Results show that the effect of regulatory fit on adoption is mediated by trust and privacy calculus processes: prevention-focused users who read the PPT message trust the tool more. Furthermore, privacy calculus favors using the tool when promotion-focused individuals read the PET message. We discuss the contribution of understanding the cognitive mechanisms behind regulatory fit in privacy decision-making to support privacy protection.
△ Less
Submitted 27 February, 2024;
originally announced February 2024.
-
Putting ChatGPT's Medical Advice to the (Turing) Test
Authors:
Oded Nov,
Nina Singh,
Devin Mann
Abstract:
Objective: Assess the feasibility of using ChatGPT or a similar AI-based chatbot for patient-provider communication. Participants: A US representative sample of 430 study participants aged 18 and above. 53.2% of respondents analyzed were women; their average age was 47.1. Exposure: Ten representative non-administrative patient-provider interactions were extracted from the EHR. Patients' questions…
▽ More
Objective: Assess the feasibility of using ChatGPT or a similar AI-based chatbot for patient-provider communication. Participants: A US representative sample of 430 study participants aged 18 and above. 53.2% of respondents analyzed were women; their average age was 47.1. Exposure: Ten representative non-administrative patient-provider interactions were extracted from the EHR. Patients' questions were placed in ChatGPT with a request for the chatbot to respond using approximately the same word count as the human provider's response. In the survey, each patient's question was followed by a provider- or ChatGPT-generated response. Participants were informed that five responses were provider-generated and five were chatbot-generated. Participants were asked, and incentivized financially, to correctly identify the response source. Participants were also asked about their trust in chatbots' functions in patient-provider communication, using a Likert scale of 1-5. Results: The correct classification of responses ranged between 49.0% to 85.7% for different questions. On average, chatbot responses were correctly identified 65.5% of the time, and provider responses were correctly distinguished 65.1% of the time. On average, responses toward patients' trust in chatbots' functions were weakly positive (mean Likert score: 3.4), with lower trust as the health-related complexity of the task in questions increased. Conclusions: ChatGPT responses to patient questions were weakly distinguishable from provider responses. Laypeople appear to trust the use of chatbots to answer lower risk health questions.
△ Less
Submitted 24 January, 2023;
originally announced January 2023.
-
Think About the Stakeholders First! Towards an Algorithmic Transparency Playbook for Regulatory Compliance
Authors:
Andrew Bell,
Oded Nov,
Julia Stoyanovich
Abstract:
Increasingly, laws are being proposed and passed by governments around the world to regulate Artificial Intelligence (AI) systems implemented into the public and private sectors. Many of these regulations address the transparency of AI systems, and related citizen-aware issues like allowing individuals to have the right to an explanation about how an AI system makes a decision that impacts them. Y…
▽ More
Increasingly, laws are being proposed and passed by governments around the world to regulate Artificial Intelligence (AI) systems implemented into the public and private sectors. Many of these regulations address the transparency of AI systems, and related citizen-aware issues like allowing individuals to have the right to an explanation about how an AI system makes a decision that impacts them. Yet, almost all AI governance documents to date have a significant drawback: they have focused on what to do (or what not to do) with respect to making AI systems transparent, but have left the brunt of the work to technologists to figure out how to build transparent systems. We fill this gap by proposing a novel stakeholder-first approach that assists technologists in designing transparent, regulatory compliant systems. We also describe a real-world case-study that illustrates how this approach can be used in practice.
△ Less
Submitted 10 June, 2022;
originally announced July 2022.
-
An Exploration And Validation of Visual Factors in Understanding Classification Rule Sets
Authors:
Jun Yuan,
Oded Nov,
Enrico Bertini
Abstract:
Rule sets are often used in Machine Learning (ML) as a way to communicate the model logic in settings where transparency and intelligibility are necessary. Rule sets are typically presented as a text-based list of logical statements (rules). Surprisingly, to date there has been limited work on exploring visual alternatives for presenting rules. In this paper, we explore the idea of designing alter…
▽ More
Rule sets are often used in Machine Learning (ML) as a way to communicate the model logic in settings where transparency and intelligibility are necessary. Rule sets are typically presented as a text-based list of logical statements (rules). Surprisingly, to date there has been limited work on exploring visual alternatives for presenting rules. In this paper, we explore the idea of designing alternative representations of rules, focusing on a number of visual factors we believe have a positive impact on rule readability and understanding. We then presents a user study exploring their impact. The results show that some design factors have a strong impact on how efficiently readers can process the rules while having minimal impact on accuracy. This work can help practitioners employ more effective solutions when using rules as a communication strategy to understand ML models.
△ Less
Submitted 19 September, 2021;
originally announced September 2021.
-
A Game-Theoretic Analysis of Competitive Editing in Wikipedia: Contributors' Effort to Influence Articles and the Community's Attempt to Ensure Neutrality
Authors:
Santhanakrishnan Anand,
Ofer Arazy,
Narayan B. Mandayam,
Oded Nov
Abstract:
Peer production, such as the collaborative authoring of Wikipedia articles, involves both cooperation and competition between contributors, and we focus on the latter. As individuals, contributors compete to align Wikipedia articles with their personal perspectives. As a community, they work collectively to ensure a neutral point of view (NPOV). We study the interplay between individuals' competit…
▽ More
Peer production, such as the collaborative authoring of Wikipedia articles, involves both cooperation and competition between contributors, and we focus on the latter. As individuals, contributors compete to align Wikipedia articles with their personal perspectives. As a community, they work collectively to ensure a neutral point of view (NPOV). We study the interplay between individuals' competition and the community's endeavor to ensure neutrality. We develop a two-level game-theoretic model, modeling the interactions of ownership-motivated individuals and neutrality-seeking communal mechanisms as a Stackelberg game. We present our model's predictions regarding the relation between contributors' effort (i.e. typical size of edit) and benefits (i.e. the portion of the article they eventually ``own''). We validate the model's prediction through an empirical analysis, by studying the interactions of 219,811 distinct contributors that co-produced 864 Wikipedia articles over a decade. The analysis and empirical results suggest that contributors who make large edits (``creators'') eventually lose the article's ownership to those who refine the articles and typically make smaller edits (``curators''). Whereas neutrality-seeking mechanisms are essential for ensuring that ownership is not concentrated within a small number of contributors, our findings suggest that the burden of excessive governance may deter contributors from participating.
△ Less
Submitted 17 May, 2021;
originally announced May 2021.
-
Visualizing Rule Sets: Exploration and Validation of a Design Space
Authors:
Jun Yuan,
Oded Nov,
Enrico Bertini
Abstract:
Rule sets are often used in Machine Learning (ML) as a way to communicate the model logic in settings where transparency and intelligibility are necessary. Rule sets are typically presented as a text-based list of logical statements (rules). Surprisingly, to date there has been limited work on exploring visual alternatives for presenting rules. In this paper, we explore the idea of designing alter…
▽ More
Rule sets are often used in Machine Learning (ML) as a way to communicate the model logic in settings where transparency and intelligibility are necessary. Rule sets are typically presented as a text-based list of logical statements (rules). Surprisingly, to date there has been limited work on exploring visual alternatives for presenting rules. In this paper, we explore the idea of designing alternative representations of rules, focusing on a number of visual factors we believe have a positive impact on rule readability and understanding. The paper presents an initial design space for visualizing rule sets and a user study exploring their impact. The results show that some design factors have a strong impact on how efficiently readers can process the rules while having minimal impact on accuracy. This work can help practitioners employ more effective solutions when using rules as a communication strategy to understand ML models.
△ Less
Submitted 4 March, 2021; v1 submitted 1 March, 2021;
originally announced March 2021.
-
Interface Features and Users' Well-Being: Measuring the Sensitivity of Users' Well-Being to Resource Constraints and Feature Types
Authors:
Oded Nov
Abstract:
Users increasingly face multiple interface features on one hand, and constraints on available resources (e.g., time, attention) on the other. Understanding the sensitivity of users' well-being to feature type and resource constraints, is critical for informed design. Building on microeconomic theory, and focusing on social information features, users' interface choices were conceptualized as an ex…
▽ More
Users increasingly face multiple interface features on one hand, and constraints on available resources (e.g., time, attention) on the other. Understanding the sensitivity of users' well-being to feature type and resource constraints, is critical for informed design. Building on microeconomic theory, and focusing on social information features, users' interface choices were conceptualized as an exchange of resources (e.g., time), in return for access to goods (social information features). We studied how sensitive users' well-being is to features' type, and to their cost level and type. We found that (1) increased cost of feature use leads to decreased well-being, (2) users' well-being is a function of features' cost type, and (3) users' well-being is sensitive to differences in feature type. The approach used here to quantify user well-being derived from interface features offers a basis for asynchronous feature comparison.
△ Less
Submitted 1 January, 2021;
originally announced January 2021.
-
SONYC-UST-V2: An Urban Sound Tagging Dataset with Spatiotemporal Context
Authors:
Mark Cartwright,
Jason Cramer,
Ana Elisa Mendez Mendez,
Yu Wang,
Ho-Hsiang Wu,
Vincent Lostanlen,
Magdalena Fuentes,
Graham Dove,
Charlie Mydlarz,
Justin Salamon,
Oded Nov,
Juan Pablo Bello
Abstract:
We present SONYC-UST-V2, a dataset for urban sound tagging with spatiotemporal information. This dataset is aimed for the development and evaluation of machine listening systems for real-world urban noise monitoring. While datasets of urban recordings are available, this dataset provides the opportunity to investigate how spatiotemporal metadata can aid in the prediction of urban sound tags. SONYC…
▽ More
We present SONYC-UST-V2, a dataset for urban sound tagging with spatiotemporal information. This dataset is aimed for the development and evaluation of machine listening systems for real-world urban noise monitoring. While datasets of urban recordings are available, this dataset provides the opportunity to investigate how spatiotemporal metadata can aid in the prediction of urban sound tags. SONYC-UST-V2 consists of 18510 audio recordings from the "Sounds of New York City" (SONYC) acoustic sensor network, including the timestamp of audio acquisition and location of the sensor. The dataset contains annotations by volunteers from the Zooniverse citizen science platform, as well as a two-stage verification with our team. In this article, we describe our data collection procedure and propose evaluation metrics for multilabel classification of urban sound tags. We report the results of a simple baseline model that exploits spatiotemporal information.
△ Less
Submitted 10 September, 2020;
originally announced September 2020.
-
Good for the Many or Best for the Few? A Dilemma in the Design of Algorithmic Advice
Authors:
Graham Dove,
Martina Balestra,
Devin Mann,
Oded Nov
Abstract:
Applications in a range of domains, including route planning and well-being, offer advice based on the social information available in prior users' aggregated activity. When designing these applications, is it better to offer: a) advice that if strictly adhered to is more likely to result in an individual successfully achieving their goal, even if fewer users will choose to adopt it? or b) advice…
▽ More
Applications in a range of domains, including route planning and well-being, offer advice based on the social information available in prior users' aggregated activity. When designing these applications, is it better to offer: a) advice that if strictly adhered to is more likely to result in an individual successfully achieving their goal, even if fewer users will choose to adopt it? or b) advice that is likely to be adopted by a larger number of users, but which is sub-optimal with regard to any particular individual achieving their goal? We identify this dilemma, characterized as Goal-Directed vs. Adoption-Directed advice, and investigate the design questions it raises through an online experiment undertaken in four advice domains (financial investment, making healthier lifestyle choices, route planning, training for a 5k run), with three user types, and across two levels of uncertainty. We report findings that suggest a preference for advice favoring individual goal attainment over higher user adoption rates, albeit with significant variation across advice domains; and discuss their design implications.
△ Less
Submitted 27 August, 2020;
originally announced August 2020.
-
The Transformation of Patient-Clinician Relationships With AI-Based Medical Advice: A "Bring Your Own Algorithm" Era in Healthcare
Authors:
Oded Nov,
Yindalon Aphinyanaphongs,
Yvonne W. Lui,
Devin Mann,
Maurizio Porfiri,
Mark Riedl,
John-Ross Rizzo,
Batia Wiesenfeld
Abstract:
One of the dramatic trends at the intersection of computing and healthcare has been patients' increased access to medical information, ranging from self-tracked physiological data to genetic data, tests, and scans. Increasingly however, patients and clinicians have access to advanced machine learning-based tools for diagnosis, prediction, and recommendation based on large amounts of data, some of…
▽ More
One of the dramatic trends at the intersection of computing and healthcare has been patients' increased access to medical information, ranging from self-tracked physiological data to genetic data, tests, and scans. Increasingly however, patients and clinicians have access to advanced machine learning-based tools for diagnosis, prediction, and recommendation based on large amounts of data, some of it patient-generated. Consequently, just as organizations have had to deal with a "Bring Your Own Device" (BYOD) reality in which employees use their personal devices (phones and tablets) for some aspects of their work, a similar reality of "Bring Your Own Algorithm" (BYOA) is emerging in healthcare with its own challenges and support demands. BYOA is changing patient-clinician interactions and the technologies, skills and workflows related to them. In this paper we argue that: (1) BYOA is changing the patient-clinician relationship and the nature of expert work in healthcare, and (2) better patient-clinician-information-interpretation relationships can be facilitated with solutions that integrate technological and organizational perspectives.
△ Less
Submitted 13 August, 2020;
originally announced August 2020.
-
Evaluation of a Recommender System for Assisting Novice Game Designers
Authors:
Tiago Machado,
Daniel Gopstein,
Oded Nov,
Angela Wang,
Andy Nealen,
Julian Togelius
Abstract:
Game development is a complex task involving multiple disciplines and technologies. Developers and researchers alike have suggested that AI-driven game design assistants may improve developer workflow. We present a recommender system for assisting humans in game design as well as a rigorous human subjects study to validate it. The AI-driven game design assistance system suggests game mechanics to…
▽ More
Game development is a complex task involving multiple disciplines and technologies. Developers and researchers alike have suggested that AI-driven game design assistants may improve developer workflow. We present a recommender system for assisting humans in game design as well as a rigorous human subjects study to validate it. The AI-driven game design assistance system suggests game mechanics to designers based on characteristics of the game being developed. We believe this method can bring creative insights and increase users' productivity. We conducted quantitative studies that showed the recommender system increases users' levels of accuracy and computational affect, and decreases their levels of workload.
△ Less
Submitted 13 August, 2019;
originally announced August 2019.
-
SONYC: A System for the Monitoring, Analysis and Mitigation of Urban Noise Pollution
Authors:
Juan Pablo Bello,
Claudio Silva,
Oded Nov,
R. Luke DuBois,
Anish Arora,
Justin Salamon,
Charles Mydlarz,
Harish Doraiswamy
Abstract:
We present the Sounds of New York City (SONYC) project, a smart cities initiative focused on develo** a cyber-physical system for the monitoring, analysis and mitigation of urban noise pollution. Noise pollution is one of the topmost quality of life issues for urban residents in the U.S. with proven effects on health, education, the economy, and the environment. Yet, most cities lack the resourc…
▽ More
We present the Sounds of New York City (SONYC) project, a smart cities initiative focused on develo** a cyber-physical system for the monitoring, analysis and mitigation of urban noise pollution. Noise pollution is one of the topmost quality of life issues for urban residents in the U.S. with proven effects on health, education, the economy, and the environment. Yet, most cities lack the resources to continuously monitor noise and understand the contribution of individual sources, the tools to analyze patterns of noise pollution at city-scale, and the means to empower city agencies to take effective, data-driven action for noise mitigation. The SONYC project advances novel technological and socio-technical solutions that help address these needs.
SONYC includes a distributed network of both sensors and people for large-scale noise monitoring. The sensors use low-cost, low-power technology, and cutting-edge machine listening techniques, to produce calibrated acoustic measurements and recognize individual sound sources in real time. Citizen science methods are used to help urban residents connect to city agencies and each other, understand their noise footprint, and facilitate reporting and self-regulation. Crucially, SONYC utilizes big data solutions to analyze, retrieve and visualize information from sensors and citizens, creating a comprehensive acoustic model of the city that can be used to identify significant patterns of noise pollution. These data can be used to drive the strategic application of noise code enforcement by city agencies to optimize the reduction of noise pollution. The entire system, integrating cyber, physical and social infrastructure, forms a closed loop of continuous sensing, analysis and actuation on the environment.
SONYC provides a blueprint for the mitigation of noise pollution that can potentially be applied to other cities in the US and abroad.
△ Less
Submitted 18 May, 2018; v1 submitted 2 May, 2018;
originally announced May 2018.
-
Using Interactive Information Labels to Assist Decision Making Under Uncertainty: The Case for Long-term Saving
Authors:
Junius Gunaratne,
Oded Nov
Abstract:
Product information labels can help users understand complex information leading them to make better decisions. One area where consumers are particularly prone to make costly decision-making errors is long-term saving, which requires understanding of complex concepts such as uncertainty and trade-offs. While most people are poorly equipped to deal with such concepts, interactive design can potenti…
▽ More
Product information labels can help users understand complex information leading them to make better decisions. One area where consumers are particularly prone to make costly decision-making errors is long-term saving, which requires understanding of complex concepts such as uncertainty and trade-offs. While most people are poorly equipped to deal with such concepts, interactive design can potentially help users make better decisions. We developed an interactive information label to assist consumers with retirement saving decision-making. To evaluate it, we exposed 382 users to one of three user interface conditions in a retirement saving simulator where they made 35 yearly decisions under changing circumstances. We found significantly better ability of users to reach their goals with the information label. Furthermore, users who interacted with the label made better decisions than those who were presented with a static information label. Lastly, we found the label particularly effective in hel** novice savers.
△ Less
Submitted 21 February, 2017; v1 submitted 17 July, 2015;
originally announced July 2015.
-
A Game Theoretic Analysis of Collaboration in Wikipedia
Authors:
S. Anand,
Ofer Arazy,
Narayan Mandayam,
Oded Nov
Abstract:
Peer production projects such as Wikipedia or open-source software development allow volunteers to collectively create knowledge based products. The inclusive nature of such projects poses difficult challenges for ensuring trustworthiness and combating vandalism. Prior studies in the area deal with descriptive aspects of peer production, failing to capture the idea that while contributors collabor…
▽ More
Peer production projects such as Wikipedia or open-source software development allow volunteers to collectively create knowledge based products. The inclusive nature of such projects poses difficult challenges for ensuring trustworthiness and combating vandalism. Prior studies in the area deal with descriptive aspects of peer production, failing to capture the idea that while contributors collaborate, they also compete for status in the community and for imposing their views on the product. In this paper we investigate collaborative authoring in Wikipedia where contributors append and overwrite previous contributions to a page. We assume that a contributors goal is to maximize ownership of content sections such that content owned (or originated) by her survived the most recent revision of the page. We model contributors interactions to increase their content ownership as a noncooperative game where a players utility is associated with content owned and cost is a function of effort expended. Our results capture several real life aspects of contributors interactions within peer production projects. We show that at the Nash equilibrium there is an inverse relationship between the effort required to make a contribution and the survival of a contributors content. In other words majority of the content that survives is necessarily contributed by experts who expend relatively less effort than non experts. An empirical analysis of Wikipedia articles provides support for our models predictions. Implications for research and practice are discussed in the context of trustworthy collaboration as well as vandalism.
△ Less
Submitted 4 February, 2014;
originally announced February 2014.
