-
A Contextual Inquiry of People with Vision Impairments in Cooking
Authors:
Franklin Mingzhe Li,
Michael Xieyang Liu,
Shaun K. Kane,
Patrick Carrington
Abstract:
Individuals with vision impairments employ a variety of strategies for object identification, such as pans or soy sauce, in the culinary process. In addition, they often rely on contextual details about objects, such as location, orientation, and current status, to autonomously execute cooking activities. To understand how people with vision impairments collect and use the contextual information o…
▽ More
Individuals with vision impairments employ a variety of strategies for object identification, such as pans or soy sauce, in the culinary process. In addition, they often rely on contextual details about objects, such as location, orientation, and current status, to autonomously execute cooking activities. To understand how people with vision impairments collect and use the contextual information of objects while cooking, we conducted a contextual inquiry study with 12 participants in their own kitchens. This research aims to analyze object interaction dynamics in culinary practices to enhance assistive vision technologies for visually impaired cooks. We outline eight different types of contextual information and the strategies that blind cooks currently use to access the information while preparing meals. Further, we discuss preferences for communicating contextual information about kitchen objects as well as considerations for the deployment of AI-powered assistive technologies.
△ Less
Submitted 23 February, 2024;
originally announced February 2024.
-
Preventing Machine Learning Poisoning Attacks Using Authentication and Provenance
Authors:
Jack W. Stokes,
Paul England,
Kevin Kane
Abstract:
Recent research has successfully demonstrated new types of data poisoning attacks. To address this problem, some researchers have proposed both offline and online data poisoning detection defenses which employ machine learning algorithms to identify such attacks. In this work, we take a different approach to preventing data poisoning attacks which relies on cryptographically-based authentication a…
▽ More
Recent research has successfully demonstrated new types of data poisoning attacks. To address this problem, some researchers have proposed both offline and online data poisoning detection defenses which employ machine learning algorithms to identify such attacks. In this work, we take a different approach to preventing data poisoning attacks which relies on cryptographically-based authentication and provenance to ensure the integrity of the data used to train a machine learning model. The same approach is also used to prevent software poisoning and model poisoning attacks. A software poisoning attack maliciously alters one or more software components used to train a model. Once the model has been trained it can also be protected against model poisoning attacks which seek to alter a model's predictions by modifying its underlying parameters or structure. Finally, an evaluation set or test set can also be protected to provide evidence if they have been modified by a second data poisoning attack. To achieve these goals, we propose VAMP which extends the previously proposed AMP system, that was designed to protect media objects such as images, video files or audio clips, to the machine learning setting. We first provide requirements for authentication and provenance for a secure machine learning system. Next, we demonstrate how VAMP's manifest meets these requirements to protect a machine learning system's datasets, software components, and models.
△ Less
Submitted 20 May, 2021;
originally announced May 2021.
-
AMP: Authentication of Media via Provenance
Authors:
Paul England,
Henrique S. Malvar,
Eric Horvitz,
Jack W. Stokes,
Cédric Fournet,
Rebecca Burke-Aguero,
Amaury Chamayou,
Sylvan Clebsch,
Manuel Costa,
John Deutscher,
Shabnam Erfani,
Matt Gaylor,
Andrew Jenks,
Kevin Kane,
Elissa Redmiles,
Alex Shamis,
Isha Sharma,
Sam Wenker,
Anika Zaman
Abstract:
Advances in graphics and machine learning have led to the general availability of easy-to-use tools for modifying and synthesizing media. The proliferation of these tools threatens to cast doubt on the veracity of all media. One approach to thwarting the flow of fake media is to detect modified or synthesized media through machine learning methods. While detection may help in the short term, we be…
▽ More
Advances in graphics and machine learning have led to the general availability of easy-to-use tools for modifying and synthesizing media. The proliferation of these tools threatens to cast doubt on the veracity of all media. One approach to thwarting the flow of fake media is to detect modified or synthesized media through machine learning methods. While detection may help in the short term, we believe that it is destined to fail as the quality of fake media generation continues to improve. Soon, neither humans nor algorithms will be able to reliably distinguish fake versus real content. Thus, pipelines for assuring the source and integrity of media will be required---and increasingly relied upon. We propose AMP, a system that ensures the authentication of media via certifying provenance. AMP creates one or more publisher-signed manifests for a media instance uploaded by a content provider. These manifests are stored in a database allowing fast lookup from applications such as browsers. For reference, the manifests are also registered and signed by a permissioned ledger, implemented using the Confidential Consortium Framework (CCF). CCF employs both software and hardware techniques to ensure the integrity and transparency of all registered manifests. AMP, through its use of CCF, enables a consortium of media providers to govern the service while making all its operations auditable. The authenticity of the media can be communicated to the user via visual elements in the browser, indicating that an AMP manifest has been successfully located and verified.
△ Less
Submitted 20 June, 2020; v1 submitted 22 January, 2020;
originally announced January 2020.
