-
Holon: a cybernetic interface for bio-semiotics
Authors:
Jon McCormack,
Elliott Wilson
Abstract:
This paper presents an interactive artwork, "Holon", a collection of 130 autonomous, cybernetic organisms that listen and make sound in collaboration with the natural environment. The work was developed for installation on water at a heritage-listed dock in Melbourne, Australia. Conceptual issues informing the work are presented, along with a detailed technical overview of the implementation. Indi…
▽ More
This paper presents an interactive artwork, "Holon", a collection of 130 autonomous, cybernetic organisms that listen and make sound in collaboration with the natural environment. The work was developed for installation on water at a heritage-listed dock in Melbourne, Australia. Conceptual issues informing the work are presented, along with a detailed technical overview of the implementation. Individual holons are of three types, inspired by biological models of animal communication: composer/generators, collector/critics and disruptors. Collectively, Holon integrates and occupies elements of the acoustic spectrum in collaboration with human and non-human agents.
△ Less
Submitted 5 April, 2024;
originally announced April 2024.
-
Privacy-Preserving Gaze Data Streaming in Immersive Interactive Virtual Reality: Robustness and User Experience
Authors:
Ethan Wilson,
Azim Ibragimov,
Michael J. Proulx,
Sai Deep Tetali,
Kevin Butler,
Eakta Jain
Abstract:
Eye tracking is routinely being incorporated into virtual reality (VR) systems. Prior research has shown that eye tracking data, if exposed, can be used for re-identification attacks. The state of our knowledge about currently existing privacy mechanisms is limited to privacy-utility trade-off curves based on data-centric metrics of utility, such as prediction error, and black-box threat models. W…
▽ More
Eye tracking is routinely being incorporated into virtual reality (VR) systems. Prior research has shown that eye tracking data, if exposed, can be used for re-identification attacks. The state of our knowledge about currently existing privacy mechanisms is limited to privacy-utility trade-off curves based on data-centric metrics of utility, such as prediction error, and black-box threat models. We propose that for interactive VR applications, it is essential to consider user-centric notions of utility and a variety of threat models. We develop a methodology to evaluate real-time privacy mechanisms for interactive VR applications that incorporate subjective user experience and task performance metrics. We evaluate selected privacy mechanisms using this methodology and find that re-identification accuracy can be decreased to as low as 14% while maintaining a high usability score and reasonable task performance. Finally, we elucidate three threat scenarios (black-box, black-box with exemplars, and white-box) and assess how well the different privacy mechanisms hold up to these adversarial scenarios.
This work advances the state of the art in VR privacy by providing a methodology for end-to-end assessment of the risk of re-identification attacks and potential mitigating solutions.
△ Less
Submitted 21 February, 2024; v1 submitted 12 February, 2024;
originally announced February 2024.
-
Towards mitigating uncann(eye)ness in face swaps via gaze-centric loss terms
Authors:
Ethan Wilson,
Frederick Shic,
Sophie Jörg,
Eakta Jain
Abstract:
Advances in face swap** have enabled the automatic generation of highly realistic faces. Yet face swaps are perceived differently than when looking at real faces, with key differences in viewer behavior surrounding the eyes. Face swap** algorithms generally place no emphasis on the eyes, relying on pixel or feature matching losses that consider the entire face to guide the training process. We…
▽ More
Advances in face swap** have enabled the automatic generation of highly realistic faces. Yet face swaps are perceived differently than when looking at real faces, with key differences in viewer behavior surrounding the eyes. Face swap** algorithms generally place no emphasis on the eyes, relying on pixel or feature matching losses that consider the entire face to guide the training process. We further investigate viewer perception of face swaps, focusing our analysis on the presence of an uncanny valley effect. We additionally propose a novel loss equation for the training of face swap** models, leveraging a pretrained gaze estimation network to directly improve representation of the eyes. We confirm that viewed face swaps do elicit uncanny responses from viewers. Our proposed improvements significant reduce viewing angle errors between face swaps and their source material. Our method additionally reduces the prevalence of the eyes as a deciding factor when viewers perform deepfake detection tasks. Our findings have implications on face swap** for special effects, as digital avatars, as privacy mechanisms, and more; negative responses from users could limit effectiveness in said applications. Our gaze improvements are a first step towards alleviating negative viewer perceptions via a targeted approach.
△ Less
Submitted 5 February, 2024;
originally announced February 2024.
-
Transdisciplinary AI Education: The Confluence of Curricular and Community Needs in the Instruction of Artificial Intelligence
Authors:
Roozbeh Aliabadi,
Aditi Singh,
Eryka Wilson
Abstract:
The integration of artificial intelligence (AI) into education has the potential to transform the way we learn and teach. In this paper, we examine the current state of AI in education and explore the potential benefits and challenges of incorporating this technology into the classroom. The approaches currently available for AI education often present students with experiences only focusing on dis…
▽ More
The integration of artificial intelligence (AI) into education has the potential to transform the way we learn and teach. In this paper, we examine the current state of AI in education and explore the potential benefits and challenges of incorporating this technology into the classroom. The approaches currently available for AI education often present students with experiences only focusing on discrete computer science concepts agnostic to a larger curriculum. However, teaching AI must not be siloed or interdisciplinary. Rather, AI instruction ought to be transdisciplinary, including connections to the broad curriculum and community in which students are learning. This paper delves into the AI program currently in development for Neom Community School and the larger Education, Research, and Innovation Sector in Neom, Saudi Arabia s new megacity under development. In this program, AI is both taught as a subject and to learn other subjects within the curriculum through the school systems International Baccalaureate (IB) approach, which deploys learning through Units of Inquiry. This approach to education connects subjects across a curriculum under one major guiding question at a time. The proposed method offers a meaningful approach to introducing AI to students throughout these Units of Inquiry, as it shifts AI from a subject that students like or not like to a subject that is taught throughout the curriculum.
△ Less
Submitted 10 November, 2023;
originally announced November 2023.
-
Patterns in Deep Time
Authors:
Dave Griffiths,
Elizabeth Wilson,
Iván Paz,
Alex McLean,
Joana Chicau,
Flor de Fuego,
Timo Hoogland,
Eloi Isern,
Michael-Jon Mizra,
Roger Pibernat
Abstract:
In this paper, we explore how textile pattern-making can be a useful activity for live coders used to manipulating software. We ran an algorithmic patterns workshop in July 2022 -- with a node at "on the fly" festival in Barcelona, a node in Sheffield and the workshop leader in Penryn -- where we created an activity recreating ancient patterns by weaving on tablet looms that we constructed from ca…
▽ More
In this paper, we explore how textile pattern-making can be a useful activity for live coders used to manipulating software. We ran an algorithmic patterns workshop in July 2022 -- with a node at "on the fly" festival in Barcelona, a node in Sheffield and the workshop leader in Penryn -- where we created an activity recreating ancient patterns by weaving on tablet looms that we constructed from card and yarn, and sent to the participants for this remote, multi location workshop. One of the aims of the Algorithmic Pattern project is to highlight the relationship people have had with patterns over history, and how we can use this to uncover certain misconceptions we have about algorithmic patterns in contemporary society. We collected responses from those who participated in the workshop and collated the responses. We found that tablet weaving allows us to connect the physical patterns with their algorithmic descriptions. Also, errors relate with the trade-off among expectations and surprise and exploring new unexpected possibilities. Finally, sharing the experiences among the participants allows us to observe how we interpret patterns when comparing it with other experiences.
△ Less
Submitted 20 July, 2023;
originally announced July 2023.
-
Introducing Explicit Gaze Constraints to Face Swap**
Authors:
Ethan Wilson,
Frederick Shic,
Eakta Jain
Abstract:
Face swap** combines one face's identity with another face's non-appearance attributes (expression, head pose, lighting) to generate a synthetic face. This technology is rapidly improving, but falls flat when reconstructing some attributes, particularly gaze. Image-based loss metrics that consider the full face do not effectively capture the perceptually important, yet spatially small, eye regio…
▽ More
Face swap** combines one face's identity with another face's non-appearance attributes (expression, head pose, lighting) to generate a synthetic face. This technology is rapidly improving, but falls flat when reconstructing some attributes, particularly gaze. Image-based loss metrics that consider the full face do not effectively capture the perceptually important, yet spatially small, eye regions. Improving gaze in face swaps can improve naturalness and realism, benefiting applications in entertainment, human computer interaction, and more. Improved gaze will also directly improve Deepfake detection efforts, serving as ideal training data for classifiers that rely on gaze for classification. We propose a novel loss function that leverages gaze prediction to inform the face swap model during training and compare against existing methods. We find all methods to significantly benefit gaze in resulting face swaps.
△ Less
Submitted 25 May, 2023;
originally announced May 2023.
-
Framework for Certification of AI-Based Systems
Authors:
Maxime Gariel,
Brian Shimanuki,
Rob Timpe,
Evan Wilson
Abstract:
The current certification process for aerospace software is not adapted to "AI-based" algorithms such as deep neural networks. Unlike traditional aerospace software, the precise parameters optimized during neural network training are as important as (or more than) the code processing the network and they are not directly mathematically understandable. Despite their lack of explainability such algo…
▽ More
The current certification process for aerospace software is not adapted to "AI-based" algorithms such as deep neural networks. Unlike traditional aerospace software, the precise parameters optimized during neural network training are as important as (or more than) the code processing the network and they are not directly mathematically understandable. Despite their lack of explainability such algorithms are appealing because for some applications they can exhibit high performance unattainable with any traditional explicit line-by-line software methods.
This paper proposes a framework and principles that could be used to establish certification methods for neural network models for which the current certification processes such as DO-178 cannot be applied. While it is not a magic recipe, it is a set of common sense steps that will allow the applicant and the regulator increase their confidence in the developed software, by demonstrating the capabilities to bring together, trace, and track the requirements, data, software, training process, and test results.
△ Less
Submitted 21 February, 2023;
originally announced February 2023.
-
Agent swarms: cooperation and coordination under stringent communications constraint
Authors:
Paul Kinsler,
Sean Holman,
Andrew Elliott,
Cathryn N. Mitchell,
R. Eddie Wilson
Abstract:
Here we consider the communications tactics appropriate for a group of agents that need to "swarm" together in a highly adversarial environment. Specfically, whilst they need to cooperate by exchanging information with each other about their location and their plans; at the same time they also need to keep such communications to an absolute minimum. This might be due to a need for stealth, or othe…
▽ More
Here we consider the communications tactics appropriate for a group of agents that need to "swarm" together in a highly adversarial environment. Specfically, whilst they need to cooperate by exchanging information with each other about their location and their plans; at the same time they also need to keep such communications to an absolute minimum. This might be due to a need for stealth, or otherwise be relevant to situations where communications are signficantly restricted. Complicating this process is that we assume each agent has (a) no means of passively locating others, (b) it must rely on being updated by reception of appropriate messages; and if no such update messages arrive, (c) then their own beliefs about other agents will gradually become out of date and increasingly inaccurate. Here we use a geometry-free multi-agent model that is capable of allowing for message-based information transfer between agents with different intrinsic connectivities, as would be present in a spatial arrangement of agents. We present agent-centric performance metrics that require only minimal assumptions, and show how simulated outcome distributions, risks, and connectivities depend on the ratio of information gain to loss. We also show that checking for too-long round-trip times can be an effective minimal-information filter for determining which agents to no longer target with messages.
△ Less
Submitted 6 April, 2023; v1 submitted 3 October, 2022;
originally announced October 2022.
-
Practical Digital Disguises: Leveraging Face Swaps to Protect Patient Privacy
Authors:
Ethan Wilson,
Frederick Shic,
Jenny Skytta,
Eakta Jain
Abstract:
With rapid advancements in image generation technology, face swap** for privacy protection has emerged as an active area of research. The ultimate benefit is improved access to video datasets, e.g. in healthcare settings. Recent literature has proposed deep network-based architectures to perform facial swaps and reported the associated reduction in facial recognition accuracy. However, there is…
▽ More
With rapid advancements in image generation technology, face swap** for privacy protection has emerged as an active area of research. The ultimate benefit is improved access to video datasets, e.g. in healthcare settings. Recent literature has proposed deep network-based architectures to perform facial swaps and reported the associated reduction in facial recognition accuracy. However, there is not much reporting on how well these methods preserve the types of semantic information needed for the privatized videos to remain useful for their intended application. Our main contribution is a novel end-to-end face swap** pipeline for recorded videos of standardized assessments of autism symptoms in children. Through this design, we are the first to provide a methodology for assessing the privacy-utility trade-offs for the face swap** approach to patient privacy protection. Our methodology can show, for example, that current deep network based face swap** is bottle-necked by face detection in real world videos, and the extent to which gaze and expression information is preserved by face swaps relative to baseline privatization methods such as blurring.
△ Less
Submitted 13 April, 2022; v1 submitted 7 April, 2022;
originally announced April 2022.
-
Empirical Evaluation of Circuit Approximations on Noisy Quantum Devices
Authors:
Ellis Wilson,
Frank Mueller,
Lindsay Bassman,
Constin Iancu
Abstract:
Noisy Intermediate-Scale Quantum (NISQ) devices fail to produce outputs with sufficient fidelity for deep circuits with many gates today. Such devices suffer from read-out, multi-qubit gate and crosstalk noise combined with short decoherence times limiting circuit depth. This work develops a methodology to generate shorter circuits with fewer multi-qubit gates whose unitary transformations approxi…
▽ More
Noisy Intermediate-Scale Quantum (NISQ) devices fail to produce outputs with sufficient fidelity for deep circuits with many gates today. Such devices suffer from read-out, multi-qubit gate and crosstalk noise combined with short decoherence times limiting circuit depth. This work develops a methodology to generate shorter circuits with fewer multi-qubit gates whose unitary transformations approximate the original reference one. It explores the benefit of such generated approximations under NISQ devices. Experimental results with Grover's algorithm, multiple-control Toffoli gates, and the Transverse Field Ising Model show that such approximate circuits produce higher fidelity results than longer, theoretically precise circuits on NISQ devices, especially when the reference circuits have many CNOT gates to begin with. With this ability to fine-tune circuits, it is demonstrated that quantum computations can be performed for more complex problems on today's devices than was feasible before, sometimes even with a gain in overall precision by up to 60%.
△ Less
Submitted 14 July, 2021;
originally announced July 2021.
-
Nine Best Practices for Research Software Registries and Repositories: A Concise Guide
Authors:
Task Force on Best Practices for Software Registries,
:,
Alain Monteil,
Alejandra Gonzalez-Beltran,
Alexandros Ioannidis,
Alice Allen,
Allen Lee,
Anita Bandrowski,
Bruce E. Wilson,
Bryce Mecum,
Cai Fan Du,
Carly Robinson,
Daniel Garijo,
Daniel S. Katz,
David Long,
Genevieve Milliken,
Hervé Ménager,
Jessica Hausman,
Jurriaan H. Spaaks,
Katrina Fenlon,
Kristin Vanderbilt,
Lorraine Hwang,
Lynn Davis,
Martin Fenner,
Michael R. Crusoe
, et al. (8 additional authors not shown)
Abstract:
Scientific software registries and repositories serve various roles in their respective disciplines. These resources improve software discoverability and research transparency, provide information for software citations, and foster preservation of computational methods that might otherwise be lost over time, thereby supporting research reproducibility and replicability. However, develo** these r…
▽ More
Scientific software registries and repositories serve various roles in their respective disciplines. These resources improve software discoverability and research transparency, provide information for software citations, and foster preservation of computational methods that might otherwise be lost over time, thereby supporting research reproducibility and replicability. However, develo** these resources takes effort, and few guidelines are available to help prospective creators of registries and repositories. To address this need, we present a set of nine best practices that can help managers define the scope, practices, and rules that govern individual registries and repositories. These best practices were distilled from the experiences of the creators of existing resources, convened by a Task Force of the FORCE11 Software Citation Implementation Working Group during the years 2019-2020. We believe that putting in place specific policies such as those presented here will help scientific software registries and repositories better serve their users and their disciplines.
△ Less
Submitted 24 December, 2020;
originally announced December 2020.
-
Just-in-time Quantum Circuit Transpilation Reduces Noise
Authors:
Ellis Wilson,
Sudhakar Singh,
Frank Mueller
Abstract:
Running quantum programs is fraught with challenges on on today's noisy intermediate scale quantum (NISQ) devices. Many of these challenges originate from the error characteristics that stem from rapid decoherence and noise during measurement, qubit connections, crosstalk, the qubits themselves, and transformations of qubit state via gates. Not only are qubits not "created equal", but their noise…
▽ More
Running quantum programs is fraught with challenges on on today's noisy intermediate scale quantum (NISQ) devices. Many of these challenges originate from the error characteristics that stem from rapid decoherence and noise during measurement, qubit connections, crosstalk, the qubits themselves, and transformations of qubit state via gates. Not only are qubits not "created equal", but their noise level also changes over time. IBM is said to calibrate their quantum systems once per day and reports noise levels (errors) at the time of such calibration. This information is subsequently used to map circuits to higher quality qubits and connections up to the next calibration point.
This work provides evidence that there is room for improvement over this daily calibration cycle. It contributes a technique to measure noise levels (errors) related to qubits immediately before executing one or more sensitive circuits and shows that just-in-time noise measurements benefit late physical qubit map**s. With this just-in-time recalibrated transpilation, the fidelity of results is improved over IBM's default map**s, which only uses their daily calibrations. The framework assess two major sources of noise, namely readout errors (measurement errors) and two-qubit gate/connection errors. Experiments indicate that the accuracy of circuit results improves by 3-304% on average and up to 400% with on-the-fly circuit map**s based on error measurements just prior to application execution.
△ Less
Submitted 26 May, 2020;
originally announced May 2020.
-
SlicerPET: A workflow based software module for PET/CT guided needle biopsy
Authors:
Dženan Zukić,
Julien Finet,
Emmanuel Wilson,
Filip Banovac,
Giuseppe Esposito,
Kevin Cleary,
Andinet Enquobahrie
Abstract:
Biopsy is commonly used to confirm cancer diagnosis when radiologically indicated. Given the ability of PET to localize malignancies in heterogeneous tumors and tumors that do not have a CT correlate, PET/CT guided biopsy may improve the diagnostic yield of biopsies. To facilitate PET/CT guided needle biopsy, we developed a workflow that allows us to bring PET image guidance into the interventiona…
▽ More
Biopsy is commonly used to confirm cancer diagnosis when radiologically indicated. Given the ability of PET to localize malignancies in heterogeneous tumors and tumors that do not have a CT correlate, PET/CT guided biopsy may improve the diagnostic yield of biopsies. To facilitate PET/CT guided needle biopsy, we developed a workflow that allows us to bring PET image guidance into the interventional CT suite. In this abstract, we present SlicerPET, a user-friendly workflow based module developed using open source software libraries to guide needle biopsy in the interventional suite.
△ Less
Submitted 25 November, 2015;
originally announced November 2015.
