-
GazeIntent: Adapting dwell-time selection in VR interaction with real-time intent modeling
Authors:
Anish S. Narkar,
Jan J. Michalak,
Candace E. Peacock,
Brendan David-John
Abstract:
The use of ML models to predict a user's cognitive state from behavioral data has been studied for various applications which includes predicting the intent to perform selections in VR. We developed a novel technique that uses gaze-based intent models to adapt dwell-time thresholds to aid gaze-only selection. A dataset of users performing selection in arithmetic tasks was used to develop intent pr…
▽ More
The use of ML models to predict a user's cognitive state from behavioral data has been studied for various applications which includes predicting the intent to perform selections in VR. We developed a novel technique that uses gaze-based intent models to adapt dwell-time thresholds to aid gaze-only selection. A dataset of users performing selection in arithmetic tasks was used to develop intent prediction models (F1 = 0.94). We developed GazeIntent to adapt selection dwell times based on intent model outputs and conducted an end-user study with returning and new users performing additional tasks with varied selection frequencies. Personalized models for returning users effectively accounted for prior experience and were preferred by 63% of users. Our work provides the field with methods to adapt dwell-based selection to users, account for experience over time, and consider tasks that vary by selection frequency
△ Less
Submitted 21 April, 2024;
originally announced April 2024.
-
Swap It Like Its Hot: Segmentation-based spoof attacks on eye-tracking images
Authors:
Anish S. Narkar,
Brendan David-John
Abstract:
Video-based eye trackers capture the iris biometric and enable authentication to secure user identity. However, biometric authentication is susceptible to spoofing another user's identity through physical or digital manipulation. The current standard to identify physical spoofing attacks on eye-tracking sensors uses liveness detection. Liveness detection classifies gaze data as real or fake, which…
▽ More
Video-based eye trackers capture the iris biometric and enable authentication to secure user identity. However, biometric authentication is susceptible to spoofing another user's identity through physical or digital manipulation. The current standard to identify physical spoofing attacks on eye-tracking sensors uses liveness detection. Liveness detection classifies gaze data as real or fake, which is sufficient to detect physical presentation attacks. However, such defenses cannot detect a spoofing attack when real eye image inputs are digitally manipulated to swap the iris pattern of another person. We propose IrisSwap as a novel attack on gaze-based liveness detection. IrisSwap allows attackers to segment and digitally swap in a victim's iris pattern to fool iris authentication. Both offline and online attacks produce gaze data that deceives the current state-of-the-art defense models at rates up to 58% and motivates the need to develop more advanced authentication methods for eye trackers.
△ Less
Submitted 21 April, 2024;
originally announced April 2024.
-
Securing Bystander Privacy in Mixed Reality While Protecting the User Experience
Authors:
Matthew Corbett,
Brendan David-John,
Jiacheng Shang,
Y. Charlie Hu,
Bo Ji
Abstract:
The modern Mixed Reality devices that make the Metaverse viable require vast information about the physical world and can also violate the privacy of unsuspecting or unwilling bystanders in their vicinity. In this article, we provide an introduction to the problem, existing solutions, and avenues for future research.
The modern Mixed Reality devices that make the Metaverse viable require vast information about the physical world and can also violate the privacy of unsuspecting or unwilling bystanders in their vicinity. In this article, we provide an introduction to the problem, existing solutions, and avenues for future research.
△ Less
Submitted 12 November, 2023; v1 submitted 24 July, 2023;
originally announced July 2023.
-
Eye-tracked Virtual Reality: A Comprehensive Survey on Methods and Privacy Challenges
Authors:
Efe Bozkir,
Süleyman Özdel,
Mengdi Wang,
Brendan David-John,
Hong Gao,
Kevin Butler,
Eakta Jain,
Enkelejda Kasneci
Abstract:
Latest developments in computer hardware, sensor technologies, and artificial intelligence can make virtual reality (VR) and virtual spaces an important part of human everyday life. Eye tracking offers not only a hands-free way of interaction but also the possibility of a deeper understanding of human visual attention and cognitive processes in VR. Despite these possibilities, eye-tracking data al…
▽ More
Latest developments in computer hardware, sensor technologies, and artificial intelligence can make virtual reality (VR) and virtual spaces an important part of human everyday life. Eye tracking offers not only a hands-free way of interaction but also the possibility of a deeper understanding of human visual attention and cognitive processes in VR. Despite these possibilities, eye-tracking data also reveal privacy-sensitive attributes of users when it is combined with the information about the presented stimulus. To address these possibilities and potential privacy issues, in this survey, we first cover major works in eye tracking, VR, and privacy areas between the years 2012 and 2022. While eye tracking in the VR part covers the complete pipeline of eye-tracking methodology from pupil detection and gaze estimation to offline use and analyses, as for privacy and security, we focus on eye-based authentication as well as computational methods to preserve the privacy of individuals and their eye-tracking data in VR. Later, taking all into consideration, we draw three main directions for the research community by mainly focusing on privacy challenges. In summary, this survey provides an extensive literature review of the utmost possibilities with eye tracking in VR and the privacy implications of those possibilities.
△ Less
Submitted 23 May, 2023;
originally announced May 2023.
-
A privacy-preserving approach to streaming eye-tracking data
Authors:
Brendan David-John,
Diane Hosfelt,
Kevin Butler,
Eakta Jain
Abstract:
Eye-tracking technology is being increasingly integrated into mixed reality devices. Although critical applications are being enabled, there are significant possibilities for violating user privacy expectations. We show that there is an appreciable risk of unique user identification even under natural viewing conditions in virtual reality. This identification would allow an app to connect a user's…
▽ More
Eye-tracking technology is being increasingly integrated into mixed reality devices. Although critical applications are being enabled, there are significant possibilities for violating user privacy expectations. We show that there is an appreciable risk of unique user identification even under natural viewing conditions in virtual reality. This identification would allow an app to connect a user's personal ID with their work ID without needing their consent, for example. To mitigate such risks we propose a framework that incorporates gatekee** via the design of the application programming interface and via software-implemented privacy mechanisms. Our results indicate that these mechanisms can reduce the rate of identification from as much as 85% to as low as 30%. The impact of introducing these mechanisms is less than 1.5$^\circ$ error in gaze position for gaze prediction. Gaze data streams can thus be made private while still allowing for gaze prediction, for example, during foveated rendering. Our approach is the first to support privacy-by-design in the flow of eye-tracking data within mixed reality use cases.
△ Less
Submitted 19 March, 2021; v1 submitted 2 February, 2021;
originally announced February 2021.
