-
Enhancing Saliency Prediction in Monitoring Tasks: The Role of Visual Highlights
Authors:
Zekun Wu,
Anna Maria Feit
Abstract:
This study examines the role of visual highlights in guiding user attention in drone monitoring tasks, employing a simulated interface for observation. The experiment results show that such highlights can significantly expedite the visual attention on the corresponding area. Based on this observation, we leverage both the temporal and spatial information in the highlight to develop a new saliency…
▽ More
This study examines the role of visual highlights in guiding user attention in drone monitoring tasks, employing a simulated interface for observation. The experiment results show that such highlights can significantly expedite the visual attention on the corresponding area. Based on this observation, we leverage both the temporal and spatial information in the highlight to develop a new saliency model: the highlight-informed saliency model (HISM), to infer the visual attention change in the highlight condition. Our findings show the effectiveness of visual highlights in enhancing user attention and demonstrate the potential of incorporating these cues into saliency prediction models.
△ Less
Submitted 15 May, 2024;
originally announced May 2024.
-
Shifting Focus with HCEye: Exploring the Dynamics of Visual Highlighting and Cognitive Load on User Attention and Saliency Prediction
Authors:
Anwesha Das,
Zekun Wu,
Iza Škrjanec,
Anna Maria Feit
Abstract:
Visual highlighting can guide user attention in complex interfaces. However, its effectiveness under limited attentional capacities is underexplored. This paper examines the joint impact of visual highlighting (permanent and dynamic) and dual-task-induced cognitive load on gaze behaviour. Our analysis, using eye-movement data from 27 participants viewing 150 unique webpages reveals that while part…
▽ More
Visual highlighting can guide user attention in complex interfaces. However, its effectiveness under limited attentional capacities is underexplored. This paper examines the joint impact of visual highlighting (permanent and dynamic) and dual-task-induced cognitive load on gaze behaviour. Our analysis, using eye-movement data from 27 participants viewing 150 unique webpages reveals that while participants' ability to attend to UI elements decreases with increasing cognitive load, dynamic adaptations (i.e., highlighting) remain attention-grabbing. The presence of these factors significantly alters what people attend to and thus what is salient. Accordingly, we show that state-of-the-art saliency models increase their performance when accounting for different cognitive loads. Our empirical insights, along with our openly available dataset, enhance our understanding of attentional processes in UIs under varying cognitive (and perceptual) loads and open the door for new models that can predict user attention while multitasking.
△ Less
Submitted 2 May, 2024; v1 submitted 22 April, 2024;
originally announced April 2024.
-
Detecting Relevance during Decision-Making from Eye Movements for UI Adaptation
Authors:
Anna Maria Feit,
Lukas Vordemann,
Seonwook Park,
Caterina Bérubé,
Otmar Hilliges
Abstract:
This paper proposes an approach to detect information relevance during decision-making from eye movements in order to enable user interface adaptation. This is a challenging task because gaze behavior varies greatly across individual users and tasks and groundtruth data is difficult to obtain. Thus, prior work has mostly focused on simpler target-search tasks or on establishing general interest, w…
▽ More
This paper proposes an approach to detect information relevance during decision-making from eye movements in order to enable user interface adaptation. This is a challenging task because gaze behavior varies greatly across individual users and tasks and groundtruth data is difficult to obtain. Thus, prior work has mostly focused on simpler target-search tasks or on establishing general interest, where gaze behavior is less complex. From the literature, we identify six metrics that capture different aspects of the gaze behavior during decision-making and combine them in a voting scheme. We empirically show, that this accounts for the large variations in gaze behavior and out-performs standalone metrics. Importantly, it offers an intuitive way to control the amount of detected information, which is crucial for different UI adaptation schemes to succeed. We show the applicability of our approach by develo** a room-search application that changes the visual saliency of content detected as relevant. In an empirical study, we show that it detects up to 97% of relevant elements with respect to user self-reporting, which allows us to meaningfully adapt the interface, as confirmed by participants. Our approach is fast, does not need any explicit user input and can be applied independent of task and user.
△ Less
Submitted 26 July, 2020;
originally announced July 2020.
-
Subgoal Planning Algorithm for Autonomous Vehicle Guidance
Authors:
Andrew Feit,
Bérénice Mettler
Abstract:
Trained humans exhibit highly agile spatial skills, enabling them to operate vehicles with complex dynamics in demanding tasks and conditions. Prior work shows that humans achieve this performance by using strategies such as satisficing, learning hierarchical task structure, and using a library of motion primitive elements. A key aspect of efficient and versatile solutions is the decomposition of…
▽ More
Trained humans exhibit highly agile spatial skills, enabling them to operate vehicles with complex dynamics in demanding tasks and conditions. Prior work shows that humans achieve this performance by using strategies such as satisficing, learning hierarchical task structure, and using a library of motion primitive elements. A key aspect of efficient and versatile solutions is the decomposition of a task into a sequence of smaller tasks, represented by subgoals. The present work uses properties of constrained optimal control theory to define conditions that specify candidate subgoal states and enable this decomposition. The proposed subgoal algorithm uses graph search to determine a subgoal sequence that links a series of unconstrained motion guidance elements into a constrained solution trajectory. In simulation experiments, the subgoal guidance algorithm generates paths with higher performance and less computation time than an RRT* benchmark. Examples illustrate the robustness and versatility of this approach in multiple environment types.
△ Less
Submitted 26 April, 2020;
originally announced April 2020.
-
AdaM: Adapting Multi-User Interfaces for Collaborative Environments in Real-Time
Authors:
Seonwook Park,
Christoph Gebhardt,
Roman Rädle,
Anna Feit,
Hana Vrzakova,
Niraj Dayama,
Hui-Shyong Yeo,
Clemens Klokmose,
Aaron Quigley,
Antti Oulasvirta,
Otmar Hilliges
Abstract:
Develo** cross-device multi-user interfaces (UIs) is a challenging problem. There are numerous ways in which content and interactivity can be distributed. However, good solutions must consider multiple users, their roles, their preferences and access rights, as well as device capabilities. Manual and rule-based solutions are tedious to create and do not scale to larger problems nor do they adapt…
▽ More
Develo** cross-device multi-user interfaces (UIs) is a challenging problem. There are numerous ways in which content and interactivity can be distributed. However, good solutions must consider multiple users, their roles, their preferences and access rights, as well as device capabilities. Manual and rule-based solutions are tedious to create and do not scale to larger problems nor do they adapt to dynamic changes, such as users leaving or joining an activity. In this paper, we cast the problem of UI distribution as an assignment problem and propose to solve it using combinatorial optimization. We present a mixed integer programming formulation which allows real-time applications in dynamically changing collaborative settings. It optimizes the allocation of UI elements based on device capabilities, user roles, preferences, and access rights. We present a proof-of-concept designer-in-the-loop tool, allowing for quick solution exploration. Finally, we compare our approach to traditional paper prototy** in a lab study.
△ Less
Submitted 29 March, 2018; v1 submitted 3 March, 2018;
originally announced March 2018.
-
First-Person Perceptual Guidance Behavior Decomposition using Active Constraint Classification
Authors:
Andrew Feit,
Berenice Mettler
Abstract:
Humans exhibit a wide range of adaptive and robust dynamic motion behavior that is yet unmatched by autonomous control systems. These capabilities are essential for real-time behavior generation in cluttered environments. Recent work suggests that human capabilities rely on task structure learning and embedded or ecological cognition in the form of perceptual guidance. This paper describes the exp…
▽ More
Humans exhibit a wide range of adaptive and robust dynamic motion behavior that is yet unmatched by autonomous control systems. These capabilities are essential for real-time behavior generation in cluttered environments. Recent work suggests that human capabilities rely on task structure learning and embedded or ecological cognition in the form of perceptual guidance. This paper describes the experimental investigation of the functional elements of human motion guidance, focusing on the control and perceptual mechanisms. The motion, control, and perceptual data from first-person guidance experiments is decomposed into elemental segments based on invariants. These elements are then analyzed to determine their functional characteristics. The resulting model explains the structure of the agent-environment interaction and provides lawful descriptions of specific perceptual guidance and control mechanisms.
△ Less
Submitted 18 October, 2017;
originally announced October 2017.
