-
Artful Path to Healing: Using Machine Learning for Visual Art Recommendation to Prevent and Reduce Post-Intensive Care
Authors:
Bereket A. Yilma,
Chan Mi Kim,
Gerald C. Cupchik,
Luis A. Leiva
Abstract:
Staying in the intensive care unit (ICU) is often traumatic, leading to post-intensive care syndrome (PICS), which encompasses physical, psychological, and cognitive impairments. Currently, there are limited interventions available for PICS. Studies indicate that exposure to visual art may help address the psychological aspects of PICS and be more effective if it is personalized. We develop Machin…
▽ More
Staying in the intensive care unit (ICU) is often traumatic, leading to post-intensive care syndrome (PICS), which encompasses physical, psychological, and cognitive impairments. Currently, there are limited interventions available for PICS. Studies indicate that exposure to visual art may help address the psychological aspects of PICS and be more effective if it is personalized. We develop Machine Learning-based Visual Art Recommendation Systems (VA RecSys) to enable personalized therapeutic visual art experiences for post-ICU patients. We investigate four state-of-the-art VA RecSys engines, evaluating the relevance of their recommendations for therapeutic purposes compared to expert-curated recommendations. We conduct an expert pilot test and a large-scale user study (n=150) to assess the appropriateness and effectiveness of these recommendations. Our results suggest all recommendations enhance temporal affective states. Visual and multimodal VA RecSys engines compare favourably with expert-curated recommendations, indicating their potential to support the delivery of personalized art therapy for PICS prevention and treatment.
△ Less
Submitted 23 February, 2024;
originally announced February 2024.
-
The Elements of Visual Art Recommendation: Learning Latent Semantic Representations of Paintings
Authors:
Bereket A. Yilma,
Luis A. Leiva
Abstract:
Artwork recommendation is challenging because it requires understanding how users interact with highly subjective content, the complexity of the concepts embedded within the artwork, and the emotional and cognitive reflections they may trigger in users. In this paper, we focus on efficiently capturing the elements (i.e., latent semantic relationships) of visual art for personalized recommendation.…
▽ More
Artwork recommendation is challenging because it requires understanding how users interact with highly subjective content, the complexity of the concepts embedded within the artwork, and the emotional and cognitive reflections they may trigger in users. In this paper, we focus on efficiently capturing the elements (i.e., latent semantic relationships) of visual art for personalized recommendation. We propose and study recommender systems based on textual and visual feature learning techniques, as well as their combinations. We then perform a small-scale and a large-scale user-centric evaluation of the quality of the recommendations. Our results indicate that textual features compare favourably with visual ones, whereas a fusion of both captures the most suitable hidden semantic relationships for artwork recommendation. Ultimately, this paper contributes to our understanding of how to deliver content that suitably matches the user's interests and how they are perceived.
△ Less
Submitted 28 February, 2023;
originally announced March 2023.
-
A Contextual Framework for Adaptive User Interfaces: Modelling the Interaction Environment
Authors:
Mateusz Dubiel,
Bereket Abera Yilma,
Kayhan Latifzadeh,
Luis A. Leiva
Abstract:
The interaction context (or environment) is key to any HCI task and especially to adaptive user interfaces (AUIs), since it represents the conditions under which users interact with computers. Unfortunately, there are currently no formal representations to model said interaction context. In order to address this gap, we propose a contextual framework for AUIs and illustrate a practical applica- ti…
▽ More
The interaction context (or environment) is key to any HCI task and especially to adaptive user interfaces (AUIs), since it represents the conditions under which users interact with computers. Unfortunately, there are currently no formal representations to model said interaction context. In order to address this gap, we propose a contextual framework for AUIs and illustrate a practical applica- tion using learning management systems as a case study. We also discuss limitations of our framework and offer discussion points about the realisation of truly context-aware AUIs.
△ Less
Submitted 31 March, 2022;
originally announced March 2022.
-
Systemic formalisation of Cyber-Physical-Social System (CPSS): A systematic literature review
Authors:
Bereket Abera Yilma,
Hervé Panetto,
Yannick Naudet
Abstract:
The notion of Cyber-Physical-Social System (CPSS) is an emerging concept developed as a result of the need to understand the impact of Cyber-Physical Systems (CPS) on humans and vice versa. This paradigm shift from CPS to CPSS was mainly attributed to the increasing use of sensor-enabled smart devices and the tight link with the users. The concept of CPSS has been around for over a decade and it h…
▽ More
The notion of Cyber-Physical-Social System (CPSS) is an emerging concept developed as a result of the need to understand the impact of Cyber-Physical Systems (CPS) on humans and vice versa. This paradigm shift from CPS to CPSS was mainly attributed to the increasing use of sensor-enabled smart devices and the tight link with the users. The concept of CPSS has been around for over a decade and it has gained increasing attention over the past few years. The evolution to incorporate human aspects in the CPS research has unlocked a number of research challenges. Particularly human dynamics brings additional complexity that is yet to be explored. The exploration to conceptualise the notion of CPSS has been partially addressed in few scientific literatures. Although its conceptualisation has always been use-case dependent. Thus, there is a lack of generic view as most works focus on specific domains. Furthermore, the systemic core and design principles linking it with the theory of systems are loose. This work aims at addressing these issues by first exploring and analysing scientific literature to understand the complete spectrum of CPSS through a Systematic Literature Review (SLR). Thereby identifying the state-of-the-art perspectives on CPSS regarding definitions, underlining principles and application areas. Subsequently, based on the findings of the SLR, we propose a domain-independent definition and a meta-model for CPSS, grounded in the Theory of Systems. Finally, a discussion on feasible future research directions is presented based on the systemic notion and the proposed meta-models.
△ Less
Submitted 11 April, 2021;
originally announced April 2021.
-
Towards a Personalisation Framework for Cyber-Physical-Social System (CPSS)
Authors:
Bereket Abera Yilma,
Yannick Naudet,
Hervé Panetto
Abstract:
A Cyber-Physical-Social System (CPSS) is an emerging paradigm often understood as a physical and virtual space of interaction which is cohabited by humans and sensor-enabled smart devices. In such settings, human interaction behaviour is often different from person to person and is guided by complex environmental and natural factors that are not yet fully explored. Thus, ensuring a seamless human-…
▽ More
A Cyber-Physical-Social System (CPSS) is an emerging paradigm often understood as a physical and virtual space of interaction which is cohabited by humans and sensor-enabled smart devices. In such settings, human interaction behaviour is often different from person to person and is guided by complex environmental and natural factors that are not yet fully explored. Thus, ensuring a seamless human-machine interaction in CPSS calls for efficient means of handling human dynamics and bringing interaction experience to a personal level. To this end in this paper, we propose a personalisation framework to support the design of CPSS in recognising and addressing human/social aspects.
△ Less
Submitted 31 March, 2021; v1 submitted 25 March, 2021;
originally announced March 2021.
-
Personalised Visual Art Recommendation by Learning Latent Semantic Representations
Authors:
Bereket Abera Yilma,
Najib Aghenda,
Marcelo Romero,
Yannick Naudet,
Herve Panetto
Abstract:
In Recommender systems, data representation techniques play a great role as they have the power to entangle, hide and reveal explanatory factors embedded within datasets. Hence, they influence the quality of recommendations. Specifically, in Visual Art (VA) recommendations the complexity of the concepts embodied within paintings, makes the task of capturing semantics by machines far from trivial.…
▽ More
In Recommender systems, data representation techniques play a great role as they have the power to entangle, hide and reveal explanatory factors embedded within datasets. Hence, they influence the quality of recommendations. Specifically, in Visual Art (VA) recommendations the complexity of the concepts embodied within paintings, makes the task of capturing semantics by machines far from trivial. In VA recommendation, prominent works commonly use manually curated metadata to drive recommendations. Recent works in this domain aim at leveraging visual features extracted using Deep Neural Networks (DNN). However, such data representation approaches are resource demanding and do not have a direct interpretation, hindering user acceptance. To address these limitations, we introduce an approach for Personalised Recommendation of Visual arts based on learning latent semantic representation of paintings. Specifically, we trained a Latent Dirichlet Allocation (LDA) model on textual descriptions of paintings. Our LDA model manages to successfully uncover non-obvious semantic relationships between paintings whilst being able to offer explainable recommendations. Experimental evaluations demonstrate that our method tends to perform better than exploiting visual features extracted using pre-trained Deep Neural Networks.
△ Less
Submitted 24 July, 2020;
originally announced August 2020.
