-
Language as a Latent Sequence: deep latent variable models for semi-supervised paraphrase generation
Authors:
Jialin Yu,
Alexandra I. Cristea,
Anoushka Harit,
Zhongtian Sun,
Olanrewaju Tahir Aduragba,
Lei Shi,
Noura Al Moubayed
Abstract:
This paper explores deep latent variable models for semi-supervised paraphrase generation, where the missing target pair for unlabelled data is modelled as a latent paraphrase sequence. We present a novel unsupervised model named variational sequence auto-encoding reconstruction (VSAR), which performs latent sequence inference given an observed text. To leverage information from text pairs, we add…
▽ More
This paper explores deep latent variable models for semi-supervised paraphrase generation, where the missing target pair for unlabelled data is modelled as a latent paraphrase sequence. We present a novel unsupervised model named variational sequence auto-encoding reconstruction (VSAR), which performs latent sequence inference given an observed text. To leverage information from text pairs, we additionally introduce a novel supervised model we call dual directional learning (DDL), which is designed to integrate with our proposed VSAR model. Combining VSAR with DDL (DDL+VSAR) enables us to conduct semi-supervised learning. Still, the combined model suffers from a cold-start problem. To further combat this issue, we propose an improved weight initialisation solution, leading to a novel two-stage training scheme we call knowledge-reinforced-learning (KRL). Our empirical evaluations suggest that the combined model yields competitive performance against the state-of-the-art supervised baselines on complete data. Furthermore, in scenarios where only a fraction of the labelled pairs are available, our combined model consistently outperforms the strong supervised model baseline (DDL) by a significant margin (p <.05; Wilcoxon test). Our code is publicly available at "https://github.com/jialin-yu/latent-sequence-paraphrase".
△ Less
Submitted 8 September, 2023; v1 submitted 5 January, 2023;
originally announced January 2023.
-
Effect of emotions and personalisation on cancer website reuse intentions
Authors:
Suncica Hadzidedic,
Alexandra I. Cristea,
Derrick G. Watson
Abstract:
The effect of emotions and personalisation on continuance use intentions in online health services is underexplored. Accordingly, we propose a research model for examining the impact of emotion- and personalisation-based factors on cancer website reuse intentions. We conducted a study using a real-world NGO cancer-support website, which was evaluated by 98 participants via an online questionnaire.…
▽ More
The effect of emotions and personalisation on continuance use intentions in online health services is underexplored. Accordingly, we propose a research model for examining the impact of emotion- and personalisation-based factors on cancer website reuse intentions. We conducted a study using a real-world NGO cancer-support website, which was evaluated by 98 participants via an online questionnaire. Model relations were estimated using the PLS-SEM method. Our findings indicated that pre-use emotions did not significantly influence perceived personalisation. However, satisfaction with personalisation, and perceived usefulness mediated by satisfaction, increased reuse intentions. In addition, post-use positive emotions potentially influenced reuse intentions. Our paper, therefore, illustrates the applicability of theory regarding continuance use intentions to cancer-support websites and highlights the importance of personalisation for these purposes.
△ Less
Submitted 2 January, 2023;
originally announced January 2023.
-
Religion and Spirituality on Social Media in the Aftermath of the Global Pandemic
Authors:
Olanrewaju Tahir Aduragba,
Alexandra I. Cristea,
Pete Phillips,
Jonas Kurlberg,
Jialin Yu
Abstract:
During the COVID-19 pandemic, the Church closed its physical doors for the first time in about 800 years, which is, arguably, a cataclysmic event. Other religions have found themselves in a similar situation, and they were practically forced to move online, which is an unprecedented occasion. In this paper, we analyse this sudden change in religious activities twofold: we create and deliver a ques…
▽ More
During the COVID-19 pandemic, the Church closed its physical doors for the first time in about 800 years, which is, arguably, a cataclysmic event. Other religions have found themselves in a similar situation, and they were practically forced to move online, which is an unprecedented occasion. In this paper, we analyse this sudden change in religious activities twofold: we create and deliver a questionnaire, as well as analyse Twitter data, to understand people's perceptions and activities related to religious activities online. Importantly, we also analyse the temporal variations in this process by analysing a period of 3 months: July-September 2020. Additionally to the separate analysis of the two data sources, we also discuss the implications from triangulating the results.
△ Less
Submitted 11 December, 2022;
originally announced December 2022.
-
Multi-task Learning for Personal Health Mention Detection on Social Media
Authors:
Olanrewaju Tahir Aduragba,
Jialin Yu,
Alexandra I. Cristea
Abstract:
Detecting personal health mentions on social media is essential to complement existing health surveillance systems. However, annotating data for detecting health mentions at a large scale is a challenging task. This research employs a multitask learning framework to leverage available annotated data from a related task to improve the performance on the main task to detect personal health experienc…
▽ More
Detecting personal health mentions on social media is essential to complement existing health surveillance systems. However, annotating data for detecting health mentions at a large scale is a challenging task. This research employs a multitask learning framework to leverage available annotated data from a related task to improve the performance on the main task to detect personal health experiences mentioned in social media texts. Specifically, we focus on incorporating emotional information into our target task by using emotion detection as an auxiliary task. Our approach significantly improves a wide range of personal health mention detection tasks compared to a strong state-of-the-art baseline.
△ Less
Submitted 9 December, 2022;
originally announced December 2022.
-
Incorporating Emotions into Health Mention Classification Task on Social Media
Authors:
Olanrewaju Tahir Aduragba,
Jialin Yu,
Alexandra I. Cristea
Abstract:
The health mention classification (HMC) task is the process of identifying and classifying mentions of health-related concepts in text. This can be useful for identifying and tracking the spread of diseases through social media posts. However, this is a non-trivial task. Here we build on recent studies suggesting that using emotional information may improve upon this task. Our study results in a f…
▽ More
The health mention classification (HMC) task is the process of identifying and classifying mentions of health-related concepts in text. This can be useful for identifying and tracking the spread of diseases through social media posts. However, this is a non-trivial task. Here we build on recent studies suggesting that using emotional information may improve upon this task. Our study results in a framework for health mention classification that incorporates affective features. We present two methods, an intermediate task fine-tuning approach (implicit) and a multi-feature fusion approach (explicit) to incorporate emotions into our target task of HMC. We evaluated our approach on 5 HMC-related datasets from different social media platforms including three from Twitter, one from Reddit and another from a combination of social media sources. Extensive experiments demonstrate that our approach results in statistically significant performance gains on HMC tasks. By using the multi-feature fusion approach, we achieve at least a 3% improvement in F1 score over BERT baselines across all datasets. We also show that considering only negative emotions does not significantly affect performance on the HMC task. Additionally, our results indicate that HMC models infused with emotional knowledge are an effective alternative, especially when other HMC datasets are unavailable for domain-specific fine-tuning. The source code for our models is freely available at https://github.com/tahirlanre/Emotion_PHM.
△ Less
Submitted 9 December, 2022;
originally announced December 2022.
-
INTERACTION: A Generative XAI Framework for Natural Language Inference Explanations
Authors:
Jialin Yu,
Alexandra I. Cristea,
Anoushka Harit,
Zhongtian Sun,
Olanrewaju Tahir Aduragba,
Lei Shi,
Noura Al Moubayed
Abstract:
XAI with natural language processing aims to produce human-readable explanations as evidence for AI decision-making, which addresses explainability and transparency. However, from an HCI perspective, the current approaches only focus on delivering a single explanation, which fails to account for the diversity of human thoughts and experiences in language. This paper thus addresses this gap, by pro…
▽ More
XAI with natural language processing aims to produce human-readable explanations as evidence for AI decision-making, which addresses explainability and transparency. However, from an HCI perspective, the current approaches only focus on delivering a single explanation, which fails to account for the diversity of human thoughts and experiences in language. This paper thus addresses this gap, by proposing a generative XAI framework, INTERACTION (explaIn aNd predicT thEn queRy with contextuAl CondiTional varIational autO-eNcoder). Our novel framework presents explanation in two steps: (step one) Explanation and Label Prediction; and (step two) Diverse Evidence Generation. We conduct intensive experiments with the Transformer architecture on a benchmark dataset, e-SNLI. Our method achieves competitive or better performance against state-of-the-art baseline models on explanation generation (up to 4.7% gain in BLEU) and prediction (up to 4.4% gain in accuracy) in step one; it can also generate multiple diverse explanations in step two.
△ Less
Submitted 2 September, 2022;
originally announced September 2022.
-
Exploring Bayesian Deep Learning for Urgent Instructor Intervention Need in MOOC Forums
Authors:
Jialin Yu,
Laila Alrajhi,
Anoushka Harit,
Zhongtian Sun,
Alexandra I. Cristea,
Lei Shi
Abstract:
Massive Open Online Courses (MOOCs) have become a popular choice for e-learning thanks to their great flexibility. However, due to large numbers of learners and their diverse backgrounds, it is taxing to offer real-time support. Learners may post their feelings of confusion and struggle in the respective MOOC forums, but with the large volume of posts and high workloads for MOOC instructors, it is…
▽ More
Massive Open Online Courses (MOOCs) have become a popular choice for e-learning thanks to their great flexibility. However, due to large numbers of learners and their diverse backgrounds, it is taxing to offer real-time support. Learners may post their feelings of confusion and struggle in the respective MOOC forums, but with the large volume of posts and high workloads for MOOC instructors, it is unlikely that the instructors can identify all learners requiring intervention. This problem has been studied as a Natural Language Processing (NLP) problem recently, and is known to be challenging, due to the imbalance of the data and the complex nature of the task. In this paper, we explore for the first time Bayesian deep learning on learner-based text posts with two methods: Monte Carlo Dropout and Variational Inference, as a new solution to assessing the need of instructor interventions for a learner's post. We compare models based on our proposed methods with probabilistic modelling to its baseline non-Bayesian models under similar circumstances, for different cases of applying prediction. The results suggest that Bayesian deep learning offers a critical uncertainty measure that is not supplied by traditional neural networks. This adds more explainability, trust and robustness to AI, which is crucial in education-based applications. Additionally, it can achieve similar or better performance compared to non-probabilistic neural networks, as well as grant lower variance.
△ Less
Submitted 15 November, 2021; v1 submitted 26 April, 2021;
originally announced April 2021.
-
Revealing the Hidden Patterns: A Comparative Study on Profiling Subpopulations of MOOC Students
Authors:
Lei Shi,
Alexandra I. Cristea,
Armando M. Toda,
Wilk Oliveira
Abstract:
Massive Open Online Courses (MOOCs) exhibit a remarkable heterogeneity of students. The advent of complex "big data" from MOOC platforms is a challenging yet rewarding opportunity to deeply understand how students are engaged in MOOCs. Past research, looking mainly into overall behavior, may have missed patterns related to student diversity. Using a large dataset from a MOOC offered by FutureLearn…
▽ More
Massive Open Online Courses (MOOCs) exhibit a remarkable heterogeneity of students. The advent of complex "big data" from MOOC platforms is a challenging yet rewarding opportunity to deeply understand how students are engaged in MOOCs. Past research, looking mainly into overall behavior, may have missed patterns related to student diversity. Using a large dataset from a MOOC offered by FutureLearn, we delve into a new way of investigating hidden patterns through both machine learning and statistical modelling. In this paper, we report on clustering analysis of student activities and comparative analysis on both behavioral patterns and demographical patterns between student subpopulations in the MOOC. Our approach allows for a deeper understanding of how MOOC students behave and achieve. Our findings may be used to design adaptive strategies towards an enhanced MOOC experience
△ Less
Submitted 12 August, 2020;
originally announced August 2020.
-
Predicting MOOCs Dropout Using Only Two Easily Obtainable Features from the First Week's Activities
Authors:
Ahmed Alamri,
Mohammad Alshehri,
Alexandra I. Cristea,
Filipe D. Pereira,
Elaine Oliveira,
Lei Shi,
Craig Stewart
Abstract:
While Massive Open Online Course (MOOCs) platforms provide knowledge in a new and unique way, the very high number of dropouts is a significant drawback. Several features are considered to contribute towards learner attrition or lack of interest, which may lead to disengagement or total dropout. The jury is still out on which factors are the most appropriate predictors. However, the literature agr…
▽ More
While Massive Open Online Course (MOOCs) platforms provide knowledge in a new and unique way, the very high number of dropouts is a significant drawback. Several features are considered to contribute towards learner attrition or lack of interest, which may lead to disengagement or total dropout. The jury is still out on which factors are the most appropriate predictors. However, the literature agrees that early prediction is vital to allow for a timely intervention. Whilst feature-rich predictors may have the best chance for high accuracy, they may be unwieldy. This study aims to predict learner dropout early-on, from the first week, by comparing several machine-learning approaches, including Random Forest, Adaptive Boost, XGBoost and GradientBoost Classifiers. The results show promising accuracies (82%-94%) using as little as 2 features. We show that the accuracies obtained outperform state of the art approaches, even when the latter deploy several features.
△ Less
Submitted 12 August, 2020;
originally announced August 2020.
-
Analysing gamification elements in educational environments using an existing Gamification taxonomy
Authors:
Armando M. Toda,
Ana C. T. Klock,
Wilk Oliveira,
Paula T. Palomino,
Luiz Rodrigues,
Lei Shi,
Ig Bittencourt,
Isabela Gasparini,
Seiji Isotani,
Alexandra I. Cristea
Abstract:
Gamification has been widely employed in the educational domain over the past eight years when the term became a trend. However, the literature states that gamification still lacks formal definitions to support the design and analysis of gamified strategies. This paper analysed the game elements employed in gamified learning environments through a previously proposed and evaluated taxonomy while d…
▽ More
Gamification has been widely employed in the educational domain over the past eight years when the term became a trend. However, the literature states that gamification still lacks formal definitions to support the design and analysis of gamified strategies. This paper analysed the game elements employed in gamified learning environments through a previously proposed and evaluated taxonomy while detailing and expanding this taxonomy. In the current paper, we describe our taxonomy in-depth as well as expand it. Our new structured results demonstrate an extension of the proposed taxonomy which results from this process, is divided into five dimensions, related to the learner and the learning environment. Our main contribution is the detailed taxonomy that can be used to design and evaluate gamification design in learning environments.
△ Less
Submitted 12 August, 2020;
originally announced August 2020.
-
Is MOOC Learning Different for Dropouts? A Visually-Driven, Multi-granularity Explanatory ML Approach
Authors:
Ahmed Alamri,
Zhongtian Sun,
Alexandra I. Cristea,
Gautham Senthilnathan,
Lei Shi,
Craig Stewart
Abstract:
Millions of people have enrolled and enrol (especially in the Covid-19 pandemic world) in MOOCs. However, the retention rate of learners is notoriously low. The majority of the research work on this issue focuses on predicting the dropout rate, but very few use explainable learning patterns as part of this analysis. However, visual representation of learning patterns could provide deeper insights…
▽ More
Millions of people have enrolled and enrol (especially in the Covid-19 pandemic world) in MOOCs. However, the retention rate of learners is notoriously low. The majority of the research work on this issue focuses on predicting the dropout rate, but very few use explainable learning patterns as part of this analysis. However, visual representation of learning patterns could provide deeper insights into learners' behaviour across different courses, whilst numerical analyses can -- and arguably, should -- be used to confirm the latter. Thus, this paper proposes and compares different granularity visualisations for learning patterns (based on clickstream data) for both course completers and non-completers. In the large-scale MOOCs we analysed, across various domains, our fine-grained, fish-eye visualisation approach showed that non-completers are more likely to jump forward in their learning sessions, often on a 'catch-up' path, whilst completers exhibit linear behaviour. For coarser, bird-eye granularity visualisation, we observed learners' transition between types of learning activity, obtaining typed transition graphs. The results, backed up by statistical significance analysis and machine learning, provide insights for course instructors to maintain engagement of learners by adapting the course design to not just 'dry' predicted values, but explainable, visually viable paths extracted.
△ Less
Submitted 12 August, 2020;
originally announced August 2020.
-
Social Engagement versus Learning Engagement -- An Exploratory Study of FutureLearn Learners
Authors:
Lei Shi,
Alexandra I. Cristea,
Armando M. Toda,
Wilk Oliveira
Abstract:
Massive Open Online Courses (MOOCs) continue to see increasing enrolment, but only a small percent of enrolees completes the MOOCs. Whilst a lot of research has focused on predicting completion, there is little research analysing the ostensible contradiction between the MOOC's popularity and the apparent disengagement of learners. Specifically, it is important to analyse engagement not just in lea…
▽ More
Massive Open Online Courses (MOOCs) continue to see increasing enrolment, but only a small percent of enrolees completes the MOOCs. Whilst a lot of research has focused on predicting completion, there is little research analysing the ostensible contradiction between the MOOC's popularity and the apparent disengagement of learners. Specifically, it is important to analyse engagement not just in learning, but also from a social perspective. This is especially crucial, as MOOCs offer a growing amount of activities, which can be classified as social interactions. Thus, this study is particularly concerned with how learners interact with peers, along with their study progression in MOOCs. Additionally, unlike most existing studies that are mainly focused on learning outcomes, this study adopts a fine-grained temporal approach to exploring how learners progress within a MOOC. The study was conducted on the less explored FutureLearn platform, which employs a social constructivist approach and promotes collaborative learning. The preliminary results suggest potential interesting fine-grained predictive models for learner behaviour, involving weekly monitoring of social, non-social behaviour of active students (further classified as completers and non-completers).
△ Less
Submitted 11 August, 2020;
originally announced August 2020.
-
Exploring Navigation Styles in a FutureLearn MOOC
Authors:
Lei Shi,
Alexandra I. Cristea,
Armando M. Toda,
Wilk Oliveira
Abstract:
This paper presents for the first time a detailed analysis of fine-grained navigation style identification in MOOCs backed by a large number of active learners. The result shows 1) whilst the sequential style is clearly in evidence, the global style is less prominent; 2) the majority of the learners do not belong to either category; 3) navigation styles are not as stable as believed in the literat…
▽ More
This paper presents for the first time a detailed analysis of fine-grained navigation style identification in MOOCs backed by a large number of active learners. The result shows 1) whilst the sequential style is clearly in evidence, the global style is less prominent; 2) the majority of the learners do not belong to either category; 3) navigation styles are not as stable as believed in the literature; and 4) learners can, and do, swap between navigation styles with detrimental effects. The approach is promising, as it provides insight into online learners' temporal engagement, as well as a tool to identify vulnerable learners, which potentially benefit personalised interventions (from teachers or automatic help) in Intelligent Tutoring Systems (ITS).
△ Less
Submitted 10 August, 2020;
originally announced August 2020.
-
Nowcasting the Stance of Social Media Users in a Sudden Vote: The Case of the Greek Referendum
Authors:
Adam Tsakalidis,
Nikolaos Aletras,
Alexandra I. Cristea,
Maria Liakata
Abstract:
Modelling user voting intention in social media is an important research area, with applications in analysing electorate behaviour, online political campaigning and advertising. Previous approaches mainly focus on predicting national general elections, which are regularly scheduled and where data of past results and opinion polls are available. However, there is no evidence of how such models woul…
▽ More
Modelling user voting intention in social media is an important research area, with applications in analysing electorate behaviour, online political campaigning and advertising. Previous approaches mainly focus on predicting national general elections, which are regularly scheduled and where data of past results and opinion polls are available. However, there is no evidence of how such models would perform during a sudden vote under time-constrained circumstances. That poses a more challenging task compared to traditional elections, due to its spontaneous nature. In this paper, we focus on the 2015 Greek bailout referendum, aiming to nowcast on a daily basis the voting intention of 2,197 Twitter users. We propose a semi-supervised multiple convolution kernel learning approach, leveraging temporally sensitive text and network information. Our evaluation under a real-time simulation framework demonstrates the effectiveness and robustness of our approach against competitive baselines, achieving a significant 20% increase in F-score compared to solely text-based models.
△ Less
Submitted 26 August, 2018;
originally announced August 2018.
-
Can We Assess Mental Health through Social Media and Smart Devices? Addressing Bias in Methodology and Evaluation
Authors:
Adam Tsakalidis,
Maria Liakata,
Theo Damoulas,
Alexandra I. Cristea
Abstract:
Predicting mental health from smartphone and social media data on a longitudinal basis has recently attracted great interest, with very promising results being reported across many studies. Such approaches have the potential to revolutionise mental health assessment, if their development and evaluation follows a real world deployment setting. In this work we take a closer look at state-of-the-art…
▽ More
Predicting mental health from smartphone and social media data on a longitudinal basis has recently attracted great interest, with very promising results being reported across many studies. Such approaches have the potential to revolutionise mental health assessment, if their development and evaluation follows a real world deployment setting. In this work we take a closer look at state-of-the-art approaches, using different mental health datasets and indicators, different feature sources and multiple simulations, in order to assess their ability to generalise. We demonstrate that under a pragmatic evaluation framework, none of the approaches deliver or even approach the reported performances. In fact, we show that current state-of-the-art approaches can barely outperform the most naïve baselines in the real-world setting, posing serious questions not only about their deployment ability, but also about the contribution of the derived features for the mental health assessment task and how to make better use of such data in the future.
△ Less
Submitted 19 July, 2018;
originally announced July 2018.