-
M3BAT: Unsupervised Domain Adaptation for Multimodal Mobile Sensing with Multi-Branch Adversarial Training
Authors:
Lakmal Meegahapola,
Hamza Hassoune,
Daniel Gatica-Perez
Abstract:
Over the years, multimodal mobile sensing has been used extensively for inferences regarding health and well being, behavior, and context. However, a significant challenge hindering the widespread deployment of such models in real world scenarios is the issue of distribution shift. This is the phenomenon where the distribution of data in the training set differs from the distribution of data in th…
▽ More
Over the years, multimodal mobile sensing has been used extensively for inferences regarding health and well being, behavior, and context. However, a significant challenge hindering the widespread deployment of such models in real world scenarios is the issue of distribution shift. This is the phenomenon where the distribution of data in the training set differs from the distribution of data in the real world, the deployment environment. While extensively explored in computer vision and natural language processing, and while prior research in mobile sensing briefly addresses this concern, current work primarily focuses on models dealing with a single modality of data, such as audio or accelerometer readings, and consequently, there is little research on unsupervised domain adaptation when dealing with multimodal sensor data. To address this gap, we did extensive experiments with domain adversarial neural networks (DANN) showing that they can effectively handle distribution shifts in multimodal sensor data. Moreover, we proposed a novel improvement over DANN, called M3BAT, unsupervised domain adaptation for multimodal mobile sensing with multi-branch adversarial training, to account for the multimodality of sensor data during domain adaptation with multiple branches. Through extensive experiments conducted on two multimodal mobile sensing datasets, three inference tasks, and 14 source-target domain pairs, including both regression and classification, we demonstrate that our approach performs effectively on unseen domains. Compared to directly deploying a model trained in the source domain to the target domain, the model shows performance increases up to 12% AUC (area under the receiver operating characteristics curves) on classification tasks, and up to 0.13 MAE (mean absolute error) on regression tasks.
△ Less
Submitted 26 April, 2024;
originally announced April 2024.
-
Learning About Social Context from Smartphone Data: Generalization Across Countries and Daily Life Moments
Authors:
Aurel Ruben Mader,
Lakmal Meegahapola,
Daniel Gatica-Perez
Abstract:
Understanding how social situations unfold in people's daily lives is relevant to designing mobile systems that can support users in their personal goals, well-being, and activities. As an alternative to questionnaires, some studies have used passively collected smartphone sensor data to infer social context (i.e., being alone or not) with machine learning models. However, the few existing studies…
▽ More
Understanding how social situations unfold in people's daily lives is relevant to designing mobile systems that can support users in their personal goals, well-being, and activities. As an alternative to questionnaires, some studies have used passively collected smartphone sensor data to infer social context (i.e., being alone or not) with machine learning models. However, the few existing studies have focused on specific daily life occasions and limited geographic cohorts in one or two countries. This limits the understanding of how inference models work in terms of generalization to everyday life occasions and multiple countries. In this paper, we used a novel, large-scale, and multimodal smartphone sensing dataset with over 216K self-reports collected from 581 young adults in five countries (Mongolia, Italy, Denmark, UK, Paraguay), first to understand whether social context inference is feasible with sensor data, and then, to know how behavioral and country-level diversity affects inferences. We found that several sensors are informative of social context, that partially personalized multi-country models (trained and tested with data from all countries) and country-specific models (trained and tested within countries) can achieve similar performance above 90% AUC, and that models do not generalize well to unseen countries regardless of geographic proximity. These findings confirm the importance of the diversity of mobile data, to better understand social context inference models in different countries.
△ Less
Submitted 1 March, 2024; v1 submitted 1 June, 2023;
originally announced June 2023.
-
Inferring Mood-While-Eating with Smartphone Sensing and Community-Based Model Personalization
Authors:
Wageesha Bangamuarachchi,
Anju Chamantha,
Lakmal Meegahapola,
Haeeun Kim,
Salvador Ruiz-Correa,
Indika Perera,
Daniel Gatica-Perez
Abstract:
The interplay between mood and eating has been the subject of extensive research within the fields of nutrition and behavioral science, indicating a strong connection between the two. Further, phone sensor data have been used to characterize both eating behavior and mood, independently, in the context of mobile food diaries and mobile health applications. However, limitations within the current bo…
▽ More
The interplay between mood and eating has been the subject of extensive research within the fields of nutrition and behavioral science, indicating a strong connection between the two. Further, phone sensor data have been used to characterize both eating behavior and mood, independently, in the context of mobile food diaries and mobile health applications. However, limitations within the current body of literature include: i) the lack of investigation around the generalization of mood inference models trained with passive sensor data from a range of everyday life situations, to specific contexts such as eating, ii) no prior studies that use sensor data to study the intersection of mood and eating, and iii) the inadequate examination of model personalization techniques within limited label settings, as we commonly experience in mood inference. In this study, we sought to examine everyday eating behavior and mood using two datasets of college students in Mexico (N_mex = 84, 1843 mood-while-eating reports) and eight countries (N_mul = 678, 329K mood reports incl. 24K mood-while-eating reports), containing both passive smartphone sensing and self-report data. Our results indicate that generic mood inference models decline in performance in certain contexts, such as when eating. Additionally, we found that population-level (non-personalized) and hybrid (partially personalized) modeling techniques were inadequate for the commonly used three-class mood inference task (positive, neutral, negative). Furthermore, we found that user-level modeling was challenging for the majority of participants due to a lack of sufficient labels and data from the negative class. To address these limitations, we employed a novel community-based approach for personalization by building models with data from a set of similar users to a target user.
△ Less
Submitted 9 December, 2023; v1 submitted 1 June, 2023;
originally announced June 2023.
-
Understanding the Social Context of Eating with Multimodal Smartphone Sensing: The Role of Country Diversity
Authors:
Nathan Kammoun,
Lakmal Meegahapola,
Daniel Gatica-Perez
Abstract:
Understanding the social context of eating is crucial for promoting healthy eating behaviors. Multimodal smartphone sensor data could provide valuable insights into eating behavior, particularly in mobile food diaries and mobile health apps. However, research on the social context of eating with smartphone sensor data is limited, despite extensive studies in nutrition and behavioral science. Moreo…
▽ More
Understanding the social context of eating is crucial for promoting healthy eating behaviors. Multimodal smartphone sensor data could provide valuable insights into eating behavior, particularly in mobile food diaries and mobile health apps. However, research on the social context of eating with smartphone sensor data is limited, despite extensive studies in nutrition and behavioral science. Moreover, the impact of country differences on the social context of eating, as measured by multimodal phone sensor data and self-reports, remains under-explored. To address this research gap, our study focuses on a dataset of approximately 24K self-reports on eating events provided by 678 college students in eight countries to investigate the country diversity that emerges from smartphone sensors during eating events for different social contexts (alone or with others). Our analysis revealed that while some smartphone usage features during eating events were similar across countries, others exhibited unique trends in each country. We further studied how user and country-specific factors impact social context inference by develo** machine learning models with population-level (non-personalized) and hybrid (partially personalized) experimental setups. We showed that models based on the hybrid approach achieve AUC scores up to 0.75 with XGBoost models. These findings emphasize the importance of considering country differences in building and deploying machine learning models to minimize biases and improve generalization across different populations.
△ Less
Submitted 4 October, 2023; v1 submitted 1 June, 2023;
originally announced June 2023.
-
Examining European Press Coverage of the Covid-19 No-Vax Movement: An NLP Framework
Authors:
David Alonso del Barrio,
Daniel Gatica-Perez
Abstract:
This paper examines how the European press dealt with the no-vax reactions against the Covid-19 vaccine and the dis- and misinformation associated with this movement. Using a curated dataset of 1786 articles from 19 European newspapers on the anti-vaccine movement over a period of 22 months in 2020-2021, we used Natural Language Processing techniques including topic modeling, sentiment analysis, s…
▽ More
This paper examines how the European press dealt with the no-vax reactions against the Covid-19 vaccine and the dis- and misinformation associated with this movement. Using a curated dataset of 1786 articles from 19 European newspapers on the anti-vaccine movement over a period of 22 months in 2020-2021, we used Natural Language Processing techniques including topic modeling, sentiment analysis, semantic relationship with word embeddings, political analysis, named entity recognition, and semantic networks, to understand the specific role of the European traditional press in the disinformation ecosystem. The results of this multi-angle analysis demonstrate that the European well-established press actively opposed a variety of hoaxes mainly spread on social media, and was critical of the anti-vax trend, regardless of the political orientation of the newspaper. This confirms the relevance of studying the role of high-quality press in the disinformation ecosystem.
△ Less
Submitted 29 April, 2023;
originally announced May 2023.
-
Framing the News:From Human Perception to Large Language Model Inferences
Authors:
David Alonso del Barrio,
Daniel Gatica-Perez
Abstract:
Identifying the frames of news is important to understand the articles' vision, intention, message to be conveyed, and which aspects of the news are emphasized. Framing is a widely studied concept in journalism, and has emerged as a new topic in computing, with the potential to automate processes and facilitate the work of journalism professionals. In this paper, we study this issue with articles…
▽ More
Identifying the frames of news is important to understand the articles' vision, intention, message to be conveyed, and which aspects of the news are emphasized. Framing is a widely studied concept in journalism, and has emerged as a new topic in computing, with the potential to automate processes and facilitate the work of journalism professionals. In this paper, we study this issue with articles related to the Covid-19 anti-vaccine movement. First, to understand the perspectives used to treat this theme, we developed a protocol for human labeling of frames for 1786 headlines of No-Vax movement articles of European newspapers from 5 countries. Headlines are key units in the written press, and worth of analysis as many people only read headlines (or use them to guide their decision for further reading.) Second, considering advances in Natural Language Processing (NLP) with large language models, we investigated two approaches for frame inference of news headlines: first with a GPT-3.5 fine-tuning approach, and second with GPT-3.5 prompt-engineering. Our work contributes to the study and analysis of the performance that these models have to facilitate journalistic tasks like classification of frames, while understanding whether the models are able to replicate human perception in the identification of these frames.
△ Less
Submitted 27 April, 2023;
originally announced April 2023.
-
ProGAP: Progressive Graph Neural Networks with Differential Privacy Guarantees
Authors:
Sina Sajadmanesh,
Daniel Gatica-Perez
Abstract:
Graph Neural Networks (GNNs) have become a popular tool for learning on graphs, but their widespread use raises privacy concerns as graph data can contain personal or sensitive information. Differentially private GNN models have been recently proposed to preserve privacy while still allowing for effective learning over graph-structured datasets. However, achieving an ideal balance between accuracy…
▽ More
Graph Neural Networks (GNNs) have become a popular tool for learning on graphs, but their widespread use raises privacy concerns as graph data can contain personal or sensitive information. Differentially private GNN models have been recently proposed to preserve privacy while still allowing for effective learning over graph-structured datasets. However, achieving an ideal balance between accuracy and privacy in GNNs remains challenging due to the intrinsic structural connectivity of graphs. In this paper, we propose a new differentially private GNN called ProGAP that uses a progressive training scheme to improve such accuracy-privacy trade-offs. Combined with the aggregation perturbation technique to ensure differential privacy, ProGAP splits a GNN into a sequence of overlap** submodels that are trained progressively, expanding from the first submodel to the complete model. Specifically, each submodel is trained over the privately aggregated node embeddings learned and cached by the previous submodels, leading to an increased expressive power compared to previous approaches while limiting the incurred privacy costs. We formally prove that ProGAP ensures edge-level and node-level privacy guarantees for both training and inference stages, and evaluate its performance on benchmark graph datasets. Experimental results demonstrate that ProGAP can achieve up to 5-10% higher accuracy than existing state-of-the-art differentially private GNNs. Our code is available at https://github.com/sisaman/ProGAP.
△ Less
Submitted 20 October, 2023; v1 submitted 18 April, 2023;
originally announced April 2023.
-
Complex Daily Activities, Country-Level Diversity, and Smartphone Sensing: A Study in Denmark, Italy, Mongolia, Paraguay, and UK
Authors:
Karim Assi,
Lakmal Meegahapola,
William Droz,
Peter Kun,
Amalia de Gotzen,
Miriam Bidoglia,
Sally Stares,
George Gaskell,
Altangerel Chagnaa,
Amarsanaa Ganbold,
Tsolmon Zundui,
Carlo Caprini,
Daniele Miorandi,
Alethia Hume,
Jose Luis Zarza,
Luca Cernuzzi,
Ivano Bison,
Marcelo Dario Rodas Britez,
Matteo Busso,
Ronald Chenu-Abente,
Fausto Giunchiglia,
Daniel Gatica-Perez
Abstract:
Smartphones enable understanding human behavior with activity recognition to support people's daily lives. Prior studies focused on using inertial sensors to detect simple activities (sitting, walking, running, etc.) and were mostly conducted in homogeneous populations within a country. However, people are more sedentary in the post-pandemic world with the prevalence of remote/hybrid work/study se…
▽ More
Smartphones enable understanding human behavior with activity recognition to support people's daily lives. Prior studies focused on using inertial sensors to detect simple activities (sitting, walking, running, etc.) and were mostly conducted in homogeneous populations within a country. However, people are more sedentary in the post-pandemic world with the prevalence of remote/hybrid work/study settings, making detecting simple activities less meaningful for context-aware applications. Hence, the understanding of (i) how multimodal smartphone sensors and machine learning models could be used to detect complex daily activities that can better inform about people's daily lives and (ii) how models generalize to unseen countries, is limited. We analyzed in-the-wild smartphone data and over 216K self-reports from 637 college students in five countries (Italy, Mongolia, UK, Denmark, Paraguay). Then, we defined a 12-class complex daily activity recognition task and evaluated the performance with different approaches. We found that even though the generic multi-country approach provided an AUROC of 0.70, the country-specific approach performed better with AUROC scores in [0.79-0.89]. We believe that research along the lines of diversity awareness is fundamental for advancing human behavior understanding through smartphones and machine learning, for more real-world utility across countries.
△ Less
Submitted 16 February, 2023;
originally announced February 2023.
-
Your Day in Your Pocket: Complex Activity Recognition from Smartphone Accelerometers
Authors:
Emma Bouton--Bessac,
Lakmal Meegahapola,
Daniel Gatica-Perez
Abstract:
Human Activity Recognition (HAR) enables context-aware user experiences where mobile apps can alter content and interactions depending on user activities. Hence, smartphones have become valuable for HAR as they allow large, and diversified data collection. Although previous work in HAR managed to detect simple activities (i.e., sitting, walking, running) with good accuracy using inertial sensors (…
▽ More
Human Activity Recognition (HAR) enables context-aware user experiences where mobile apps can alter content and interactions depending on user activities. Hence, smartphones have become valuable for HAR as they allow large, and diversified data collection. Although previous work in HAR managed to detect simple activities (i.e., sitting, walking, running) with good accuracy using inertial sensors (i.e., accelerometer), the recognition of complex daily activities remains an open problem, specially in remote work/study settings when people are more sedentary. Moreover, understanding the everyday activities of a person can support the creation of applications that aim to support their well-being. This paper investigates the recognition of complex activities exclusively using smartphone accelerometer data. We used a large smartphone sensing dataset collected from over 600 users in five countries during the pandemic and showed that deep learning-based, binary classification of eight complex activities (slee**, eating, watching videos, online communication, attending a lecture, sports, shop**, studying) can be achieved with AUROC scores up to 0.76 with partially personalized models. This shows encouraging signs toward assessing complex activities only using phone accelerometer data in the post-pandemic world.
△ Less
Submitted 17 January, 2023;
originally announced January 2023.
-
Generalization and Personalization of Mobile Sensing-Based Mood Inference Models: An Analysis of College Students in Eight Countries
Authors:
Lakmal Meegahapola,
William Droz,
Peter Kun,
Amalia de Gotzen,
Chaitanya Nutakki,
Shyam Diwakar,
Salvador Ruiz Correa,
Donglei Song,
Hao Xu,
Miriam Bidoglia,
George Gaskell,
Altangerel Chagnaa,
Amarsanaa Ganbold,
Tsolmon Zundui,
Carlo Caprini,
Daniele Miorandi,
Alethia Hume,
Jose Luis Zarza,
Luca Cernuzzi,
Ivano Bison,
Marcelo Rodas Britez,
Matteo Busso,
Ronald Chenu-Abente,
Can Gunel,
Fausto Giunchiglia
, et al. (2 additional authors not shown)
Abstract:
Mood inference with mobile sensing data has been studied in ubicomp literature over the last decade. This inference enables context-aware and personalized user experiences in general mobile apps and valuable feedback and interventions in mobile health apps. However, even though model generalization issues have been highlighted in many studies, the focus has always been on improving the accuracies…
▽ More
Mood inference with mobile sensing data has been studied in ubicomp literature over the last decade. This inference enables context-aware and personalized user experiences in general mobile apps and valuable feedback and interventions in mobile health apps. However, even though model generalization issues have been highlighted in many studies, the focus has always been on improving the accuracies of models using different sensing modalities and machine learning techniques, with datasets collected in homogeneous populations. In contrast, less attention has been given to studying the performance of mood inference models to assess whether models generalize to new countries. In this study, we collected a mobile sensing dataset with 329K self-reports from 678 participants in eight countries (China, Denmark, India, Italy, Mexico, Mongolia, Paraguay, UK) to assess the effect of geographical diversity on mood inference models. We define and evaluate country-specific (trained and tested within a country), continent-specific (trained and tested within a continent), country-agnostic (tested on a country not seen on training data), and multi-country (trained and tested with multiple countries) approaches trained on sensor data for two mood inference tasks with population-level (non-personalized) and hybrid (partially personalized) models. We show that partially personalized country-specific models perform the best yielding area under the receiver operating characteristic curve (AUROC) scores of the range 0.78-0.98 for two-class (negative vs. positive valence) and 0.76-0.94 for three-class (negative vs. neutral vs. positive valence) inference. Overall, we uncover generalization issues of mood inference models to new countries and how the geographical similarity of countries might impact mood inference.
△ Less
Submitted 5 November, 2022;
originally announced November 2022.
-
Sensing Eating Events in Context: A Smartphone-Only Approach
Authors:
Wageesha Bangamuarachchi,
Anju Chamantha,
Lakmal Meegahapola,
Salvador Ruiz-Correa,
Indika Perera,
Daniel Gatica-Perez
Abstract:
While the task of automatically detecting eating events has been examined in prior work using various wearable devices, the use of smartphones as standalone devices to infer eating events remains an open issue. This paper proposes a framework that infers eating vs. non-eating events from passive smartphone sensing and evaluates it on a dataset of 58 college students. First, we show that time of th…
▽ More
While the task of automatically detecting eating events has been examined in prior work using various wearable devices, the use of smartphones as standalone devices to infer eating events remains an open issue. This paper proposes a framework that infers eating vs. non-eating events from passive smartphone sensing and evaluates it on a dataset of 58 college students. First, we show that time of the day and features from modalities such as screen usage, accelerometer, app usage, and location are indicative of eating and non-eating events. Then, we show that eating events can be inferred with an AUROC (area under the receiver operating characteristics curve) of 0.65 using subject-independent machine learning models, which can be further improved up to 0.81 for subject-dependent and 0.81 for hybrid models using personalization techniques. Moreover, we show that users have different behavioral and contextual routines around eating episodes requiring specific feature groups to train fully personalized models. These findings are of potential value for future mobile food diary apps that are context-aware by enabling scalable sensing-based eating studies using only smartphones; detecting under-reported eating events, thus increasing data quality in self report-based studies; providing functionality to track food consumption and generate reminders for on-time collection of food diaries; and supporting mobile interventions towards healthy eating practices.
△ Less
Submitted 31 May, 2022; v1 submitted 27 May, 2022;
originally announced May 2022.
-
GAP: Differentially Private Graph Neural Networks with Aggregation Perturbation
Authors:
Sina Sajadmanesh,
Ali Shahin Shamsabadi,
Aurélien Bellet,
Daniel Gatica-Perez
Abstract:
In this paper, we study the problem of learning Graph Neural Networks (GNNs) with Differential Privacy (DP). We propose a novel differentially private GNN based on Aggregation Perturbation (GAP), which adds stochastic noise to the GNN's aggregation function to statistically obfuscate the presence of a single edge (edge-level privacy) or a single node and all its adjacent edges (node-level privacy)…
▽ More
In this paper, we study the problem of learning Graph Neural Networks (GNNs) with Differential Privacy (DP). We propose a novel differentially private GNN based on Aggregation Perturbation (GAP), which adds stochastic noise to the GNN's aggregation function to statistically obfuscate the presence of a single edge (edge-level privacy) or a single node and all its adjacent edges (node-level privacy). Tailored to the specifics of private learning, GAP's new architecture is composed of three separate modules: (i) the encoder module, where we learn private node embeddings without relying on the edge information; (ii) the aggregation module, where we compute noisy aggregated node embeddings based on the graph structure; and (iii) the classification module, where we train a neural network on the private aggregations for node classification without further querying the graph edges. GAP's major advantage over previous approaches is that it can benefit from multi-hop neighborhood aggregations, and guarantees both edge-level and node-level DP not only for training, but also at inference with no additional costs beyond the training's privacy budget. We analyze GAP's formal privacy guarantees using Rényi DP and conduct empirical experiments over three real-world graph datasets. We demonstrate that GAP offers significantly better accuracy-privacy trade-offs than state-of-the-art DP-GNN approaches and naive MLP-based baselines. Our code is publicly available at https://github.com/sisaman/GAP.
△ Less
Submitted 18 November, 2022; v1 submitted 2 March, 2022;
originally announced March 2022.
-
The Theory, Practice, and Ethical Challenges of Designing a Diversity-Aware Platform for Social Relations
Authors:
Laura Schelenz,
Ivano Bison,
Matteo Busso,
Amalia de Götzen,
Daniel Gatica-Perez,
Fausto Giunchiglia,
Lakmal Meegahapola,
Salvador Ruiz-Correa
Abstract:
Diversity-aware platform design is a paradigm that responds to the ethical challenges of existing social media platforms. Available platforms have been criticized for minimizing users' autonomy, marginalizing minorities, and exploiting users' data for profit maximization. This paper presents a design solution that centers the well-being of users. It presents the theory and practice of designing a…
▽ More
Diversity-aware platform design is a paradigm that responds to the ethical challenges of existing social media platforms. Available platforms have been criticized for minimizing users' autonomy, marginalizing minorities, and exploiting users' data for profit maximization. This paper presents a design solution that centers the well-being of users. It presents the theory and practice of designing a diversity-aware platform for social relations. In this approach, the diversity of users is leveraged in a way that allows like-minded individuals to pursue similar interests or diverse individuals to complement each other in a complex activity. The end users of the envisioned platform are students, who participate in the design process. Diversity-aware platform design involves numerous steps, of which two are highlighted in this paper: 1) defining a framework and operationalizing the "diversity" of students, 2) collecting "diversity" data to build diversity-aware algorithms. The paper further reflects on the ethical challenges encountered during the design of a diversity-aware platform.
△ Less
Submitted 23 February, 2022;
originally announced February 2022.
-
Examining the Social Context of Alcohol Drinking in Young Adults with Smartphone Sensing
Authors:
Lakmal Meegahapola,
Florian Labhart,
Thanh-Trung Phan,
Daniel Gatica-Perez
Abstract:
According to prior work, the type of relationship between the person consuming alcohol and others in the surrounding (friends, family, spouse, etc.), and the number of those people (alone, with one person, with a group, etc.) are related to many aspects of alcohol consumption, such as the drinking amount, location, motives, and mood. Even though the social context is recognized as an important asp…
▽ More
According to prior work, the type of relationship between the person consuming alcohol and others in the surrounding (friends, family, spouse, etc.), and the number of those people (alone, with one person, with a group, etc.) are related to many aspects of alcohol consumption, such as the drinking amount, location, motives, and mood. Even though the social context is recognized as an important aspect that influences the drinking behavior of young adults in alcohol research, relatively little work has been conducted in smartphone sensing research on this topic. In this study, we analyze the weekend nightlife drinking behavior of 241 young adults in Switzerland, using a dataset consisting of self-reports and passive smartphone sensing data over a period of three months. Using multiple statistical analyses, we show that features from modalities such as accelerometer, location, application usage, bluetooth, and proximity could be informative about different social contexts of drinking. We define and evaluate seven social context inference tasks using smartphone sensing data, obtaining accuracies of the range 75%-86% in four two-class and three three-class inferences. Further, we discuss the possibility of identifying the sex composition of a group of friends using smartphone sensor data with accuracies over 70%. The results are encouraging towards (a) supporting future interventions on alcohol consumption that incorporate users' social context more meaningfully, and (b) reducing the need for user self-reports when creating drink logs.
△ Less
Submitted 2 August, 2021; v1 submitted 13 July, 2021;
originally announced July 2021.
-
Smartphone Sensing for the Well-being of Young Adults: A Review
Authors:
Lakmal Meegahapola,
Daniel Gatica-Perez
Abstract:
Over the years, mobile phones have become versatile devices with a multitude of capabilities due to the plethora of embedded sensors that enable them to capture rich data unobtrusively. In a world where people are more conscious regarding their health and well-being, the pervasiveness of smartphones has enabled researchers to build apps that assist people to live healthier lifestyles, and to diagn…
▽ More
Over the years, mobile phones have become versatile devices with a multitude of capabilities due to the plethora of embedded sensors that enable them to capture rich data unobtrusively. In a world where people are more conscious regarding their health and well-being, the pervasiveness of smartphones has enabled researchers to build apps that assist people to live healthier lifestyles, and to diagnose and monitor various health conditions. Motivated by the high smartphone coverage among young adults and the unique issues they face, in this review paper, we focus on studies that have used smartphone sensing for the well-being of young adults. We analyze existing work in the domain from two perspectives, namely Data Perspective and System Perspective. For both these perspectives, we propose taxonomies motivated from human science literature, which enable to identify important study areas. Furthermore, we emphasize the importance of diversity-awareness in smartphone sensing, and provide insights and future directions for researchers in ubiquitous and mobile computing, and especially to new researchers who want to understand the basics of smartphone sensing research targeting the well-being of young adults.
△ Less
Submitted 4 January, 2021; v1 submitted 17 December, 2020;
originally announced December 2020.
-
Alone or With Others? Understanding Eating Episodes of College Students with Mobile Sensing
Authors:
Lakmal Meegahapola,
Salvador Ruiz-Correa,
Daniel Gatica-Perez
Abstract:
Understanding food consumption patterns and contexts using mobile sensing is fundamental to build mobile health applications that require minimal user interaction to generate mobile food diaries. Many available mobile food diaries, both commercial and in research, heavily rely on self-reports, and this dependency limits the long term adoption of these apps by people. The social context of eating (…
▽ More
Understanding food consumption patterns and contexts using mobile sensing is fundamental to build mobile health applications that require minimal user interaction to generate mobile food diaries. Many available mobile food diaries, both commercial and in research, heavily rely on self-reports, and this dependency limits the long term adoption of these apps by people. The social context of eating (alone, with friends, with family, with a partner, etc.) is an important self-reported feature that influences aspects such as food type, psychological state while eating, and the amount of food, according to prior research in nutrition and behavioral sciences. In this work, we use two datasets regarding the everyday eating behavior of college students in two countries, namely Switzerland (N_ch=122) and Mexico (N_mx=84), to examine the relation between the social context of eating and passive sensing data from wearables and smartphones. Moreover, we design a classification task, namely inferring eating-alone vs. eating-with-others episodes using passive sensing data and time of eating, obtaining accuracies between 77% and 81%. We believe that this is a first step towards understanding more complex social contexts related to food consumption using mobile sensing.
△ Less
Submitted 28 November, 2020; v1 submitted 23 November, 2020;
originally announced November 2020.
-
Locally Private Graph Neural Networks
Authors:
Sina Sajadmanesh,
Daniel Gatica-Perez
Abstract:
Graph Neural Networks (GNNs) have demonstrated superior performance in learning node representations for various graph inference tasks. However, learning over graph data can raise privacy concerns when nodes represent people or human-related variables that involve sensitive or personal information. While numerous techniques have been proposed for privacy-preserving deep learning over non-relationa…
▽ More
Graph Neural Networks (GNNs) have demonstrated superior performance in learning node representations for various graph inference tasks. However, learning over graph data can raise privacy concerns when nodes represent people or human-related variables that involve sensitive or personal information. While numerous techniques have been proposed for privacy-preserving deep learning over non-relational data, there is less work addressing the privacy issues pertained to applying deep learning algorithms on graphs. In this paper, we study the problem of node data privacy, where graph nodes have potentially sensitive data that is kept private, but they could be beneficial for a central server for training a GNN over the graph. To address this problem, we develop a privacy-preserving, architecture-agnostic GNN learning algorithm with formal privacy guarantees based on Local Differential Privacy (LDP). Specifically, we propose an LDP encoder and an unbiased rectifier, by which the server can communicate with the graph nodes to privately collect their data and approximate the GNN's first layer. To further reduce the effect of the injected noise, we propose to prepend a simple graph convolution layer, called KProp, which is based on the multi-hop aggregation of the nodes' features acting as a denoising mechanism. Finally, we propose a robust training framework, in which we benefit from KProp's denoising capability to increase the accuracy of inference in the presence of noisy labels. Extensive experiments conducted over real-world datasets demonstrate that our method can maintain a satisfying level of accuracy with low privacy loss.
△ Less
Submitted 6 July, 2021; v1 submitted 9 June, 2020;
originally announced June 2020.
-
Characterizing the Demographics Behind the #BlackLivesMatter Movement
Authors:
Alexandra Olteanu,
Ingmar Weber,
Daniel Gatica-Perez
Abstract:
The debates on minority issues are often dominated by or held among the concerned minority: gender equality debates have often failed to engage men, while those about race fail to effectively engage the dominant group. To test this observation, we study the #BlackLivesMatter}movement and hashtag on Twitter--which has emerged and gained traction after a series of events typically involving the deat…
▽ More
The debates on minority issues are often dominated by or held among the concerned minority: gender equality debates have often failed to engage men, while those about race fail to effectively engage the dominant group. To test this observation, we study the #BlackLivesMatter}movement and hashtag on Twitter--which has emerged and gained traction after a series of events typically involving the death of African-Americans as a result of police brutality--and aim to quantify the population biases across user types (individuals vs. organizations), and (for individuals) across various demographics factors (race, gender and age). Our results suggest that more African-Americans engage with the hashtag, and that they are also more active than other demographic groups. We also discuss ethical caveats with broader implications for studies on sensitive topics (e.g. discrimination, mental health, or religion) that focus on users.
△ Less
Submitted 17 December, 2015;
originally announced December 2015.
-
CommuniSense: Crowdsourcing Road Hazards in Nairobi
Authors:
Darshan Santani,
Jidraph Njuguna,
Tierra Bills,
Aisha W. Bryant,
Reginald Bryant,
Jonathan Ledgard,
Daniel Gatica-Perez
Abstract:
Nairobi is one of the fastest growing metropolitan cities and a major business and technology powerhouse in Africa. However, Nairobi currently lacks monitoring technologies to obtain reliable data on traffic and road infrastructure conditions. In this paper, we investigate the use of mobile crowdsourcing as means to gather and document Nairobi's road quality information. We first present the key f…
▽ More
Nairobi is one of the fastest growing metropolitan cities and a major business and technology powerhouse in Africa. However, Nairobi currently lacks monitoring technologies to obtain reliable data on traffic and road infrastructure conditions. In this paper, we investigate the use of mobile crowdsourcing as means to gather and document Nairobi's road quality information. We first present the key findings of a city-wide road quality survey about the perception of existing road quality conditions in Nairobi. Based on the survey's findings, we then developed a mobile crowdsourcing application, called CommuniSense, to collect road quality data. The application serves as a tool for users to locate, describe, and photograph road hazards. We tested our application through a two-week field study amongst 30 participants to document various forms of road hazards from different areas in Nairobi. To verify the authenticity of user-contributed reports from our field study, we proposed to use online crowdsourcing using Amazon's Mechanical Turk (MTurk) to verify whether submitted reports indeed depict road hazards. We found 92% of user-submitted reports to match the MTurkers judgements. While our prototype was designed and tested on a specific city, our methodology is applicable to other develo** cities.
△ Less
Submitted 24 June, 2015;
originally announced June 2015.
-
Loud and Trendy: Crowdsourcing Impressions of Social Ambiance in Popular Indoor Urban Places
Authors:
Darshan Santani,
Daniel Gatica-Perez
Abstract:
New research cutting across architecture, urban studies, and psychology is contextualizing the understanding of urban spaces according to the perceptions of their inhabitants. One fundamental construct that relates place and experience is ambiance, which is defined as "the mood or feeling associated with a particular place". We posit that the systematic study of ambiance dimensions in cities is a…
▽ More
New research cutting across architecture, urban studies, and psychology is contextualizing the understanding of urban spaces according to the perceptions of their inhabitants. One fundamental construct that relates place and experience is ambiance, which is defined as "the mood or feeling associated with a particular place". We posit that the systematic study of ambiance dimensions in cities is a new domain for which multimedia research can make pivotal contributions. We present a study to examine how images collected from social media can be used for the crowdsourced characterization of indoor ambiance impressions in popular urban places. We design a crowdsourcing framework to understand suitability of social images as data source to convey place ambiance, to examine what type of images are most suitable to describe ambiance, and to assess how people perceive places socially from the perspective of ambiance along 13 dimensions. Our study is based on 50,000 Foursquare images collected from 300 popular places across six cities worldwide. The results show that reliable estimates of ambiance can be obtained for several of the dimensions. Furthermore, we found that most aggregate impressions of ambiance are similar across popular places in all studied cities. We conclude by presenting a multidisciplinary research agenda for future research in this domain.
△ Less
Submitted 18 December, 2015; v1 submitted 5 March, 2014;
originally announced March 2014.
