-
Analyzing Speech Motor Movement using Surface Electromyography in Minimally Verbal Adults with Autism Spectrum Disorder
Authors:
Wazeer Zulfikar,
Nishat Protyasha,
Camila Canales,
Heli Patel,
James Williamson,
Laura Sarnie,
Lisa Nowinski,
Nataliya Kosmyna,
Paige Townsend,
Sophia Yuditskaya,
Tanya Talkar,
Utkarsh Oggy Sarawgi,
Christopher McDougle,
Thomas Quatieri,
Pattie Maes,
Maria Mody
Abstract:
Adults who are minimally verbal with autism spectrum disorder (mvASD) have pronounced speech difficulties linked to impaired motor skills. Existing research and clinical assessments primarily use indirect methods such as standardized tests, video-based facial features, and handwriting tasks, which may not directly target speech-related motor skills. In this study, we measure activity from eight fa…
▽ More
Adults who are minimally verbal with autism spectrum disorder (mvASD) have pronounced speech difficulties linked to impaired motor skills. Existing research and clinical assessments primarily use indirect methods such as standardized tests, video-based facial features, and handwriting tasks, which may not directly target speech-related motor skills. In this study, we measure activity from eight facial muscles associated with speech using surface electromyography (sEMG), during carefully designed tasks. The findings reveal a higher power in the sEMG signals and a significantly greater correlation between the sEMG channels in mvASD adults (N=12) compared to age and gender-matched neurotypical controls (N=14). This suggests stronger muscle activation and greater synchrony in the discharge patterns of motor units. Further, eigenvalues derived from correlation matrices indicate lower complexity in muscle coordination in mvASD, implying fewer degrees of freedom in motor control.
△ Less
Submitted 11 July, 2024;
originally announced July 2024.
-
PhysioLLM: Supporting Personalized Health Insights with Wearables and Large Language Models
Authors:
Cathy Mengying Fang,
Valdemar Danry,
Nathan Whitmore,
Andria Bao,
Andrew Hutchison,
Cayden Pierce,
Pattie Maes
Abstract:
We present PhysioLLM, an interactive system that leverages large language models (LLMs) to provide personalized health understanding and exploration by integrating physiological data from wearables with contextual information. Unlike commercial health apps for wearables, our system offers a comprehensive statistical analysis component that discovers correlations and trends in user data, allowing u…
▽ More
We present PhysioLLM, an interactive system that leverages large language models (LLMs) to provide personalized health understanding and exploration by integrating physiological data from wearables with contextual information. Unlike commercial health apps for wearables, our system offers a comprehensive statistical analysis component that discovers correlations and trends in user data, allowing users to ask questions in natural language and receive generated personalized insights, and guides them to develop actionable goals. As a case study, we focus on improving sleep quality, given its measurability through physiological data and its importance to general well-being. Through a user study with 24 Fitbit watch users, we demonstrate that PhysioLLM outperforms both the Fitbit App alone and a generic LLM chatbot in facilitating a deeper, personalized understanding of health data and supporting actionable steps toward personal health goals.
△ Less
Submitted 27 June, 2024;
originally announced June 2024.
-
Future You: A Conversation with an AI-Generated Future Self Reduces Anxiety, Negative Emotions, and Increases Future Self-Continuity
Authors:
Pat Pataranutaporn,
Kavin Winson,
Peggy Yin,
Auttasak Lapapirojn,
Pichayoot Ouppaphan,
Monchai Lertsutthiwong,
Pattie Maes,
Hal Hershfield
Abstract:
We introduce "Future You," an interactive, brief, single-session, digital chat intervention designed to improve future self-continuity--the degree of connection an individual feels with a temporally distant future self--a characteristic that is positively related to mental health and wellbeing. Our system allows users to chat with a relatable yet AI-powered virtual version of their future selves t…
▽ More
We introduce "Future You," an interactive, brief, single-session, digital chat intervention designed to improve future self-continuity--the degree of connection an individual feels with a temporally distant future self--a characteristic that is positively related to mental health and wellbeing. Our system allows users to chat with a relatable yet AI-powered virtual version of their future selves that is tuned to their future goals and personal qualities. To make the conversation realistic, the system generates a "synthetic memory"--a unique backstory for each user--that creates a throughline between the user's present age (between 18-30) and their life at age 60. The "Future You" character also adopts the persona of an age-progressed image of the user's present self. After a brief interaction with the "Future You" character, users reported decreased anxiety, and increased future self-continuity. This is the first study successfully demonstrating the use of personalized AI-generated characters to improve users' future self-continuity and wellbeing.
△ Less
Submitted 9 July, 2024; v1 submitted 21 May, 2024;
originally announced May 2024.
-
Enabling Waypoint Generation for Collaborative Robots using LLMs and Mixed Reality
Authors:
Cathy Mengying Fang,
Krzysztof Zieliński,
Pattie Maes,
Joe Paradiso,
Bruce Blumberg,
Mikkel Baun Kjærgaard
Abstract:
Programming a robotic is a complex task, as it demands the user to have a good command of specific programming languages and awareness of the robot's physical constraints. We propose a framework that simplifies robot deployment by allowing direct communication using natural language. It uses large language models (LLM) for prompt processing, workspace understanding, and waypoint generation. It als…
▽ More
Programming a robotic is a complex task, as it demands the user to have a good command of specific programming languages and awareness of the robot's physical constraints. We propose a framework that simplifies robot deployment by allowing direct communication using natural language. It uses large language models (LLM) for prompt processing, workspace understanding, and waypoint generation. It also employs Augmented Reality (AR) to provide visual feedback of the planned outcome. We showcase the effectiveness of our framework with a simple pick-and-place task, which we implement on a real robot. Moreover, we present an early concept of expressive robot behavior and skill generation that can be used to communicate with the user and learn new skills (e.g., object gras**).
△ Less
Submitted 14 March, 2024;
originally announced March 2024.
-
Memoro: Using Large Language Models to Realize a Concise Interface for Real-Time Memory Augmentation
Authors:
Wazeer Zulfikar,
Samantha Chan,
Pattie Maes
Abstract:
People have to remember an ever-expanding volume of information. Wearables that use information capture and retrieval for memory augmentation can help but can be disruptive and cumbersome in real-world tasks, such as in social settings. To address this, we developed Memoro, a wearable audio-based memory assistant with a concise user interface. Memoro uses a large language model (LLM) to infer the…
▽ More
People have to remember an ever-expanding volume of information. Wearables that use information capture and retrieval for memory augmentation can help but can be disruptive and cumbersome in real-world tasks, such as in social settings. To address this, we developed Memoro, a wearable audio-based memory assistant with a concise user interface. Memoro uses a large language model (LLM) to infer the user's memory needs in a conversational context, semantically search memories, and present minimal suggestions. The assistant has two interaction modes: Query Mode for voicing queries and Queryless Mode for on-demand predictive assistance, without explicit query. Our study of (N=20) participants engaged in a real-time conversation demonstrated that using Memoro reduced device interaction time and increased recall confidence while preserving conversational quality. We report quantitative results and discuss the preferences and experiences of users. This work contributes towards utilizing LLMs to design wearable memory augmentation systems that are minimally disruptive.
△ Less
Submitted 4 March, 2024;
originally announced March 2024.
-
Exploring the Impact of AI Value Alignment in Collaborative Ideation: Effects on Perception, Ownership, and Output
Authors:
Alicia Guo,
Pat Pataranutaporn,
Pattie Maes
Abstract:
AI-based virtual assistants are increasingly used to support daily ideation tasks. The values or bias present in these agents can influence output in hidden ways. They may also affect how people perceive the ideas produced with these AI agents and lead to implications for the design of AI-based tools. We explored the effects of AI agents with different values on the ideation process and user perce…
▽ More
AI-based virtual assistants are increasingly used to support daily ideation tasks. The values or bias present in these agents can influence output in hidden ways. They may also affect how people perceive the ideas produced with these AI agents and lead to implications for the design of AI-based tools. We explored the effects of AI agents with different values on the ideation process and user perception of idea quality, ownership, agent competence, and values present in the output. Our study tasked 180 participants with brainstorming practical solutions to a set of problems with AI agents of different values. Results show no significant difference in self-evaluation of idea quality and perception of the agent based on value alignment; however, ideas generated reflected the AI's values and feeling of ownership is affected. This highlights an intricate interplay between AI values and human ideation, suggesting careful design considerations for future AI-supported brainstorming tools.
△ Less
Submitted 22 April, 2024; v1 submitted 20 February, 2024;
originally announced February 2024.
-
BINGO innovative assembly for background reduction in bolometric $0νββ$ experiments
Authors:
A. Armatol,
C. Augier,
I. C. Bandac,
D. Baudin,
G. Benato,
V. Berest,
L. Bergé,
J. Billard,
J. M. Calvo-Mozota,
P. Carniti,
M. Chapellier,
F. A. Danevich,
M. De Jesus,
T. Dixon,
L. Dumoulin,
F. Ferri,
J. Gascon,
A. Giuliani,
H. Gomez,
C. Gotti,
Ph. Gras,
M. Gros,
A. Juillard,
H. Khalife,
V. V. Kobychev
, et al. (23 additional authors not shown)
Abstract:
BINGO is a project aiming to set the grounds for large-scale bolometric neutrinoless double-beta-decay experiments capable of investigating the effective Majorana neutrino mass at a few meV level. It focuses on develo** innovative technologies (a detector assembly, cryogenic photodetectors and active veto) to achieve a very low background index, of the order of $10^{-5}$ counts/(keV kg yr) in th…
▽ More
BINGO is a project aiming to set the grounds for large-scale bolometric neutrinoless double-beta-decay experiments capable of investigating the effective Majorana neutrino mass at a few meV level. It focuses on develo** innovative technologies (a detector assembly, cryogenic photodetectors and active veto) to achieve a very low background index, of the order of $10^{-5}$ counts/(keV kg yr) in the region of interest. The BINGO demonstrator, called MINI-BINGO, is designed to investigate the promising double-beta-decay isotopes $^{100}$Mo and $^{130}$Te and it will be composed of Li$_2$MoO$_4$ and TeO$_2$ crystals coupled to bolometric light detectors and surrounded by a Bi$_4$Ge$_3$O$_{12}$-based veto. This will allow us to reject a significant background in bolometers caused by surface contamination from $α$-active radionuclides by means of light yield selection and to mitigate other sources of background, such as surface contamination from $β$-active radionuclides, external $γ$ radioactivity, and pile-up due to random coincidence of background events. This paper describes an R\&D program towards the BINGO goals, particularly focusing on the development of an innovative assembly designed to reduce the passive materials within the line of sight of the detectors, which is expected to be a dominant source of background in next-generation bolometric experiments. We present the performance of two prototype modules -- housing four cubic (4.5-cm side) Li$_2$MoO$_4$ crystals in total -- operated in the Canfranc underground laboratory in Spain within a facility developed for the CROSS double-beta-decay experiment.
△ Less
Submitted 8 July, 2024; v1 submitted 19 February, 2024;
originally announced February 2024.
-
The virtual drum circle: polyrhythmic music interactions in extended reality
Authors:
Bavo Van Kerrebroeck,
Kristel Crombé,
Stéphanie Wilain,
Marc Leman,
Pieter-Jan Maes
Abstract:
Emerging technologies in the domain of extended reality offer rich, new possibilities for the study and practice of joint music performance. Apart from the technological challenges, bringing music players together in extended reality raises important questions on their performance and embodied coordination. In this study, we designed an extended reality platform to assess a remote, bidirectional p…
▽ More
Emerging technologies in the domain of extended reality offer rich, new possibilities for the study and practice of joint music performance. Apart from the technological challenges, bringing music players together in extended reality raises important questions on their performance and embodied coordination. In this study, we designed an extended reality platform to assess a remote, bidirectional polyrhythmic interaction between two players, mediated in real time by their three-dimensional embodied avatars and a shared, virtual drum circle. We leveraged a multi-layered analysis framework to assess their performance quality, embodied co-regulation and first-person interaction experience, using statistical techniques for time-series analysis and mixed-effect regression and focusing on contrasts of visual coupling (not seeing / seeing as avatars / seeing as real) and auditory context (metronome / music). Results reveal that an auditory context with music improved the performance output as measured by a prediction error, increased movement energy and levels of experienced agency. Visual coupling impacted experiential qualities and induced prosocial effects with increased levels of partner realism resulting in increased levels of shared agency and self-other merging. Embodied co-regulation between players was impacted by auditory context and visual coupling, suggesting prediction-based compensatory mechanisms to deal with the novelty, difficulty, and expressivity in the musical interaction. This study contributes to the understanding of music performance in extended reality by using a methodological approach to demonstrate how co-regulation between players is impacted by visual coupling and auditory context and provides a basis and future directions for further action-oriented research.
△ Less
Submitted 30 August, 2023; v1 submitted 3 August, 2023;
originally announced August 2023.
-
A first test of CUPID prototypal light detectors with NTD-Ge sensors in a pulse-tube cryostat
Authors:
CUPID collaboration,
K. Alfonso,
A. Armatol,
C. Augier,
F. T. Avignone III,
O. Azzolini,
M. Balata,
A. S. Barabash,
G. Bari,
A. Barresi,
D. Baudin,
F. Bellini,
G. Benato,
V. Berest,
M. Beretta,
M. Bettelli,
M. Biassoni,
J. Billard,
V. Boldrini,
A. Branca,
C. Brofferio,
C. Bucci,
J. Camilleri,
A. Campani,
C. Capelli
, et al. (154 additional authors not shown)
Abstract:
CUPID is a next-generation bolometric experiment aiming at searching for neutrinoless double-beta decay with ~250 kg of isotopic mass of $^{100}$Mo. It will operate at $\sim$10 mK in a cryostat currently hosting a similar-scale bolometric array for the CUORE experiment at the Gran Sasso National Laboratory (Italy). CUPID will be based on large-volume scintillating bolometers consisting of…
▽ More
CUPID is a next-generation bolometric experiment aiming at searching for neutrinoless double-beta decay with ~250 kg of isotopic mass of $^{100}$Mo. It will operate at $\sim$10 mK in a cryostat currently hosting a similar-scale bolometric array for the CUORE experiment at the Gran Sasso National Laboratory (Italy). CUPID will be based on large-volume scintillating bolometers consisting of $^{100}$Mo-enriched Li$_2$MoO$_4$ crystals, facing thin Ge-wafer-based bolometric light detectors. In the CUPID design, the detector structure is novel and needs to be validated. In particular, the CUORE cryostat presents a high level of mechanical vibrations due to the use of pulse tubes and the effect of vibrations on the detector performance must be investigated. In this paper we report the first test of the CUPID-design bolometric light detectors with NTD-Ge sensors in a dilution refrigerator equipped with a pulse tube in an above-ground lab. Light detectors are characterized in terms of sensitivity, energy resolution, pulse time constants, and noise power spectrum. Despite the challenging noisy environment due to pulse-tube-induced vibrations, we demonstrate that all the four tested light detectors comply with the CUPID goal in terms of intrinsic energy resolution of 100 eV RMS baseline noise. Indeed, we have measured 70--90 eV RMS for the four devices, which show an excellent reproducibility. We have also obtained outstanding energy resolutions at the 356 keV line from a $^{133}$Ba source with one light detector achieving 0.71(5) keV FWHM, which is -- to our knowledge -- the best ever obtained when compared to $γ$ detectors of any technology in this energy range.
△ Less
Submitted 10 April, 2023;
originally announced April 2023.
-
Twelve-crystal prototype of Li$_2$MoO$_4$ scintillating bolometers for CUPID and CROSS experiments
Authors:
CUPID,
CROSS collaborations,
:,
K. Alfonso,
A. Armatol,
C. Augier,
F. T. Avignone III,
O. Azzolini,
M. Balata,
I. C. Bandac,
A. S. Barabash,
G. Bari,
A. Barresi,
D. Baudin,
F. Bellini,
G. Benato,
V. Berest,
M. Beretta,
M. Bettelli,
M. Biassoni,
J. Billard,
V. Boldrini,
A. Branca,
C. Brofferio,
C. Bucci
, et al. (160 additional authors not shown)
Abstract:
An array of twelve 0.28 kg lithium molybdate (LMO) low-temperature bolometers equipped with 16 bolometric Ge light detectors, aiming at optimization of detector structure for CROSS and CUPID double-beta decay experiments, was constructed and tested in a low-background pulse-tube-based cryostat at the Canfranc underground laboratory in Spain. Performance of the scintillating bolometers was studied…
▽ More
An array of twelve 0.28 kg lithium molybdate (LMO) low-temperature bolometers equipped with 16 bolometric Ge light detectors, aiming at optimization of detector structure for CROSS and CUPID double-beta decay experiments, was constructed and tested in a low-background pulse-tube-based cryostat at the Canfranc underground laboratory in Spain. Performance of the scintillating bolometers was studied depending on the size of phonon NTD-Ge sensors glued to both LMO and Ge absorbers, shape of the Ge light detectors (circular vs. square, from two suppliers), in different light collection conditions (with and without reflector, with aluminum coated LMO crystal surface). The scintillating bolometer array was operated over 8 months in the low-background conditions that allowed to probe a very low, $μ$Bq/kg, level of the LMO crystals radioactive contamination by $^{228}$Th and $^{226}$Ra.
△ Less
Submitted 10 April, 2023;
originally announced April 2023.
-
Deceptive AI Systems That Give Explanations Are Just as Convincing as Honest AI Systems in Human-Machine Decision Making
Authors:
Valdemar Danry,
Pat Pataranutaporn,
Ziv Epstein,
Matthew Groh,
Pattie Maes
Abstract:
The ability to discern between true and false information is essential to making sound decisions. However, with the recent increase in AI-based disinformation campaigns, it has become critical to understand the influence of deceptive systems on human information processing. In experiment (N=128), we investigated how susceptible people are to deceptive AI systems by examining how their ability to d…
▽ More
The ability to discern between true and false information is essential to making sound decisions. However, with the recent increase in AI-based disinformation campaigns, it has become critical to understand the influence of deceptive systems on human information processing. In experiment (N=128), we investigated how susceptible people are to deceptive AI systems by examining how their ability to discern true news from fake news varies when AI systems are perceived as either human fact-checkers or AI fact-checking systems, and when explanations provided by those fact-checkers are either deceptive or honest. We find that deceitful explanations significantly reduce accuracy, indicating that people are just as likely to believe deceptive AI explanations as honest AI explanations. Although before getting assistance from an AI-system, people have significantly higher weighted discernment accuracy on false headlines than true headlines, we found that with assistance from an AI system, discernment accuracy increased significantly when given honest explanations on both true headlines and false headlines, and decreased significantly when given deceitful explanations on true headlines and false headlines. Further, we did not observe any significant differences in discernment between explanations perceived as coming from a human fact checker compared to an AI-fact checker. Similarly, we found no significant differences in trust. These findings exemplify the dangers of deceptive AI systems and the need for finding novel ways to limit their influence human information processing.
△ Less
Submitted 23 September, 2022;
originally announced October 2022.
-
Adaptive Virtual Neuroarchitecture
Authors:
Abhinandan Jain,
Pattie Maes,
Misha Sra
Abstract:
Our surrounding environment impacts our cognitive-emotional processes on a daily basis and shapes our physical, psychological and social wellbeing. Although the effects of the built environment on our psycho-physiological processes are well studied, virtual environment design with a potentially similar impact on the user, has received limited attention. Based on the influence of space design on a…
▽ More
Our surrounding environment impacts our cognitive-emotional processes on a daily basis and shapes our physical, psychological and social wellbeing. Although the effects of the built environment on our psycho-physiological processes are well studied, virtual environment design with a potentially similar impact on the user, has received limited attention. Based on the influence of space design on a user and combining that with the dynamic affordances of virtual spaces, we present the idea of adaptive virtual neuroarchitecture (AVN), where virtual environments respond to the user and the user's real world context while simultaneously influencing them both in realtime. To show how AVN has been explored in current research, we present a sampling of recent work that demonstrates reciprocal relationships using physical affordances (space, objects), the user's state (physiological, cognitive, emotional), and the virtual world used in the design of novel virtual reality experiences. We believe AVN has the potential to help us learn how to design spaces and environments that can enhance the wellbeing of their inhabitants.
△ Less
Submitted 10 July, 2022;
originally announced July 2022.
-
First cryogenic tests on BINGO innovations
Authors:
A. Armatol,
C. Augier,
D. Baudin,
G. Benato,
J. Billard,
P. Carniti,
M. Chapellier,
A. Charrier,
F. Danevich,
M. De Combarieu,
M. De Jesus,
L. Dumoulin,
F. Ferri,
J. Gascon,
A. Giuliani,
H. Gomez,
C. Gotti,
Ph. Gras,
M. Gros,
A. Juillard,
H. Khalife,
V. V. Kobychev,
M. Lefevre,
P. Loaiza,
S. Marnieros
, et al. (11 additional authors not shown)
Abstract:
Neutrinoless double-beta decay ($0\nu2β$) is a hypothetical rare nuclear transition. Its observation would provide an important insight about the nature of neutrinos (Dirac or Majorana particle) demonstrating that the lepton number is not conserved. BINGO (Bi-Isotope $0\nu2β$ Next Generation Observatory) aims to set the technological grounds for future bolometric $0\nu2β$ experiments. It is based…
▽ More
Neutrinoless double-beta decay ($0\nu2β$) is a hypothetical rare nuclear transition. Its observation would provide an important insight about the nature of neutrinos (Dirac or Majorana particle) demonstrating that the lepton number is not conserved. BINGO (Bi-Isotope $0\nu2β$ Next Generation Observatory) aims to set the technological grounds for future bolometric $0\nu2β$ experiments. It is based on a dual heat-light readout, i.e. a main scintillating absorber embedding the double-beta decay isotope accompanied by a cryogenic light detector. BINGO will study two of the most promising isotopes: $^{100}$Mo embedded in Li$_2$MoO$_4$ (LMO) crystals and $^{130}$Te embedded in TeO$_2$. BINGO technology will reduce dramatically the background in the region of interest, thus boosting the discovery sensitivity of $0\nu2β$. The proposed solutions will have a high impact on next-generation bolometric tonne-scale experiments, like CUPID. In this contribution, we present the results obtained during the first tests performed in the framework of BINGO R&D.
△ Less
Submitted 29 April, 2022;
originally announced April 2022.
-
Optimization of the first CUPID detector module
Authors:
CUPID collaboration,
A. Armatol,
C. Augier,
F. T. Avignone III,
O. Azzolini,
M. Balata,
K. Ballen,
A. S. Barabash,
G. Bari,
A. Barresi,
D. Baudin,
F. Bellini,
G. Benato,
M. Beretta,
M. Bettelli,
M. Biassoni,
J. Billard,
V. Boldrini,
A. Branca,
C. Brofferio,
C. Bucci,
J. Camilleri,
C. Capelli,
S. Capelli,
L. Cappelli
, et al. (153 additional authors not shown)
Abstract:
CUPID will be a next generation experiment searching for the neutrinoless double $β$ decay, whose discovery would establish the Majorana nature of the neutrino. Based on the experience achieved with the CUORE experiment, presently taking data at LNGS, CUPID aims to reach a background free environment by means of scintillating Li$_{2}$$^{100}$MoO$_4$ crystals coupled to light detectors. Indeed, the…
▽ More
CUPID will be a next generation experiment searching for the neutrinoless double $β$ decay, whose discovery would establish the Majorana nature of the neutrino. Based on the experience achieved with the CUORE experiment, presently taking data at LNGS, CUPID aims to reach a background free environment by means of scintillating Li$_{2}$$^{100}$MoO$_4$ crystals coupled to light detectors. Indeed, the simultaneous heat and light detection allows us to reject the dominant background of $α$ particles, as proven by the CUPID-0 and CUPID-Mo demonstrators. In this work we present the results of the first test of the CUPID baseline module. In particular, we propose a new optimized detector structure and light sensors design to enhance the engineering and the light collection, respectively. We characterized the heat detectors, achieving an energy resolution of (5.9 $\pm$ 0.2) keV FWHM at the $Q$-value of $^{100}$Mo (about 3034 keV). We studied the light collection of the baseline CUPID design with respect to an alternative configuration which features gravity-assisted light detectors' mounting. In both cases we obtained an improvement in the light collection with respect to past measures and we validated the particle identification capability of the detector, which ensures an $α$ particle rejection higher than 99.9%, fully satisfying the requirements for CUPID.
△ Less
Submitted 13 February, 2022;
originally announced February 2022.
-
Changing Computer-Usage Behaviours: What Users Want, Use, and Experience
Authors:
Mina Khan,
Zeel Patel,
Kathryn Wantlin,
Elena Glassman,
Pattie Maes
Abstract:
Technology based screentime, the time an individual spends engaging with their computer or cell phone, has increased exponentially over the past decade, but perhaps most alarmingly amidst the COVID-19 pandemic. Although many software based interventions exist to reduce screentime, users report a variety of issues relating to the timing of the intervention, the strictness of the tool, and its abili…
▽ More
Technology based screentime, the time an individual spends engaging with their computer or cell phone, has increased exponentially over the past decade, but perhaps most alarmingly amidst the COVID-19 pandemic. Although many software based interventions exist to reduce screentime, users report a variety of issues relating to the timing of the intervention, the strictness of the tool, and its ability to encourage organic, long-term habit formation. We develop guidelines for the design of behaviour intervention software by conducting a survey to investigate three research questions and further inform the mechanisms of computer-related behaviour change applications. RQ1: What do people want to change and why/how? RQ2: What applications do people use or have used, why do they work or not, and what additional support is desired? RQ3: What are helpful/unhelpful computer breaks and why? Our survey had 68 participants and three key findings. First, time management is a primary concern, but emotional and physical side-effects are equally important. Second, site blockers, self-trackers, and timers are commonly used, but they are ineffective as they are easy-to-ignore and not personalized. Third, away-from-computer breaks, especially involving physical activity, are helpful, whereas on-screen breaks are unhelpful, especially when they are long, because they are not refreshing. We recommend personalized and closed-loop computer-usage behaviour change support and especially encouraging off-the-computer screentime breaks.
△ Less
Submitted 2 January, 2022;
originally announced January 2022.
-
Txt2Vid: Ultra-Low Bitrate Compression of Talking-Head Videos via Text
Authors:
Pulkit Tandon,
Shubham Chandak,
Pat Pataranutaporn,
Yimeng Liu,
Anesu M. Mapuranga,
Pattie Maes,
Tsachy Weissman,
Misha Sra
Abstract:
Video represents the majority of internet traffic today, driving a continual race between the generation of higher quality content, transmission of larger file sizes, and the development of network infrastructure. In addition, the recent COVID-19 pandemic fueled a surge in the use of video conferencing tools. Since videos take up considerable bandwidth (~100 Kbps to a few Mbps), improved video com…
▽ More
Video represents the majority of internet traffic today, driving a continual race between the generation of higher quality content, transmission of larger file sizes, and the development of network infrastructure. In addition, the recent COVID-19 pandemic fueled a surge in the use of video conferencing tools. Since videos take up considerable bandwidth (~100 Kbps to a few Mbps), improved video compression can have a substantial impact on network performance for live and pre-recorded content, providing broader access to multimedia content worldwide. We present a novel video compression pipeline, called Txt2Vid, which dramatically reduces data transmission rates by compressing webcam videos ("talking-head videos") to a text transcript. The text is transmitted and decoded into a realistic reconstruction of the original video using recent advances in deep learning based voice cloning and lip syncing models. Our generative pipeline achieves two to three orders of magnitude reduction in the bitrate as compared to the standard audio-video codecs (encoders-decoders), while maintaining equivalent Quality-of-Experience based on a subjective evaluation by users (n = 242) in an online study. The Txt2Vid framework opens up the potential for creating novel applications such as enabling audio-video communication during poor internet connectivity, or in remote terrains with limited bandwidth. The code for this work is available at https://github.com/tpulkit/txt2vid.git.
△ Less
Submitted 2 April, 2022; v1 submitted 26 June, 2021;
originally announced June 2021.
-
Pretrained Encoders are All You Need
Authors:
Mina Khan,
P Srivatsa,
Advait Rane,
Shriram Chenniappa,
Rishabh Anand,
Sherjil Ozair,
Pattie Maes
Abstract:
Data-efficiency and generalization are key challenges in deep learning and deep reinforcement learning as many models are trained on large-scale, domain-specific, and expensive-to-label datasets. Self-supervised models trained on large-scale uncurated datasets have shown successful transfer to diverse settings. We investigate using pretrained image representations and spatio-temporal attention for…
▽ More
Data-efficiency and generalization are key challenges in deep learning and deep reinforcement learning as many models are trained on large-scale, domain-specific, and expensive-to-label datasets. Self-supervised models trained on large-scale uncurated datasets have shown successful transfer to diverse settings. We investigate using pretrained image representations and spatio-temporal attention for state representation learning in Atari. We also explore fine-tuning pretrained representations with self-supervised techniques, i.e., contrastive predictive coding, spatio-temporal contrastive learning, and augmentations. Our results show that pretrained representations are at par with state-of-the-art self-supervised methods trained on domain-specific data. Pretrained representations, thus, yield data and compute-efficient state representations. https://github.com/PAL-ML/PEARL_v1
△ Less
Submitted 9 June, 2021;
originally announced June 2021.
-
Personalizing Pre-trained Models
Authors:
Mina Khan,
P Srivatsa,
Advait Rane,
Shriram Chenniappa,
Asadali Hazariwala,
Pattie Maes
Abstract:
Self-supervised or weakly supervised models trained on large-scale datasets have shown sample-efficient transfer to diverse datasets in few-shot settings. We consider how upstream pretrained models can be leveraged for downstream few-shot, multilabel, and continual learning tasks. Our model CLIPPER (CLIP PERsonalized) uses image representations from CLIP, a large-scale image representation learnin…
▽ More
Self-supervised or weakly supervised models trained on large-scale datasets have shown sample-efficient transfer to diverse datasets in few-shot settings. We consider how upstream pretrained models can be leveraged for downstream few-shot, multilabel, and continual learning tasks. Our model CLIPPER (CLIP PERsonalized) uses image representations from CLIP, a large-scale image representation learning model trained using weak natural language supervision. We developed a technique, called Multi-label Weight Imprinting (MWI), for multi-label, continual, and few-shot learning, and CLIPPER uses MWI with image representations from CLIP. We evaluated CLIPPER on 10 single-label and 5 multi-label datasets. Our model shows robust and competitive performance, and we set new benchmarks for few-shot, multi-label, and continual learning. Our lightweight technique is also compute-efficient and enables privacy-preserving applications as the data is not sent to the upstream model for fine-tuning.
△ Less
Submitted 2 June, 2021;
originally announced June 2021.
-
The large inner Micromegas modules for the Atlas Muon Spectrometer Upgrade: construction, quality control and characterization
Authors:
J. Allard,
M. Anfreville,
N. Andari,
D. Attié,
S. Aune,
H. Bachacou,
F. Balli,
F. Bauer,
J. Bennet,
T. Benoit,
J. Beltramelli,
H. Bervas,
T. Bey,
S. Bouaziz,
M. Boyer,
T. Challey,
T. Chevalérias,
X. Copollani,
J. Costa,
G. Cara,
G. Decock,
F. Deliot,
D. Denysiuk,
D. Desforge,
G. Disset
, et al. (49 additional authors not shown)
Abstract:
The steadily increasing luminosity of the LHC requires an upgrade with high-rate and high-resolution detector technology for the inner end cap of the ATLAS muon spectrometer: the New Small Wheels (NSW). In order to achieve the goal of precision tracking at a hit rate of about 15 kHz/cm$^2$ at the inner radius of the NSW, large area Micromegas quadruplets with 100\,\microns spatial resolution per p…
▽ More
The steadily increasing luminosity of the LHC requires an upgrade with high-rate and high-resolution detector technology for the inner end cap of the ATLAS muon spectrometer: the New Small Wheels (NSW). In order to achieve the goal of precision tracking at a hit rate of about 15 kHz/cm$^2$ at the inner radius of the NSW, large area Micromegas quadruplets with 100\,\microns spatial resolution per plane have been produced. % IRFU, from the CEA research center of Saclay, is responsible for the production and validation of LM1 Micromegas modules. The construction, production, qualification and validation of the largest Micromegas detectors ever built are reported here. Performance results under cosmic muon characterisation will also be discussed.
△ Less
Submitted 28 May, 2021;
originally announced May 2021.
-
PAL: Intelligence Augmentation using Egocentric Visual Context Detection
Authors:
Mina Khan,
Pattie Maes
Abstract:
Egocentric visual context detection can support intelligence augmentation applications. We created a wearable system, called PAL, for wearable, personalized, and privacy-preserving egocentric visual context detection. PAL has a wearable device with a camera, heart-rate sensor, on-device deep learning, and audio input/output. PAL also has a mobile/web application for personalized context labeling.…
▽ More
Egocentric visual context detection can support intelligence augmentation applications. We created a wearable system, called PAL, for wearable, personalized, and privacy-preserving egocentric visual context detection. PAL has a wearable device with a camera, heart-rate sensor, on-device deep learning, and audio input/output. PAL also has a mobile/web application for personalized context labeling. We used on-device deep learning models for generic object and face detection, low-shot custom face and context recognition (e.g., activities like brushing teeth), and custom context clustering (e.g., indoor locations). The models had over 80\% accuracy in in-the-wild contexts (~1000 images) and we tested PAL for intelligence augmentation applications like behavior change. We have made PAL is open-source to further support intelligence augmentation using personalized and privacy-preserving egocentric visual contexts.
△ Less
Submitted 22 May, 2021;
originally announced May 2021.
-
Uncertainty-Aware Boosted Ensembling in Multi-Modal Settings
Authors:
Utkarsh Sarawgi,
Rishab Khincha,
Wazeer Zulfikar,
Satrajit Ghosh,
Pattie Maes
Abstract:
Reliability of machine learning (ML) systems is crucial in safety-critical applications such as healthcare, and uncertainty estimation is a widely researched method to highlight the confidence of ML systems in deployment. Sequential and parallel ensemble techniques have shown improved performance of ML systems in multi-modal settings by leveraging the feature sets together. We propose an uncertain…
▽ More
Reliability of machine learning (ML) systems is crucial in safety-critical applications such as healthcare, and uncertainty estimation is a widely researched method to highlight the confidence of ML systems in deployment. Sequential and parallel ensemble techniques have shown improved performance of ML systems in multi-modal settings by leveraging the feature sets together. We propose an uncertainty-aware boosting technique for multi-modal ensembling in order to focus on the data points with higher associated uncertainty estimates, rather than the ones with higher loss values. We evaluate this method on healthcare tasks related to Dementia and Parkinson's disease which involve real-world multi-modal speech and text data, wherein our method shows an improved performance. Additional analysis suggests that introducing uncertainty-awareness into the boosted ensembles decreases the overall entropy of the system, making it more robust to heteroscedasticity in the data, as well as better calibrating each of the modalities along with high quality prediction intervals. We open-source our entire codebase at https://github.com/usarawgi911/Uncertainty-aware-boosting
△ Less
Submitted 21 April, 2021;
originally announced April 2021.
-
Art and Science Interaction Lab -- A highly flexible and modular interaction science research facility
Authors:
Niels Van Kets,
Bart Moens,
Klaas Bombeke,
Wouter Durnez,
Pieter-Jan Maes,
Glenn Van Wallendael,
Lieven De Marez,
Marc Leman,
Peter Lambert
Abstract:
The Art and Science Interaction Lab (ASIL) is a unique, highly flexible and modular interaction science research facility to effectively bring, analyse and test experiences and interactions in mixed virtual/augmented contexts as well as to conduct research on next-gen immersive technologies. It brings together the expertise and creativity of engineers, performers, designers and scientists creating…
▽ More
The Art and Science Interaction Lab (ASIL) is a unique, highly flexible and modular interaction science research facility to effectively bring, analyse and test experiences and interactions in mixed virtual/augmented contexts as well as to conduct research on next-gen immersive technologies. It brings together the expertise and creativity of engineers, performers, designers and scientists creating solutions and experiences sha** the lives of people. The lab is equipped with state-of-the-art visual, auditory and user-tracking equipment, fully synchronized and connected to a central backend. This synchronization allows for highly accurate multi-sensor measurements and analysis.
△ Less
Submitted 27 January, 2021;
originally announced January 2021.
-
Robustness to Missing Features using Hierarchical Clustering with Split Neural Networks
Authors:
Rishab Khincha,
Utkarsh Sarawgi,
Wazeer Zulfikar,
Pattie Maes
Abstract:
The problem of missing data has been persistent for a long time and poses a major obstacle in machine learning and statistical data analysis. Past works in this field have tried using various data imputation techniques to fill in the missing data, or training neural networks (NNs) with the missing data. In this work, we propose a simple yet effective approach that clusters similar input features t…
▽ More
The problem of missing data has been persistent for a long time and poses a major obstacle in machine learning and statistical data analysis. Past works in this field have tried using various data imputation techniques to fill in the missing data, or training neural networks (NNs) with the missing data. In this work, we propose a simple yet effective approach that clusters similar input features together using hierarchical clustering and then trains proportionately split neural networks with a joint loss. We evaluate this approach on a series of benchmark datasets and show promising improvements even with simple imputation techniques. We attribute this to learning through clusters of similar features in our model architecture. The source code is available at https://github.com/usarawgi911/Robustness-to-Missing-Features
△ Less
Submitted 18 November, 2020;
originally announced November 2020.
-
Uncertainty-Aware Multi-Modal Ensembling for Severity Prediction of Alzheimer's Dementia
Authors:
Utkarsh Sarawgi,
Wazeer Zulfikar,
Rishab Khincha,
Pattie Maes
Abstract:
Reliability in Neural Networks (NNs) is crucial in safety-critical applications like healthcare, and uncertainty estimation is a widely researched method to highlight the confidence of NNs in deployment. In this work, we propose an uncertainty-aware boosting technique for multi-modal ensembling to predict Alzheimer's Dementia Severity. The propagation of uncertainty across acoustic, cognitive, and…
▽ More
Reliability in Neural Networks (NNs) is crucial in safety-critical applications like healthcare, and uncertainty estimation is a widely researched method to highlight the confidence of NNs in deployment. In this work, we propose an uncertainty-aware boosting technique for multi-modal ensembling to predict Alzheimer's Dementia Severity. The propagation of uncertainty across acoustic, cognitive, and linguistic features produces an ensemble system robust to heteroscedasticity in the data. Weighing the different modalities based on the uncertainty estimates, we experiment on the benchmark ADReSS dataset, a subject-independent and balanced dataset, to show that our method outperforms the state-of-the-art methods while also reducing the overall entropy of the system. This work aims to encourage fair and aware models. The source code is available at https://github.com/wazeerzulfikar/alzheimers-dementia
△ Less
Submitted 18 November, 2020; v1 submitted 3 October, 2020;
originally announced October 2020.
-
Why have a Unified Predictive Uncertainty? Disentangling it using Deep Split Ensembles
Authors:
Utkarsh Sarawgi,
Wazeer Zulfikar,
Rishab Khincha,
Pattie Maes
Abstract:
Understanding and quantifying uncertainty in black box Neural Networks (NNs) is critical when deployed in real-world settings such as healthcare. Recent works using Bayesian and non-Bayesian methods have shown how a unified predictive uncertainty can be modelled for NNs. Decomposing this uncertainty to disentangle the granular sources of heteroscedasticity in data provides rich information about i…
▽ More
Understanding and quantifying uncertainty in black box Neural Networks (NNs) is critical when deployed in real-world settings such as healthcare. Recent works using Bayesian and non-Bayesian methods have shown how a unified predictive uncertainty can be modelled for NNs. Decomposing this uncertainty to disentangle the granular sources of heteroscedasticity in data provides rich information about its underlying causes. We propose a conceptually simple non-Bayesian approach, deep split ensemble, to disentangle the predictive uncertainties using a multivariate Gaussian mixture model. The NNs are trained with clusters of input features, for uncertainty estimates per cluster. We evaluate our approach on a series of benchmark regression datasets, while also comparing with unified uncertainty methods. Extensive analyses using dataset shits and empirical rule highlight our inherently well-calibrated models. Our work further demonstrates its applicability in a multi-modal setting using a benchmark Alzheimer's dataset and also shows how deep split ensembles can highlight hidden modality-specific biases. The minimal changes required to NNs and the training procedure, and the high flexibility to group features into clusters makes it readily deployable and useful. The source code is available at https://github.com/wazeerzulfikar/deep-split-ensembles
△ Less
Submitted 25 September, 2020;
originally announced September 2020.
-
Multimodal Inductive Transfer Learning for Detection of Alzheimer's Dementia and its Severity
Authors:
Utkarsh Sarawgi,
Wazeer Zulfikar,
Nouran Soliman,
Pattie Maes
Abstract:
Alzheimer's disease is estimated to affect around 50 million people worldwide and is rising rapidly, with a global economic burden of nearly a trillion dollars. This calls for scalable, cost-effective, and robust methods for detection of Alzheimer's dementia (AD). We present a novel architecture that leverages acoustic, cognitive, and linguistic features to form a multimodal ensemble system. It us…
▽ More
Alzheimer's disease is estimated to affect around 50 million people worldwide and is rising rapidly, with a global economic burden of nearly a trillion dollars. This calls for scalable, cost-effective, and robust methods for detection of Alzheimer's dementia (AD). We present a novel architecture that leverages acoustic, cognitive, and linguistic features to form a multimodal ensemble system. It uses specialized artificial neural networks with temporal characteristics to detect AD and its severity, which is reflected through Mini-Mental State Exam (MMSE) scores. We first evaluate it on the ADReSS challenge dataset, which is a subject-independent and balanced dataset matched for age and gender to mitigate biases, and is available through DementiaBank. Our system achieves state-of-the-art test accuracy, precision, recall, and F1-score of 83.3% each for AD classification, and state-of-the-art test root mean squared error (RMSE) of 4.60 for MMSE score regression. To the best of our knowledge, the system further achieves state-of-the-art AD classification accuracy of 88.0% when evaluated on the full benchmark DementiaBank Pitt database. Our work highlights the applicability and transferability of spontaneous speech to produce a robust inductive transfer learning model, and demonstrates generalizability through a task-agnostic feature-space. The source code is available at https://github.com/wazeerzulfikar/alzheimers-dementia
△ Less
Submitted 30 August, 2020;
originally announced September 2020.
-
Using social media to measure demographic responses to natural disaster: Insights from a large-scale Facebook survey following the 2019 Australia Bushfires
Authors:
Paige Maas,
Zack Almquist,
Eugenia Giraudy,
JW Schneider
Abstract:
In this paper we explore a novel method for collecting survey data following a natural disaster and then combine this data with device-derived mobility information to explore demographic outcomes. Using social media as a survey platform for measuring demographic outcomes, especially those that are challenging or expensive to field for, is increasingly of interest to the demographic community. Rece…
▽ More
In this paper we explore a novel method for collecting survey data following a natural disaster and then combine this data with device-derived mobility information to explore demographic outcomes. Using social media as a survey platform for measuring demographic outcomes, especially those that are challenging or expensive to field for, is increasingly of interest to the demographic community. Recent work by Schneider and Harknett (2019) explores the use of Facebook targeted advertisements to collect data on low-income shift workers in the United States. Other work has addressed immigrant assimilation (Stewart et al, 2019), world fertility (Ribeiro et al, 2020), and world migration stocks (Zagheni et al, 2017). We build on this work by introducing a rapid-response survey of post-disaster demographic and economic outcomes fielded through the Facebook app itself. We use these survey responses to augment app-derived mobility data that comprises Facebook Displacement Maps to assess the validity of and drivers underlying those observed behavioral trends. This survey was deployed following the 2019 Australia bushfires to better understand how these events displaced residents. In doing so we are able to test a number of key hypotheses around displacement and demographics. In particular, we uncover several gender differences in key areas, including in displacement decision-making and timing, and in access to protective equipment such as smoke masks. We conclude with a brief discussion of research and policy implications.
△ Less
Submitted 9 August, 2020;
originally announced August 2020.
-
PAL: A Wearable Platform for Real-time, Personalized and Context-Aware Health and Cognition Support
Authors:
Mina Khan,
Glenn Fernandes,
Utkarsh Sarawgi,
Prudhvi Rampey,
Pattie Maes
Abstract:
Personalized Active Learner (PAL) is a wearable system for real-time, personalized, and context-aware health and cognition support. PAL's system consists of a wearable device, mobile app, cloud database, data visualization web app, and machine learning server. PAL's wearable device uses multi-modal sensors (camera, microphone, heart-rate) with on-device machine learning and open-ear audio output t…
▽ More
Personalized Active Learner (PAL) is a wearable system for real-time, personalized, and context-aware health and cognition support. PAL's system consists of a wearable device, mobile app, cloud database, data visualization web app, and machine learning server. PAL's wearable device uses multi-modal sensors (camera, microphone, heart-rate) with on-device machine learning and open-ear audio output to provide real-time and context-aware cognitive, behavioral and psychological interventions. PAL also allows users to track the long-term correlations between their activities and physiological states to make well-informed lifestyle decisions. In this paper, we present and open-source PAL's system so that people can use it for health and cognition support applications. We also open-source three fully-developed example applications using PAL for face-based memory augmentation, contextual language learning, and heart-rate-based psychological support. PAL's flexible, modular and extensible platform combines trends in data-driven medicine, mobile psychology, and cognitive enhancement to support data-driven and empowering health and cognition applications.
△ Less
Submitted 3 May, 2019;
originally announced May 2019.
-
Real-Time Sleep Staging using Deep Learning on a Smartphone for a Wearable EEG
Authors:
Abhay Koushik,
Judith Amores,
Pattie Maes
Abstract:
We present the first real-time sleep staging system that uses deep learning without the need for servers in a smartphone application for a wearable EEG. We employ real-time adaptation of a single channel Electroencephalography (EEG) to infer from a Time-Distributed 1-D Deep Convolutional Neural Network. Polysomnography (PSG)-the gold standard for sleep staging, requires a human scorer and is both…
▽ More
We present the first real-time sleep staging system that uses deep learning without the need for servers in a smartphone application for a wearable EEG. We employ real-time adaptation of a single channel Electroencephalography (EEG) to infer from a Time-Distributed 1-D Deep Convolutional Neural Network. Polysomnography (PSG)-the gold standard for sleep staging, requires a human scorer and is both complex and resource-intensive. Our work demonstrates an end-to-end on-smartphone pipeline that can infer sleep stages in just single 30-second epochs, with an overall accuracy of 83.5% on 20-fold cross validation for five-class classification of sleep stages using the open Sleep-EDF dataset.
△ Less
Submitted 27 November, 2018; v1 submitted 25 November, 2018;
originally announced November 2018.
-
Search Intelligence: Deep Learning For Dominant Category Prediction
Authors:
Zeeshan Khawar Malik,
Mo Kobrosli,
Peter Maas
Abstract:
Deep Neural Networks, and specifically fully-connected convolutional neural networks are achieving remarkable results across a wide variety of domains. They have been trained to achieve state-of-the-art performance when applied to problems such as speech recognition, image classification, natural language processing and bioinformatics. Most of these deep learning models when applied to classificat…
▽ More
Deep Neural Networks, and specifically fully-connected convolutional neural networks are achieving remarkable results across a wide variety of domains. They have been trained to achieve state-of-the-art performance when applied to problems such as speech recognition, image classification, natural language processing and bioinformatics. Most of these deep learning models when applied to classification employ the softmax activation function for prediction and aim to minimize cross-entropy loss. In this paper, we have proposed a supervised model for dominant category prediction to improve search recall across all eBay classifieds platforms. The dominant category label for each query in the last 90 days is first calculated by summing the total number of collaborative clicks among all categories. The category having the highest number of collaborative clicks for the given query will be considered its dominant category. Second, each query is transformed to a numeric vector by map** each unique word in the query document to a unique integer value; all padded to equal length based on the maximum document length within the pre-defined vocabulary size. A fully-connected deep convolutional neural network (CNN) is then applied for classification. The proposed model achieves very high classification accuracy compared to other state-of-the-art machine learning techniques.
△ Less
Submitted 6 February, 2017;
originally announced February 2017.
-
Effective medium theory of conduction in stretched polymer electrolytes
Authors:
Oliver Duerr,
Wolfgang Dieterich,
Philipp Maas,
Abraham Nitzan
Abstract:
Recent experimental observations of anisotropic conductivity in stretched polymer electrolytes films of the polyethylene oxide family are discussed. The main experimental observations, enhancement of the ionic diffusion and conductivity in the stretch direction and decrease in these transport coefficients in the normal direction are interpreted in terms of an effective two-phase model. This two-…
▽ More
Recent experimental observations of anisotropic conductivity in stretched polymer electrolytes films of the polyethylene oxide family are discussed. The main experimental observations, enhancement of the ionic diffusion and conductivity in the stretch direction and decrease in these transport coefficients in the normal direction are interpreted in terms of an effective two-phase model. This two-phase model is based on the idea that a highly conducting phase is associated with oriented molecular structures which are surrounded by poorly conducting boundary regions. This model is evaluated within the framework of differential effective medium theory (DEMT). Under stretching these regions change from spherical to prolate-spheroidal shapes. The computed dependence of the DC conductivity tensor and its AC counterpart on the stretch parameters is in good agreement with experimental results.
△ Less
Submitted 9 February, 2002;
originally announced February 2002.
