Tuned Compositional Feature Replays for Efficient Stream Learning
Authors:
Morgan B. Talbot,
Rushikesh Zawar,
Rohil Badkundri,
Mengmi Zhang,
Gabriel Kreiman
Abstract:
Our brains extract durable, generalizable knowledge from transient experiences of the world. Artificial neural networks come nowhere close to this ability. When tasked with learning to classify objects by training on non-repeating video frames in temporal order (online stream learning), models that learn well from shuffled datasets catastrophically forget old knowledge upon learning new stimuli. W…
▽ More
Our brains extract durable, generalizable knowledge from transient experiences of the world. Artificial neural networks come nowhere close to this ability. When tasked with learning to classify objects by training on non-repeating video frames in temporal order (online stream learning), models that learn well from shuffled datasets catastrophically forget old knowledge upon learning new stimuli. We propose a new continual learning algorithm, Compositional Replay Using Memory Blocks (CRUMB), which mitigates forgetting by replaying feature maps reconstructed by combining generic parts. CRUMB concatenates trainable and re-usable "memory block" vectors to compositionally reconstruct feature map tensors in convolutional neural networks. Storing the indices of memory blocks used to reconstruct new stimuli enables memories of the stimuli to be replayed during later tasks. This reconstruction mechanism also primes the neural network to minimize catastrophic forgetting by biasing it towards attending to information about object shapes more than information about image textures, and stabilizes the network during stream learning by providing a shared feature-level basis for all training examples. These properties allow CRUMB to outperform an otherwise identical algorithm that stores and replays raw images, while occupying only 3.6% as much memory. We stress-tested CRUMB alongside 13 competing methods on 7 challenging datasets. To address the limited number of existing online stream learning datasets, we introduce 2 new benchmarks by adapting existing datasets for stream learning. With only 3.7-4.1% as much memory and 15-43% as much runtime, CRUMB mitigates catastrophic forgetting more effectively than the state-of-the-art. Our code is available at https://github.com/MorganBDT/crumb.git.
△ Less
Submitted 2 January, 2024; v1 submitted 5 April, 2021;
originally announced April 2021.
Forecasting the 2017-2018 Yemen Cholera Outbreak with Machine Learning
Authors:
Rohil Badkundri,
Victor Valbuena,
Srikusmanjali Pinnamareddy,
Brittney Cantrell,
Janet Standeven
Abstract:
The ongoing Yemen cholera outbreak has been deemed one of the worst cholera outbreaks in history, with over a million people impacted and thousands dead. Triggered by a civil war, the outbreak has been shaped by various political, environmental, and epidemiological factors and continues to worsen. While cholera has several effective treatments, the untimely and inefficient distribution of existing…
▽ More
The ongoing Yemen cholera outbreak has been deemed one of the worst cholera outbreaks in history, with over a million people impacted and thousands dead. Triggered by a civil war, the outbreak has been shaped by various political, environmental, and epidemiological factors and continues to worsen. While cholera has several effective treatments, the untimely and inefficient distribution of existing medicines has been the primary cause of cholera mortality. With the hope of facilitating resource allocation, various mathematical models have been created to track the Yemeni outbreak and identify at-risk administrative divisions, called governorates. Existing models are not powerful enough to accurately and consistently forecast cholera cases per governorate over multiple timeframes. To address the need for a complex, reliable model, we offer the Cholera Artificial Learning Model (CALM); a system of 4 extreme-gradient-boosting (XGBoost) machine learning models that forecast the number of new cholera cases a Yemeni governorate will experience from a time range of 2 weeks to 2 months. CALM provides a novel machine learning approach that makes use of rainfall data, past cholera cases and deaths data, civil war fatalities, and inter-governorate interactions represented across multiple time frames. Additionally, the use of machine learning, along with extensive feature engineering, allows CALM to easily learn complex non-linear relations apparent in an epidemiological phenomenon. CALM is able to forecast cholera incidence 2 weeks to 2 months in advance within a margin of just 5 cholera cases per 10,000 people in real-world simulation.
△ Less
Submitted 16 February, 2019;
originally announced February 2019.
