-
Learning Continually by Spectral Regularization
Authors:
Alex Lewandowski,
Saurabh Kumar,
Dale Schuurmans,
András György,
Marlos C. Machado
Abstract:
Loss of plasticity is a phenomenon where neural networks become more difficult to train during the course of learning. Continual learning algorithms seek to mitigate this effect by sustaining good predictive performance while maintaining network trainability. We develop new techniques for improving continual learning by first reconsidering how initialization can ensure trainability during early ph…
▽ More
Loss of plasticity is a phenomenon where neural networks become more difficult to train during the course of learning. Continual learning algorithms seek to mitigate this effect by sustaining good predictive performance while maintaining network trainability. We develop new techniques for improving continual learning by first reconsidering how initialization can ensure trainability during early phases of learning. From this perspective, we derive new regularization strategies for continual learning that ensure beneficial initialization properties are better maintained throughout training. In particular, we investigate two new regularization techniques for continual learning: (i) Wasserstein regularization toward the initial weight distribution, which is less restrictive than regularizing toward initial weights; and (ii) regularizing weight matrix singular values, which directly ensures gradient diversity is maintained throughout training. We present an experimental analysis that shows these alternative regularizers can improve continual learning performance across a range of supervised learning tasks and model architectures. The alternative regularizers prove to be less sensitive to hyperparameters while demonstrating better training in individual tasks, sustaining trainability as new tasks arrive, and achieving better generalization performance.
△ Less
Submitted 10 June, 2024;
originally announced June 2024.
-
Directions of Curvature as an Explanation for Loss of Plasticity
Authors:
Alex Lewandowski,
Haruto Tanaka,
Dale Schuurmans,
Marlos C. Machado
Abstract:
Loss of plasticity is a phenomenon in which neural networks lose their ability to learn from new experience. Despite being empirically observed in several problem settings, little is understood about the mechanisms that lead to loss of plasticity. In this paper, we offer a consistent explanation for loss of plasticity: Neural networks lose directions of curvature during training and that loss of p…
▽ More
Loss of plasticity is a phenomenon in which neural networks lose their ability to learn from new experience. Despite being empirically observed in several problem settings, little is understood about the mechanisms that lead to loss of plasticity. In this paper, we offer a consistent explanation for loss of plasticity: Neural networks lose directions of curvature during training and that loss of plasticity can be attributed to this reduction in curvature. To support such a claim, we provide a systematic investigation of loss of plasticity across continual learning tasks using MNIST, CIFAR-10 and ImageNet. Our findings illustrate that loss of curvature directions coincides with loss of plasticity, while also showing that previous explanations are insufficient to explain loss of plasticity in all settings. Lastly, we show that regularizers which mitigate loss of plasticity also preserve curvature, motivating a simple distributional regularizer that proves to be effective across the problem settings we considered.
△ Less
Submitted 27 June, 2024; v1 submitted 30 November, 2023;
originally announced December 2023.
-
Sharing Experience Around Component Compositions
Authors:
Grégory Bourguin,
Arnaud Lewandowski,
Myriam Lewkowicz
Abstract:
Society currently lives in a world of tailorable systems in which end-users are able to transform their working environment while achieving their tasks, day to day and over the time. Tailorability is most of the time achieved through dynamic component integration thanks to a huge number of components available over the Internet. In this context, the main problem for users is not anymore the integr…
▽ More
Society currently lives in a world of tailorable systems in which end-users are able to transform their working environment while achieving their tasks, day to day and over the time. Tailorability is most of the time achieved through dynamic component integration thanks to a huge number of components available over the Internet. In this context, the main problem for users is not anymore the integration of new components, but how to find the most interesting set of components that will fulfill their needs. Facing this issue, the authors' assumption is that it would be helpful for users to take benefit of the experience of other users and our work aims at enhancing current software ecosystems to support this sharing of experience. The authors have applied this approach in the context of software development while considering Eclipse as one of the most advanced and used software ecosystem. The authors then offer ShareXP, an Eclipse feature that allows members of a group to share their expertise, this expertise being embodied in the ``compositions'' each of them has built. ShareXP was already presented in (Bourguin et al., 2012). The current paper is an extension where the authors deeper show that ShareXP is only a first step in their global approach trying to enhance not only the Eclipse ecosystem, but software ecosystems in general.
△ Less
Submitted 30 November, 2023;
originally announced November 2023.
-
Reinforcement Teaching
Authors:
Alex Lewandowski,
Calarina Muslimani,
Dale Schuurmans,
Matthew E. Taylor,
Jun Luo
Abstract:
Meta-learning strives to learn about and improve a student's machine learning algorithm. However, existing meta-learning methods either only work with differentiable algorithms or are hand-crafted to improve one specific component of an algorithm. We develop a unifying meta-learning framework, called Reinforcement Teaching, to improve the learning process of any algorithm. Under Reinforcement Teac…
▽ More
Meta-learning strives to learn about and improve a student's machine learning algorithm. However, existing meta-learning methods either only work with differentiable algorithms or are hand-crafted to improve one specific component of an algorithm. We develop a unifying meta-learning framework, called Reinforcement Teaching, to improve the learning process of any algorithm. Under Reinforcement Teaching, a teaching policy is learned, through reinforcement, to improve a student's learning. To effectively learn such a teaching policy, we introduce a parametric-behavior embedder that learns a representation of the student's learnable parameters from its input/output behavior. Further, we use learning progress to shape the teacher's reward, allowing it to more quickly maximize the student's performance. To demonstrate the generality of Reinforcement Teaching, we conduct experiments where a teacher learns to significantly improve both reinforcement and supervised learning algorithms, outperforming hand-crafted heuristics and previously proposed parameter representations. Results show that Reinforcement Teaching is capable of not only unifying different meta-learning approaches, but also effectively leveraging existing tools from reinforcement learning research.
△ Less
Submitted 22 May, 2022; v1 submitted 25 April, 2022;
originally announced April 2022.
-
ZORB: A Derivative-Free Backpropagation Algorithm for Neural Networks
Authors:
Varun Ranganathan,
Alex Lewandowski
Abstract:
Gradient descent and backpropagation have enabled neural networks to achieve remarkable results in many real-world applications. Despite ongoing success, training a neural network with gradient descent can be a slow and strenuous affair. We present a simple yet faster training algorithm called Zeroth-Order Relaxed Backpropagation (ZORB). Instead of calculating gradients, ZORB uses the pseudoinvers…
▽ More
Gradient descent and backpropagation have enabled neural networks to achieve remarkable results in many real-world applications. Despite ongoing success, training a neural network with gradient descent can be a slow and strenuous affair. We present a simple yet faster training algorithm called Zeroth-Order Relaxed Backpropagation (ZORB). Instead of calculating gradients, ZORB uses the pseudoinverse of targets to backpropagate information. ZORB is designed to reduce the time required to train deep neural networks without penalizing performance. To illustrate the speed up, we trained a feed-forward neural network with 11 layers on MNIST and observed that ZORB converged 300 times faster than Adam while achieving a comparable error rate, without any hyperparameter tuning. We also broaden the scope of ZORB to convolutional neural networks, and apply it to subsamples of the CIFAR-10 dataset. Experiments on standard classification and regression benchmarks demonstrate ZORB's advantage over traditional backpropagation with Gradient Descent.
△ Less
Submitted 17 November, 2020;
originally announced November 2020.
