-
Finding Increasingly Large Extremal Graphs with AlphaZero and Tabu Search
Authors:
Abbas Mehrabian,
Ankit Anand,
Hyunjik Kim,
Nicolas Sonnerat,
Matej Balog,
Gheorghe Comanici,
Tudor Berariu,
Andrew Lee,
Anian Ruoss,
Anna Bulanova,
Daniel Toyama,
Sam Blackwell,
Bernardino Romera Paredes,
Petar Veličković,
Laurent Orseau,
Joonkyung Lee,
Anurag Murty Naredla,
Doina Precup,
Adam Zsolt Wagner
Abstract:
This work studies a central extremal graph theory problem inspired by a 1975 conjecture of Erdős, which aims to find graphs with a given size (number of nodes) that maximize the number of edges without having 3- or 4-cycles. We formulate this problem as a sequential decision-making problem and compare AlphaZero, a neural network-guided tree search, with tabu search, a heuristic local search method…
▽ More
This work studies a central extremal graph theory problem inspired by a 1975 conjecture of Erdős, which aims to find graphs with a given size (number of nodes) that maximize the number of edges without having 3- or 4-cycles. We formulate this problem as a sequential decision-making problem and compare AlphaZero, a neural network-guided tree search, with tabu search, a heuristic local search method. Using either method, by introducing a curriculum -- jump-starting the search for larger graphs using good graphs found at smaller sizes -- we improve the state-of-the-art lower bounds for several sizes. We also propose a flexible graph-generation environment and a permutation-invariant network architecture for learning to search in the space of graphs.
△ Less
Submitted 6 November, 2023;
originally announced November 2023.
-
When Does Re-initialization Work?
Authors:
Sheheryar Zaidi,
Tudor Berariu,
Hyunjik Kim,
Jörg Bornschein,
Claudia Clopath,
Yee Whye Teh,
Razvan Pascanu
Abstract:
Re-initializing a neural network during training has been observed to improve generalization in recent works. Yet it is neither widely adopted in deep learning practice nor is it often used in state-of-the-art training protocols. This raises the question of when re-initialization works, and whether it should be used together with regularization techniques such as data augmentation, weight decay an…
▽ More
Re-initializing a neural network during training has been observed to improve generalization in recent works. Yet it is neither widely adopted in deep learning practice nor is it often used in state-of-the-art training protocols. This raises the question of when re-initialization works, and whether it should be used together with regularization techniques such as data augmentation, weight decay and learning rate schedules. In this work, we conduct an extensive empirical comparison of standard training with a selection of re-initialization methods to answer this question, training over 15,000 models on a variety of image classification benchmarks. We first establish that such methods are consistently beneficial for generalization in the absence of any other regularization. However, when deployed alongside other carefully tuned regularization techniques, re-initialization methods offer little to no added benefit for generalization, although optimal generalization performance becomes less sensitive to the choice of learning rate and weight decay hyperparameters. To investigate the impact of re-initialization methods on noisy data, we also consider learning under label noise. Surprisingly, in this case, re-initialization significantly improves upon standard training, even in the presence of other carefully tuned regularization techniques.
△ Less
Submitted 2 April, 2023; v1 submitted 20 June, 2022;
originally announced June 2022.
-
A study on the plasticity of neural networks
Authors:
Tudor Berariu,
Wojciech Czarnecki,
Soham De,
Jorg Bornschein,
Samuel Smith,
Razvan Pascanu,
Claudia Clopath
Abstract:
One aim shared by multiple settings, such as continual learning or transfer learning, is to leverage previously acquired knowledge to converge faster on the current task. Usually this is done through fine-tuning, where an implicit assumption is that the network maintains its plasticity, meaning that the performance it can reach on any given task is not affected negatively by previously seen tasks.…
▽ More
One aim shared by multiple settings, such as continual learning or transfer learning, is to leverage previously acquired knowledge to converge faster on the current task. Usually this is done through fine-tuning, where an implicit assumption is that the network maintains its plasticity, meaning that the performance it can reach on any given task is not affected negatively by previously seen tasks. It has been observed recently that a pretrained model on data from the same distribution as the one it is fine-tuned on might not reach the same generalisation as a freshly initialised one. We build and extend this observation, providing a hypothesis for the mechanics behind it. We discuss the implication of losing plasticity for continual learning which heavily relies on optimising pretrained models.
△ Less
Submitted 14 October, 2023; v1 submitted 31 May, 2021;
originally announced June 2021.
-
Spectral Normalisation for Deep Reinforcement Learning: an Optimisation Perspective
Authors:
Florin Gogianu,
Tudor Berariu,
Mihaela Rosca,
Claudia Clopath,
Lucian Busoniu,
Razvan Pascanu
Abstract:
Most of the recent deep reinforcement learning advances take an RL-centric perspective and focus on refinements of the training objective. We diverge from this view and show we can recover the performance of these developments not by changing the objective, but by regularising the value-function estimator. Constraining the Lipschitz constant of a single layer using spectral normalisation is suffic…
▽ More
Most of the recent deep reinforcement learning advances take an RL-centric perspective and focus on refinements of the training objective. We diverge from this view and show we can recover the performance of these developments not by changing the objective, but by regularising the value-function estimator. Constraining the Lipschitz constant of a single layer using spectral normalisation is sufficient to elevate the performance of a Categorical-DQN agent to that of a more elaborated \rainbow{} agent on the challenging Atari domain. We conduct ablation studies to disentangle the various effects normalisation has on the learning dynamics and show that is sufficient to modulate the parameter updates to recover most of the performance of spectral normalisation. These findings hint towards the need to also focus on the neural component and its learning dynamics to tackle the peculiarities of Deep Reinforcement Learning.
△ Less
Submitted 11 May, 2021;
originally announced May 2021.
