-
Glassy dynamics in deep neural networks: A structural comparison
Authors:
Max Kerr Winter,
Liesbeth M. C. Janssen
Abstract:
Deep Neural Networks (DNNs) share important similarities with structural glasses. Both have many degrees of freedom, and their dynamics are governed by a high-dimensional, non-convex landscape representing either the loss or energy, respectively. Furthermore, both experience gradient descent dynamics subject to noise. In this work we investigate, by performing quantitative measurements on realisti…
▽ More
Deep Neural Networks (DNNs) share important similarities with structural glasses. Both have many degrees of freedom, and their dynamics are governed by a high-dimensional, non-convex landscape representing either the loss or energy, respectively. Furthermore, both experience gradient descent dynamics subject to noise. In this work we investigate, by performing quantitative measurements on realistic networks trained on the MNIST and CIFAR-10 datasets, the extent to which this qualitative similarity gives rise to glass-like dynamics in neural networks. We demonstrate the existence of a Topology Trivialisation Transition as well as the previously studied under-to-overparameterised transition analogous to jamming. By training DNNs with overdamped Langevin dynamics in the resulting disordered phases, we do not observe diverging relaxation times at non-zero temperature, nor do we observe any caging effects, in contrast to glass phenomenology. However, the weight overlap function follows a power law in time, with an exponent of approximately -0.5, in agreement with the Mode-Coupling Theory of structural glasses. In addition, the DNN dynamics obey a form of time-temperature superposition. Finally, dynamic heterogeneity and ageing are observed at low temperatures. These results highlight important and surprising points of both difference and agreement between the behaviour of DNNs and structural glasses.
△ Less
Submitted 24 May, 2024; v1 submitted 21 May, 2024;
originally announced May 2024.
-
A deep learning approach to the measurement of long-lived memory kernels from Generalised Langevin Dynamics
Authors:
Max Kerr Winter,
Ilian Pihlajamaa,
Vincent E. Debets,
Liesbeth M. C. Janssen
Abstract:
Memory effects are ubiquitous in a wide variety of complex physical phenomena, ranging from glassy dynamics and metamaterials to climate models. The Generalised Langevin Equation (GLE) provides a rigorous way to describe memory effects via the so-called memory kernel in an integro-differential equation. However, the memory kernel is often unknown, and accurately predicting or measuring it via e.g.…
▽ More
Memory effects are ubiquitous in a wide variety of complex physical phenomena, ranging from glassy dynamics and metamaterials to climate models. The Generalised Langevin Equation (GLE) provides a rigorous way to describe memory effects via the so-called memory kernel in an integro-differential equation. However, the memory kernel is often unknown, and accurately predicting or measuring it via e.g. a numerical inverse Laplace transform remains a herculean task. Here we describe a novel method using deep neural networks (DNNs) to measure memory kernels from dynamical data. As proof-of-principle, we focus on the notoriously long-lived memory effects of glassy systems, which have proved a major challenge to existing methods. Specifically, we learn a training set generated with the Mode-Coupling Theory (MCT) of hard spheres. Our DNNs are remarkably robust against noise, in contrast to conventional techniques which require ensemble averaging over many independent trajectories. Finally, we demonstrate that a network trained on data generated from analytic theory (hard-sphere MCT) generalises well to data from simulations of a different system (Brownian Weeks-Chandler-Andersen particles). We provide a general pipeline, KernelLearner, for training networks to extract memory kernels from any non-Markovian system described by a GLE. The success of our DNN method applied to glassy systems suggests deep learning can play an important role in the study of dynamical systems that exhibit memory effects.
△ Less
Submitted 28 June, 2023; v1 submitted 27 February, 2023;
originally announced February 2023.
-
Tissue hydraulics: physics of lumen formation and interaction
Authors:
Alejandro Torres-Sánchez,
Max Kerr Winter,
Guillaume Salbreux
Abstract:
Lumen formation plays an essential role in the morphogenesis of tissues during development. Here we review the physical principles that play a role in the growth and coarsening of lumens. Solute pum** by the cell, hydraulic flows driven by differences of osmotic and hydrostatic pressures, balance of forces between extracellular fluids and cell-generated cytoskeletal forces, and electro-osmotic e…
▽ More
Lumen formation plays an essential role in the morphogenesis of tissues during development. Here we review the physical principles that play a role in the growth and coarsening of lumens. Solute pum** by the cell, hydraulic flows driven by differences of osmotic and hydrostatic pressures, balance of forces between extracellular fluids and cell-generated cytoskeletal forces, and electro-osmotic effects have been implicated in determining the dynamics and steady-state of lumens. We use the framework of linear irreversible thermodynamics to discuss the relevant force, time and length scales involved in these processes. We focus on order of magnitude estimates of physical parameters controlling lumen formation and coarsening.
△ Less
Submitted 12 July, 2021; v1 submitted 12 April, 2021;
originally announced April 2021.
-
Photometric Supernova Classification With Machine Learning
Authors:
Michelle Lochner,
Jason D. McEwen,
Hiranya V. Peiris,
Ofer Lahav,
Max K. Winter
Abstract:
Automated photometric supernova classification has become an active area of research in recent years in light of current and upcoming imaging surveys such as the Dark Energy Survey (DES) and the Large Synoptic Survey Telescope, given that spectroscopic confirmation of type for all supernovae discovered will be impossible. Here, we develop a multi-faceted classification pipeline, combining existing…
▽ More
Automated photometric supernova classification has become an active area of research in recent years in light of current and upcoming imaging surveys such as the Dark Energy Survey (DES) and the Large Synoptic Survey Telescope, given that spectroscopic confirmation of type for all supernovae discovered will be impossible. Here, we develop a multi-faceted classification pipeline, combining existing and new approaches. Our pipeline consists of two stages: extracting descriptive features from the light curves and classification using a machine learning algorithm. Our feature extraction methods vary from model-dependent techniques, namely SALT2 fits, to more independent techniques fitting parametric models to curves, to a completely model-independent wavelet approach. We cover a range of representative machine learning algorithms, including naive Bayes, k-nearest neighbors, support vector machines, artificial neural networks and boosted decision trees (BDTs). We test the pipeline on simulated multi-band DES light curves from the Supernova Photometric Classification Challenge. Using the commonly used area under the curve (AUC) of the Receiver Operating Characteristic as a metric, we find that the SALT2 fits and the wavelet approach, with the BDTs algorithm, each achieves an AUC of 0.98, where 1 represents perfect classification. We find that a representative training set is essential for good classification, whatever the feature set or algorithm, with implications for spectroscopic follow-up. Importantly, we find that by using either the SALT2 or the wavelet feature sets with a BDT algorithm, accurate classification is possible purely from light curve data, without the need for any redshift information.
△ Less
Submitted 7 September, 2016; v1 submitted 2 March, 2016;
originally announced March 2016.
