-
Charting the Topography of the Neural Network Landscape with Thermal-Like Noise
Authors:
Theo Jules,
Gal Brener,
Tal Kachman,
Noam Levi,
Yohai Bar-Sinai
Abstract:
The training of neural networks is a complex, high-dimensional, non-convex and noisy optimization problem whose theoretical understanding is interesting both from an applicative perspective and for fundamental reasons. A core challenge is to understand the geometry and topography of the landscape that guides the optimization. In this work, we employ standard Statistical Mechanics methods, namely,…
▽ More
The training of neural networks is a complex, high-dimensional, non-convex and noisy optimization problem whose theoretical understanding is interesting both from an applicative perspective and for fundamental reasons. A core challenge is to understand the geometry and topography of the landscape that guides the optimization. In this work, we employ standard Statistical Mechanics methods, namely, phase-space exploration using Langevin dynamics, to study this landscape for an over-parameterized fully connected network performing a classification task on random data. Analyzing the fluctuation statistics, in analogy to thermal dynamics at a constant temperature, we infer a clear geometric description of the low-loss region. We find that it is a low-dimensional manifold whose dimension can be readily obtained from the fluctuations. Furthermore, this dimension is controlled by the number of data points that reside near the classification decision boundary. Importantly, we find that a quadratic approximation of the loss near the minimum is fundamentally inadequate due to the exponential nature of the decision boundary and the flatness of the low-loss region. This causes the dynamics to sample regions with higher curvature at higher temperatures, while producing quadratic-like statistics at any given temperature. We explain this behavior by a simplified loss model which is analytically tractable and reproduces the observed fluctuation statistics.
△ Less
Submitted 18 April, 2023; v1 submitted 3 April, 2023;
originally announced April 2023.
-
Dynamically writing coupled memories using a reinforcement learning agent, meeting physical bounds
Authors:
Théo Jules,
Laura Michel,
Adèle Douin,
Frédéric Lechenault
Abstract:
Traditional memory writing operations proceed one bit at a time, where e.g. an individual magnetic domain is force-flipped by a localized external field. One way to increase material storage capacity would be to write several bits at a time in the bulk of the material. However, the manipulation of bits is commonly done through quasi-static operations. While simple to model, this method is known to…
▽ More
Traditional memory writing operations proceed one bit at a time, where e.g. an individual magnetic domain is force-flipped by a localized external field. One way to increase material storage capacity would be to write several bits at a time in the bulk of the material. However, the manipulation of bits is commonly done through quasi-static operations. While simple to model, this method is known to reduce memory capacity. In this paper, we demonstrate how a reinforcement learning agent can exploit the dynamical response of a simple multi-bit mechanical system to restore its memory to full capacity. To do so, we introduce a model framework consisting of a chain of bi-stable springs, which is manipulated on one end by the external action of the agent. We show that the agent manages to learn how to reach all available states for three springs, even though some states are not reachable through adiabatic manipulation, and that both the training speed and convergence within physical parameter space are improved using transfer learning techniques. Interestingly, the agent also points to an optimal design of the system in terms of writing time. In fact, it appears to learn how to take advantage of the underlying physics: the control time exhibits a non-monotonic dependence on the internal dissipation, reaching a minimum at a cross-over shown to verify a mechanically motivated scaling relation.
△ Less
Submitted 6 May, 2022;
originally announced May 2022.
-
The delicate memory structure of origami switches
Authors:
Théo Jules,
Austin Reid,
Karen E. Daniels,
Muhittin Mungan,
Frédéric Lechenault
Abstract:
While memory effects emerge from systems of wildly varying length- and time-scales, the reduction of a complex system with many interacting elements into one simple enough to be understood without also losing the complex behavior continues to be a challenge. Here, we investigate how bistable cylindrical origamis provide such a reduction via tunably-interactive memory behaviors. We base our investi…
▽ More
While memory effects emerge from systems of wildly varying length- and time-scales, the reduction of a complex system with many interacting elements into one simple enough to be understood without also losing the complex behavior continues to be a challenge. Here, we investigate how bistable cylindrical origamis provide such a reduction via tunably-interactive memory behaviors. We base our investigation on folded sheets of Kresling patterns that function as two-state memory units. By linking several units, each with a selected activation energy, we construct a one-dimensional material that exhibits return-point memory. After a comprehensive experimental analysis of the relation between the geometry of the pattern and the mechanical response for a single bit, we study the memory of a bellows composed of 4 bits arranged in series. Since these bits are decoupled, the system reduces to the Preisach model and we can drive the bellows to any of its 16 allowable states by following a prescribed sequence of compression and extension. We show how to reasonably discriminate between states by measuring the system's total height and stiffness near equilibrium. Furthermore, we establish the existence of geometrically-disallowed defective stable configurations which expand the configuration space to 64 states with a more complex transition pattern. Using empirical considerations of the mechanics, we analyze the hierarchical structure of the corresponding diagram, which includes Garden of Eden states and subgraphs. We highlight two irreversible transformations, shifting and erasure of the defect, leading to memory behaviors reminiscent of those observed with more complex glassy systems.
△ Less
Submitted 4 November, 2021; v1 submitted 15 June, 2021;
originally announced June 2021.
-
Curving Origami with Mechanical Frustration
Authors:
Théo Jules,
Frédéric Lechenault,
Mokhtar Adda-Bedia
Abstract:
We study the three-dimensional equilibrium shape of a shell formed by a deployed accordion-like origami, made from an elastic sheet decorated by a series of parallel creases crossed by a central longitudinal crease. Surprisingly, while the imprinted crease network does not exhibit a geodesic curvature, the emergent structure is characterized by an effective curvature produced by the deformed centr…
▽ More
We study the three-dimensional equilibrium shape of a shell formed by a deployed accordion-like origami, made from an elastic sheet decorated by a series of parallel creases crossed by a central longitudinal crease. Surprisingly, while the imprinted crease network does not exhibit a geodesic curvature, the emergent structure is characterized by an effective curvature produced by the deformed central fold. Moreover, both finite element analysis and manually made mylar origamis show a robust empirical relation between the imprinted crease network's dimensions and the apparent curvature. A detailed examination of this geometrical relation shows the existence of three typical elastic deformations, which in turn induce three distinct types of morphogenesis. We characterize the corresponding kinematics of crease network deformations and determine their phase diagram. Taking advantage of the frustration caused by the competition between crease stiffness and kinematics of crease network deformations, we provide a novel tool for designing curved origami structures constrained by strong geometrical properties.
△ Less
Submitted 7 February, 2021;
originally announced February 2021.
-
Plasticity and Aging of Folded Elastic Sheets
Authors:
Théo Jules,
Frédéric Lechenault,
Mokhtar Adda-Bedia
Abstract:
We investigate the dissipative mechanisms exhibited by creased material sheets when subjected to mechanical loading, which comes in the form of plasticity and relaxation phenomena within the creases. After demonstrating that plasticity mostly affects the rest angle of the creases, we devise a map** between this quantity and the macroscopic state of the system that allows us to track its referenc…
▽ More
We investigate the dissipative mechanisms exhibited by creased material sheets when subjected to mechanical loading, which comes in the form of plasticity and relaxation phenomena within the creases. After demonstrating that plasticity mostly affects the rest angle of the creases, we devise a map** between this quantity and the macroscopic state of the system that allows us to track its reference configuration along an arbitrary loading path, resulting in a powerful monitoring and design tool for crease-based metamaterials. Furthermore, we show that complex relaxation phenomena, in particular memory effects, can give rise to a non-monotonic response at the crease level, possibly relating to the similar behavior reported for crumpled sheets. We describe our observations through a classical double-logarithmic time evolution and obtain a constitutive behavior compatible with that of the underlying material. Thus the lever effect provided by the crease allows magnified access to the material's rheology.
△ Less
Submitted 24 April, 2020;
originally announced April 2020.
-
Rotation of a submerged finite cylinder moving down a soft incline
Authors:
Baudouin Saintyves,
Bhargav Rallabandi,
Theo Jules,
Jesse Ault,
Thomas Salez,
Clarissa Schönecker,
Howard Stone,
L. Mahadevan
Abstract:
A submerged finite cylinder moving under its own weight along a soft incline lifts off and slides at a steady velocity while also spinning. Here, we experimentally quantify the steady spinning of the cylinder and show theoretically that it is due to a combination of an elastohydrodynamic torque generated by flow in the variable gap, and the viscous friction on the edges of the finite-length cylind…
▽ More
A submerged finite cylinder moving under its own weight along a soft incline lifts off and slides at a steady velocity while also spinning. Here, we experimentally quantify the steady spinning of the cylinder and show theoretically that it is due to a combination of an elastohydrodynamic torque generated by flow in the variable gap, and the viscous friction on the edges of the finite-length cylinder. The relative influence of the latter depends on the aspect ratio of the cylinder as well as the deformability of the substrate, which we express in term of a single scaled compliance parameter. By varying this compliance parameter, we show that our experimental results are consistent with a transition from an edge-effect dominated regime for short cylinders to a gap-dominated elastohydrodynamic regime when the cylinder is very long.
△ Less
Submitted 6 December, 2019;
originally announced December 2019.
-
Local Mechanical Description of an Elastic Fold
Authors:
Théo Jules,
Frédéric Lechenault,
Mokhtar Adda-Bedia
Abstract:
To go beyond the simple model for the fold as two flexible surfaces or faces linked by a crease that behaves as an elastic hinge, we carefully shape and anneal a crease within a polymer sheet and study its mechanical response. First, we carry out an experimental study that consists on recording both the shape of the fold in various loading configurations and the associated force needed to deform i…
▽ More
To go beyond the simple model for the fold as two flexible surfaces or faces linked by a crease that behaves as an elastic hinge, we carefully shape and anneal a crease within a polymer sheet and study its mechanical response. First, we carry out an experimental study that consists on recording both the shape of the fold in various loading configurations and the associated force needed to deform it. Then, an elastic model of the fold is built upon a continuous description of both the faces and the crease as a thin sheet with a non flat reference configuration. The comparison between the model and experiments yields the local fold properties and explains the significant differences we observe between tensile and compression regimes. Furthermore, an asymptotic study of the fold deformation enables us to determine the local shape of the crease and identify the origin of its mechanical behaviour.
△ Less
Submitted 14 August, 2018;
originally announced August 2018.
-
Rotation of an immersed cylinder sliding near a thin elastic coating
Authors:
Bhargav Rallabandi,
Baudouin Saintyves,
Theo Jules,
Thomas Salez,
Clarissa Schönecker,
L. Mahadevan,
Howard A. Stone
Abstract:
It is known that an object translating parallel to a soft wall in a viscous fluid produces hydro- dynamic stresses that deform the wall, which, in turn, results in a lift force on the object. Recent experiments with cylinders sliding under gravity near a soft incline, which confirmed theoretical arguments for the lift force, also reported an unexplained steady-state rotation of the cylinders [Sain…
▽ More
It is known that an object translating parallel to a soft wall in a viscous fluid produces hydro- dynamic stresses that deform the wall, which, in turn, results in a lift force on the object. Recent experiments with cylinders sliding under gravity near a soft incline, which confirmed theoretical arguments for the lift force, also reported an unexplained steady-state rotation of the cylinders [Saintyves et al. PNAS 113(21), 2016]. Motivated by these observations, we show, in the lubrication limit, that an infinite cylinder that translates in a viscous fluid parallel to a soft wall at constant speed and separation distance must also rotate in order to remain free of torque. Using the Lorentz reciprocal theorem, we show analytically that for small deformations of the elastic layer, the angular velocity of the cylinder scales with the cube of the sliding speed. These predictions are confirmed numerically. We then apply the theory to the gravity-driven motion of a cylinder near a soft incline and find qualitative agreement with the experimental observations, namely that a softer elastic layer results in a greater angular speed of the cylinder.
△ Less
Submitted 17 May, 2017; v1 submitted 10 November, 2016;
originally announced November 2016.
-
Self-sustained lift and low friction via soft lubrication
Authors:
Baudouin Saintyves,
Théo Jules,
Thomas Salez,
L. Mahadevan
Abstract:
Relative motion between soft wet solids arises in a number of applications in natural and artificial settings, and invariably couples elastic deformation and fluid flow. We explore this in a minimal setting by considering a fluid-immersed negatively-buoyant cylinder moving along a soft inclined wall. Our experiments show that there is an emergent robust steady-state sliding regime of the cylinder…
▽ More
Relative motion between soft wet solids arises in a number of applications in natural and artificial settings, and invariably couples elastic deformation and fluid flow. We explore this in a minimal setting by considering a fluid-immersed negatively-buoyant cylinder moving along a soft inclined wall. Our experiments show that there is an emergent robust steady-state sliding regime of the cylinder with an effective friction that is significantly reduced relative to that of rigid fluid-lubricated contacts. A simple scaling approach that couples the cylinder-induced flow to substrate deformation allows us to explain the emergence of an elastohydrodynamic lift that underlies the self-sustained lubricated motion of the cylinder, consistent with recent theoretical predictions. Our results suggest an explanation for a range of effects such as reduced wear in animal joints and long-runout landslides, and can be couched as a design principle for low-friction interfaces.
△ Less
Submitted 4 August, 2016; v1 submitted 12 January, 2016;
originally announced January 2016.
