Physics-informed attention-based neural network for solving non-linear partial differential equations
Authors:
Ruben Rodriguez-Torrado,
Pablo Ruiz,
Luis Cueto-Felgueroso,
Michael Cerny Green,
Tyler Friesen,
Sebastien Matringe,
Julian Togelius
Abstract:
Physics-Informed Neural Networks (PINNs) have enabled significant improvements in modelling physical processes described by partial differential equations (PDEs). PINNs are based on simple architectures, and learn the behavior of complex physical systems by optimizing the network parameters to minimize the residual of the underlying PDE. Current network architectures share some of the limitations…
▽ More
Physics-Informed Neural Networks (PINNs) have enabled significant improvements in modelling physical processes described by partial differential equations (PDEs). PINNs are based on simple architectures, and learn the behavior of complex physical systems by optimizing the network parameters to minimize the residual of the underlying PDE. Current network architectures share some of the limitations of classical numerical discretization schemes when applied to non-linear differential equations in continuum mechanics. A paradigmatic example is the solution of hyperbolic conservation laws that develop highly localized nonlinear shock waves. Learning solutions of PDEs with dominant hyperbolic character is a challenge for current PINN approaches, which rely, like most grid-based numerical schemes, on adding artificial dissipation. Here, we address the fundamental question of which network architectures are best suited to learn the complex behavior of non-linear PDEs. We focus on network architecture rather than on residual regularization. Our new methodology, called Physics-Informed Attention-based Neural Networks, (PIANNs), is a combination of recurrent neural networks and attention mechanisms. The attention mechanism adapts the behavior of the deep neural network to the non-linear features of the solution, and break the current limitations of PINNs. We find that PIANNs effectively capture the shock front in a hyperbolic model problem, and are capable of providing high-quality solutions inside and beyond the training set.
△ Less
Submitted 17 May, 2021;
originally announced May 2021.
Self-organization of network dynamics into local quantized states
Authors:
Christos Nicolaides,
Ruben Juanes,
Luis Cueto-Felgueroso
Abstract:
Self-organization and pattern formation in network-organized systems emerges from the collective activation and interaction of many interconnected units. A striking feature of these non-equilibrium structures is that they are often localized and robust: only a small subset of the nodes, or cell assembly, is activated. Understanding the role of cell assemblies as basic functional units in neural ne…
▽ More
Self-organization and pattern formation in network-organized systems emerges from the collective activation and interaction of many interconnected units. A striking feature of these non-equilibrium structures is that they are often localized and robust: only a small subset of the nodes, or cell assembly, is activated. Understanding the role of cell assemblies as basic functional units in neural networks and socio-technical systems emerges as a fundamental challenge in network theory. A key open question is how these elementary building blocks emerge, and how they operate, linking structure and function in complex networks. Here we show that a network analogue of the Swift-Hohenberg continuum model---a minimal-ingredients model of nodal activation and interaction within a complex network---is able to produce a complex suite of localized patterns. Hence, the spontaneous formation of robust operational cell assemblies in complex networks can be explained as the result of self-organization, even in the absence of synaptic reinforcements. Our results show that these self-organized, local structures can provide robust functional units to understand natural and socio-technical network-organized processes.
△ Less
Submitted 19 February, 2016; v1 submitted 17 September, 2015;
originally announced September 2015.
