-
Arbitrage equilibria in active matter systems
Authors:
Venkat Venkatasubramanian,
Abhishek Sivaram,
N. Sanjeevrajan,
Arun Sankar
Abstract:
The motility-induced phase separation (MIPS) phenomenon in active matter has been of great interest for the past decade or so. A central conceptual puzzle is that this behavior, which is generally characterized as a nonequilibrium phenomenon, can yet be explained using simple equilibrium models of thermodynamics. Here, we address this problem using a new theory, statistical teleodynamics, which is…
▽ More
The motility-induced phase separation (MIPS) phenomenon in active matter has been of great interest for the past decade or so. A central conceptual puzzle is that this behavior, which is generally characterized as a nonequilibrium phenomenon, can yet be explained using simple equilibrium models of thermodynamics. Here, we address this problem using a new theory, statistical teleodynamics, which is a conceptual synthesis of game theory and statistical mechanics. In this framework, active agents compete in their pursuit of maximum effective utility, and this self-organizing dynamics results in an arbitrage equilibrium in which all agents have the same effective utility. We show that MIPS is an example of arbitrage equilibrium and that it is mathematically equivalent to other phase-separation phenomena in entirely different domains, such as sociology and economics. As examples, we present the behavior of Janus particles in a potential trap and the effect of chemotaxis on MIPS.
△ Less
Submitted 18 May, 2024;
originally announced May 2024.
-
Arbitrage equilibrium and the emergence of universal microstructure in deep neural networks
Authors:
Venkat Venkatasubramanian,
N Sanjeevrajan,
Manasi Khandekar,
Abhishek Sivaram,
Collin Szczepanski
Abstract:
Despite the stunning progress recently in large-scale deep neural network applications, our understanding of their microstructure, 'energy' functions, and optimal design remains incomplete. Here, we present a new game-theoretic framework, called statistical teleodynamics, that reveals important insights into these key properties. The optimally robust design of such networks inherently involves com…
▽ More
Despite the stunning progress recently in large-scale deep neural network applications, our understanding of their microstructure, 'energy' functions, and optimal design remains incomplete. Here, we present a new game-theoretic framework, called statistical teleodynamics, that reveals important insights into these key properties. The optimally robust design of such networks inherently involves computational benefit-cost trade-offs that are not adequately captured by physics-inspired models. These trade-offs occur as neurons and connections compete to increase their effective utilities under resource constraints during training. In a fully trained network, this results in a state of arbitrage equilibrium, where all neurons in a given layer have the same effective utility, and all connections to a given layer have the same effective utility. The equilibrium is characterized by the emergence of two lognormal distributions of connection weights and neuronal output as the universal microstructure of large deep neural networks. We call such a network the Jaynes Machine. Our theoretical predictions are shown to be supported by empirical data from seven large-scale deep neural networks. We also show that the Hopfield network and the Boltzmann Machine are the same special case of the Jaynes Machine.
△ Less
Submitted 5 June, 2024; v1 submitted 29 March, 2024;
originally announced May 2024.
-
Jaynes Machine: The universal microstructure of deep neural networks
Authors:
Venkat Venkatasubramanian,
N. Sanjeevrajan,
Manasi Khandekar
Abstract:
We present a novel theory of the microstructure of deep neural networks. Using a theoretical framework called statistical teleodynamics, which is a conceptual synthesis of statistical thermodynamics and potential game theory, we predict that all highly connected layers of deep neural networks have a universal microstructure of connection strengths that is distributed lognormally ($LN(μ, σ)$). Furt…
▽ More
We present a novel theory of the microstructure of deep neural networks. Using a theoretical framework called statistical teleodynamics, which is a conceptual synthesis of statistical thermodynamics and potential game theory, we predict that all highly connected layers of deep neural networks have a universal microstructure of connection strengths that is distributed lognormally ($LN(μ, σ)$). Furthermore, under ideal conditions, the theory predicts that $μ$ and $σ$ are the same for all layers in all networks. This is shown to be the result of an arbitrage equilibrium where all connections compete and contribute the same effective utility towards the minimization of the overall loss function. These surprising predictions are shown to be supported by empirical data from six large-scale deep neural networks in real life. We also discuss how these results can be exploited to reduce the amount of data, time, and computational resources needed to train large deep neural networks.
△ Less
Submitted 10 October, 2023;
originally announced October 2023.
