-
Random-Energy Secret Sharing via Extreme Synergy
Authors:
Vudtiwat Ngampruetikorn,
David J. Schwab
Abstract:
The random-energy model (REM), a solvable spin-glass model, has impacted an incredibly diverse set of problems, from protein folding to combinatorial optimization to many-body localization. Here, we explore a new connection to secret sharing. We formulate a secret-sharing scheme, based on the REM, and analyze its information-theoretic properties. Our analyses reveal that the correlations between s…
▽ More
The random-energy model (REM), a solvable spin-glass model, has impacted an incredibly diverse set of problems, from protein folding to combinatorial optimization to many-body localization. Here, we explore a new connection to secret sharing. We formulate a secret-sharing scheme, based on the REM, and analyze its information-theoretic properties. Our analyses reveal that the correlations between subsystems of the REM are highly synergistic and form the basis for secure secret-sharing schemes. We derive the ranges of temperatures and secret lengths over which the REM satisfies the requirement of secure secret sharing. We show further that a special point in the phase diagram exists at which the REM-based scheme is optimal in its information encoding. Our analytical results for the thermodynamic limit are in good qualitative agreement with numerical simulations of finite systems, for which the strict security requirement is replaced by a tradeoff between secrecy and recoverability. Our work offers a further example of information theory as a unifying concept, connecting problems in statistical physics to those in computation.
△ Less
Submitted 25 September, 2023;
originally announced September 2023.
-
Extrinsic vs Intrinsic Criticality in Systems with Many Components
Authors:
Vudtiwat Ngampruetikorn,
Ilya Nemenman,
David J. Schwab
Abstract:
Biological systems with many components often exhibit seemingly critical behaviors, characterized by atypically large correlated fluctuations. Yet the underlying causes remain unclear. Here we define and examine two types of criticality. Intrinsic criticality arises from interactions within the system which are fine-tuned to a critical point. Extrinsic criticality, in contrast, emerges without fin…
▽ More
Biological systems with many components often exhibit seemingly critical behaviors, characterized by atypically large correlated fluctuations. Yet the underlying causes remain unclear. Here we define and examine two types of criticality. Intrinsic criticality arises from interactions within the system which are fine-tuned to a critical point. Extrinsic criticality, in contrast, emerges without fine tuning when observable degrees of freedom are coupled to unobserved fluctuating variables. We unify both types of criticality using the language of learning and information theory. We show that critical correlations, intrinsic or extrinsic, lead to diverging mutual information between two halves of the system, and are a feature of learning problems, in which the unobserved fluctuations are inferred from the observable degrees of freedom. We argue that extrinsic criticality is equivalent to standard inference, whereas intrinsic criticality describes fractional learning, in which the amount to be learned depends on the system size. We show further that both types of criticality are on the same continuum, connected by a smooth crossover. In addition, we investigate the observability of Zipf's law, a power-law rank-frequency distribution often used as an empirical signature of criticality. We find that Zipf's law is a robust feature of extrinsic criticality but can be nontrivial to observe for some intrinsically critical systems, including critical mean-field models. We further demonstrate that models with global dynamics, such as oscillatory models, can produce observable Zipf's law without relying on either external fluctuations or fine tuning. Our findings suggest that while possible in theory, fine tuning is not the only, nor the most likely, explanation for the apparent ubiquity of criticality in biological systems with many components.
△ Less
Submitted 25 September, 2023;
originally announced September 2023.
-
Generalized Information Bottleneck for Gaussian Variables
Authors:
Vudtiwat Ngampruetikorn,
David J. Schwab
Abstract:
The information bottleneck (IB) method offers an attractive framework for understanding representation learning, however its applications are often limited by its computational intractability. Analytical characterization of the IB method is not only of practical interest, but it can also lead to new insights into learning phenomena. Here we consider a generalized IB problem, in which the mutual in…
▽ More
The information bottleneck (IB) method offers an attractive framework for understanding representation learning, however its applications are often limited by its computational intractability. Analytical characterization of the IB method is not only of practical interest, but it can also lead to new insights into learning phenomena. Here we consider a generalized IB problem, in which the mutual information in the original IB method is replaced by correlation measures based on Renyi and Jeffreys divergences. We derive an exact analytical IB solution for the case of Gaussian correlated variables. Our analysis reveals a series of structural transitions, similar to those previously observed in the original IB case. We find further that although solving the original, Renyi and Jeffreys IB problems yields different representations in general, the structural transitions occur at the same critical tradeoff parameters, and the Renyi and Jeffreys IB solutions perform well under the original IB objective. Our results suggest that formulating the IB method with alternative correlation measures could offer a strategy for obtaining an approximate solution to the original IB problem.
△ Less
Submitted 30 March, 2023;
originally announced March 2023.
-
Information bottleneck theory of high-dimensional regression: relevancy, efficiency and optimality
Authors:
Vudtiwat Ngampruetikorn,
David J. Schwab
Abstract:
Avoiding overfitting is a central challenge in machine learning, yet many large neural networks readily achieve zero training loss. This puzzling contradiction necessitates new approaches to the study of overfitting. Here we quantify overfitting via residual information, defined as the bits in fitted models that encode noise in training data. Information efficient learning algorithms minimize resi…
▽ More
Avoiding overfitting is a central challenge in machine learning, yet many large neural networks readily achieve zero training loss. This puzzling contradiction necessitates new approaches to the study of overfitting. Here we quantify overfitting via residual information, defined as the bits in fitted models that encode noise in training data. Information efficient learning algorithms minimize residual information while maximizing the relevant bits, which are predictive of the unknown generative models. We solve this optimization to obtain the information content of optimal algorithms for a linear regression problem and compare it to that of randomized ridge regression. Our results demonstrate the fundamental trade-off between residual and relevant information and characterize the relative information efficiency of randomized regression with respect to optimal algorithms. Finally, using results from random matrix theory, we reveal the information complexity of learning a linear map in high dimensions and unveil information-theoretic analogs of double and multiple descent phenomena.
△ Less
Submitted 11 October, 2022; v1 submitted 7 August, 2022;
originally announced August 2022.
-
Inferring couplings in networks across order-disorder phase transitions
Authors:
Vudtiwat Ngampruetikorn,
Vedant Sachdeva,
Johanna Torrence,
Jan Humplik,
David J. Schwab,
Stephanie E. Palmer
Abstract:
Statistical inference is central to many scientific endeavors, yet how it works remains unresolved. Answering this requires a quantitative understanding of the intrinsic interplay between statistical models, inference methods and data structure. To this end, we characterize the efficacy of direct coupling analysis (DCA)--a highly successful method for analyzing amino acid sequence data--in inferri…
▽ More
Statistical inference is central to many scientific endeavors, yet how it works remains unresolved. Answering this requires a quantitative understanding of the intrinsic interplay between statistical models, inference methods and data structure. To this end, we characterize the efficacy of direct coupling analysis (DCA)--a highly successful method for analyzing amino acid sequence data--in inferring pairwise interactions from samples of ferromagnetic Ising models on random graphs. Our approach allows for physically motivated exploration of qualitatively distinct data regimes separated by phase transitions. We show that inference quality depends strongly on the nature of generative models: optimal accuracy occurs at an intermediate temperature where the detrimental effects from macroscopic order and thermal noise are minimal. Importantly our results indicate that DCA does not always outperform its local-statistics-based predecessors; while DCA excels at low temperatures, it becomes inferior to simple correlation thresholding at virtually all temperatures when data are limited. Our findings offer new insights into the regime in which DCA operates so successfully and more broadly how inference interacts with data structure.
△ Less
Submitted 25 August, 2021; v1 submitted 4 June, 2021;
originally announced June 2021.
-
Perturbation Theory for the Information Bottleneck
Authors:
Vudtiwat Ngampruetikorn,
David J. Schwab
Abstract:
Extracting relevant information from data is crucial for all forms of learning. The information bottleneck (IB) method formalizes this, offering a mathematically precise and conceptually appealing framework for understanding learning phenomena. However the nonlinearity of the IB problem makes it computationally expensive and analytically intractable in general. Here we derive a perturbation theory…
▽ More
Extracting relevant information from data is crucial for all forms of learning. The information bottleneck (IB) method formalizes this, offering a mathematically precise and conceptually appealing framework for understanding learning phenomena. However the nonlinearity of the IB problem makes it computationally expensive and analytically intractable in general. Here we derive a perturbation theory for the IB method and report the first complete characterization of the learning onset, the limit of maximum relevant information per bit extracted from data. We test our results on synthetic probability distributions, finding good agreement with the exact numerical solution near the onset of learning. We explore the difference and subtleties in our derivation and previous attempts at deriving a perturbation theory for the learning onset and attribute the discrepancy to a flawed assumption. Our work also provides a fresh perspective on the intimate relationship between the IB method and the strong data processing inequality.
△ Less
Submitted 25 October, 2021; v1 submitted 28 May, 2021;
originally announced May 2021.
-
Impurity-Induced Anomalous Thermal Hall Effect in Chiral Superconductors
Authors:
Vudtiwat Ngampruetikorn,
J. A. Sauls
Abstract:
Chiral superconductors exhibit novel transport properties that depend on the topology of the order parameter, topology of the Fermi surface, the spectrum of bulk and edge Fermionic excitations, and the structure of the impurity potential. In the case of electronic heat transport, impurities induce an anomalous (zero-field) thermal Hall conductivity that is easily orders of magnitude larger than th…
▽ More
Chiral superconductors exhibit novel transport properties that depend on the topology of the order parameter, topology of the Fermi surface, the spectrum of bulk and edge Fermionic excitations, and the structure of the impurity potential. In the case of electronic heat transport, impurities induce an anomalous (zero-field) thermal Hall conductivity that is easily orders of magnitude larger than the quantized edge contribution. The effect originates from branch-conversion scattering of Bogoliubov quasiparticles by the chiral order parameter, induced by potential scattering. The former transfers angular momentum between the condensate and the excitations that transport heat. The anomalous thermal Hall conductivity is shown to depend to the structure of the electron-impurity potential, as well as the winding number, $ν$, of the chiral order parameter, $Δ(p)=|Δ(p)|\,e^{iνφ_{p}}$. The results provide quantitative formulae for interpreting heat transport experiments seeking to identify broken T and P symmetries, as well as the topology of the order parameter for chiral superconductors.
△ Less
Submitted 13 April, 2020;
originally announced April 2020.
-
Anomalous Hall Effects in Chiral Superconductors
Authors:
Vudtiwat Ngampruetikorn,
J. A. Sauls
Abstract:
We report theoretical results for the electronic contribution to thermal and electrical transport for chiral superconductors belonging to even or odd-parity E$_1$ and E$_2$ representations of the tetragonal and hexagonal point groups. Chiral superconductors exhibit novel properties that depend on the topology of the order parameter and Fermi surface, and -- as we highlight -- the structure of the…
▽ More
We report theoretical results for the electronic contribution to thermal and electrical transport for chiral superconductors belonging to even or odd-parity E$_1$ and E$_2$ representations of the tetragonal and hexagonal point groups. Chiral superconductors exhibit novel properties that depend on the topology of the order parameter and Fermi surface, and -- as we highlight -- the structure of the impurity potential. An anomalous thermal Hall effect is predicted and shown to be sensitive to the winding number, $ν$, of the chiral order parameter via Andreev scattering that transfers angular momentum from the chiral condensate to excitations that scatter off the random potential. For heat transport in a chiral superconductor with isotropic impurity scattering, i.e., point-like impurities, a transverse heat current is obtained for $ν=\pm 1$, but vanishes for $|ν|>1$. This is not a universal result. For finite-size impurities with radii of order or greater than the Fermi wavelength, $R\ge\hbar/p_f$, the thermal Hall conductivity is finite for chiral order with $|ν|\ge2$, and determined by a specific Fermi-surface average of the differential cross-section for electron-impurity scattering. Our results also provide quantitative formulae for analyzing and interpreting thermal transport measurements for superconductors predicted to exhibit broken time-reversal and mirror symmetries.
△ Less
Submitted 8 February, 2024; v1 submitted 14 November, 2019;
originally announced November 2019.
-
Energy consumption and cooperation for optimal sensing
Authors:
Vudtiwat Ngampruetikorn,
David J. Schwab,
Greg J. Stephens
Abstract:
The reliable detection of environmental molecules in the presence of noise is an important cellular function, yet the underlying computational mechanisms are not well understood. We introduce a model of two interacting sensors which allows for the principled exploration of signal statistics, cooperation strategies and the role of energy consumption in optimal sensing, quantified through the mutual…
▽ More
The reliable detection of environmental molecules in the presence of noise is an important cellular function, yet the underlying computational mechanisms are not well understood. We introduce a model of two interacting sensors which allows for the principled exploration of signal statistics, cooperation strategies and the role of energy consumption in optimal sensing, quantified through the mutual information between the signal and the sensors. Here we report that in general the optimal sensing strategy depends both on the noise level and the statistics of the signals. For joint, correlated signals, energy consuming (nonequilibrium), asymmetric couplings result in maximum information gain in the low-noise, high-signal-correlation limit. Surprisingly we also find that energy consumption is not always required for optimal sensing. We generalise our model to incorporate time integration of the sensor state by a population of readout molecules, and demonstrate that sensor interaction and energy consumption remain important for optimal sensing.
△ Less
Submitted 6 February, 2020; v1 submitted 11 September, 2018;
originally announced September 2018.
-
The Effect of Inhomogeneous Surface Disorder on the Superheating Field of Superconducting RF Cavities
Authors:
Vudtiwat Ngampruetikorn,
J. A. Sauls
Abstract:
Recent advances in surface treatments of Niobium superconducting radio frequency (SRF) cavities have led to substantially increased Q-factors and maximum surface field. This poses theoretical challenges to identify the mechanisms responsible for such performance enhancements. We report theoretical results for the effects of inhomogeneous surface disorder on the superheating field --- the surface m…
▽ More
Recent advances in surface treatments of Niobium superconducting radio frequency (SRF) cavities have led to substantially increased Q-factors and maximum surface field. This poses theoretical challenges to identify the mechanisms responsible for such performance enhancements. We report theoretical results for the effects of inhomogeneous surface disorder on the superheating field --- the surface magnetic field above which the Meissner state is globally unstable. We find that inhomogeneous disorder, such as that introduced by infusion of Nitrogen into the surface layers of Niobium SRF cavities, can increase the superheating field above the maximum for superconductors in the clean limit or with homogeneously distributed disorder. Homogeneous disorder increases the penetration of screening current, but also suppresses the maximum supercurrent. Inhomogeneous disorder in the form of an impurity diffusion layer biases this trade-off by increasing the penetration of the screening currents into cleaner regions with larger critical currents, thus limiting the suppression of the screening current to a thin dirty region close to the surface. Our results suggest that the impurity diffusion layers play a role in enhancing the maximum accelerating gradient of Nitrogen treated Niobium SRF cavities.
△ Less
Submitted 11 September, 2018;
originally announced September 2018.
-
Exploiting ecology in drug pulse sequences in favour of population reduction
Authors:
Marianne Bauer,
Isabella R. Graf,
Vudtiwat Ngampruetikorn,
Greg J. Stephens,
Erwin Frey
Abstract:
A deterministic population dynamics model involving birth and death for a two-species system, comprising a wild-type and more resistant species competing via logistic growth, is subjected to two distinct stress environments designed to mimic those that would typically be induced by temporal variation in the concentration of a drug (antibiotic or chemotherapeutic) as it permeates through the popula…
▽ More
A deterministic population dynamics model involving birth and death for a two-species system, comprising a wild-type and more resistant species competing via logistic growth, is subjected to two distinct stress environments designed to mimic those that would typically be induced by temporal variation in the concentration of a drug (antibiotic or chemotherapeutic) as it permeates through the population and is progressively degraded. Different treatment regimes, involving single or periodical doses, are evaluated in terms of the minimal population size (a measure of the extinction probability), and the population composition (a measure of the selection pressure for resistance or tolerance during the treatment). We show that there exist timescales over which the low-stress regime is as effective as the high-stress regime, due to the competition between the two species. For multiple periodic treatments, competition can ensure that the minimal population size is attained during the first pulse when the high-stress regime is short, which implies that a single short pulse can be more effective than a more protracted regime. Our results suggest that when the duration of the high-stress environment is restricted, a treatment with one or multiple shorter pulses can produce better outcomes than a single long treatment. If ecological competition is to be exploited for treatments, it is crucial to determine these timescales, and estimate for the minimal population threshold that suffices for extinction. These parameters can be quantified by experiment.
△ Less
Submitted 7 February, 2018;
originally announced February 2018.
-
Bias, Belief and Consensus: Collective opinion formation on fluctuating networks
Authors:
V. Ngampruetikorn,
Greg J Stephens
Abstract:
With the advent of online networks, societies are substantially more connected with individual members able to easily modify and maintain their own social links. Here, we show that active network maintenance exposes agents to confirmation bias, the tendency to confirm one's beliefs, and we explore how this affects collective opinion formation. We introduce a model of binary opinion dynamics on a c…
▽ More
With the advent of online networks, societies are substantially more connected with individual members able to easily modify and maintain their own social links. Here, we show that active network maintenance exposes agents to confirmation bias, the tendency to confirm one's beliefs, and we explore how this affects collective opinion formation. We introduce a model of binary opinion dynamics on a complex network with fast, stochastic rewiring and show that confirmation bias induces a segregation of individuals with different opinions. We use the dynamics of global opinion to generally categorize opinion update rules and find that confirmation bias always stabilizes the consensus state. Finally, we show that the time to reach consensus has a non-monotonic dependence on the magnitude of the bias, suggesting a novel avenue for large-scale opinion engineering.
△ Less
Submitted 30 December, 2015;
originally announced December 2015.
-
High-temperature limit of the resonant Fermi gas
Authors:
V. Ngampruetikorn,
Meera M. Parish,
Jesper Levinsen
Abstract:
We use the virial expansion to investigate the behavior of the two-component, attractive Fermi gas in the high-temperature limit, where the system smoothly evolves from weakly attractive fermions to weakly repulsive bosonic dimers as the short-range attraction is increased. We present a new formalism for computing the virial coefficients that employs a diagrammatic approach to the grand potential…
▽ More
We use the virial expansion to investigate the behavior of the two-component, attractive Fermi gas in the high-temperature limit, where the system smoothly evolves from weakly attractive fermions to weakly repulsive bosonic dimers as the short-range attraction is increased. We present a new formalism for computing the virial coefficients that employs a diagrammatic approach to the grand potential and allows one to easily include an effective range $R^*$ in the interaction. In the limit where the thermal wavelength $λ\ll R^*$, the calculation of the virial coefficients is perturbative even at unitarity and the system resembles a weakly interacting Bose-Fermi mixture for all scattering lengths $a$. By interpolating from the perturbative limits $λ/|a| \gg 1$ and $R^*/λ\gg 1$, we estimate the value of the fourth virial coefficient at unitarity for $R^*=0$ and we find that it is close to the value obtained in recent experiments. We also derive the equations of state for the pressure, density and entropy, as well as the spectral function at high temperatures.
△ Less
Submitted 9 January, 2015; v1 submitted 28 October, 2014;
originally announced October 2014.
-
Pair Correlations in the Two-Dimensional Fermi Gas
Authors:
V. Ngampruetikorn,
J. Levinsen,
Meera M. Parish
Abstract:
We consider the two-dimensional Fermi gas at finite temperature with attractive short-range interactions. Using the virial expansion, which provides a controlled approach at high temperatures, we determine the spectral function and contact for the normal state. Our calculated spectra are in qualitative agreement with recent photoemission measurements [M. Feld et al., Nature 480, 75 (2011)], thus s…
▽ More
We consider the two-dimensional Fermi gas at finite temperature with attractive short-range interactions. Using the virial expansion, which provides a controlled approach at high temperatures, we determine the spectral function and contact for the normal state. Our calculated spectra are in qualitative agreement with recent photoemission measurements [M. Feld et al., Nature 480, 75 (2011)], thus suggesting that the observed pairing gap is a feature of the high-temperature gas rather than being evidence of a pseudogap regime just above the superfluid transition temperature. We further argue that the strong pair correlations result from the fact that the crossover to bosonic dimers occurs at weaker interactions than previously assumed.
△ Less
Submitted 6 January, 2014; v1 submitted 1 September, 2013;
originally announced September 2013.
-
Three-body problem in a two-dimensional Fermi gas
Authors:
V. Ngampruetikorn,
Meera M. Parish,
J. Levinsen
Abstract:
We investigate the three-body properties of two identical "up" fermions and one distinguishable "down" atom interacting in a strongly confined two-dimensional geometry. We compute exactly the atom-dimer scattering properties and the three-body recombination rate as a function of collision energy and mass ratio m_up/m_down. We find that the recombination rate for fermions is strongly energy depende…
▽ More
We investigate the three-body properties of two identical "up" fermions and one distinguishable "down" atom interacting in a strongly confined two-dimensional geometry. We compute exactly the atom-dimer scattering properties and the three-body recombination rate as a function of collision energy and mass ratio m_up/m_down. We find that the recombination rate for fermions is strongly energy dependent, with significant contributions from higher partial waves at low energies. For m_up < m_down, the s-wave atom-dimer scattering below threshold is completely described by the scattering length. Furthermore, we examine the "up-up-down" bound states (trimers) appearing at large m_up/m_down and find that the energy spectrum for the deepest bound trimers resembles that of a hydrogen atom confined to two dimensions.
△ Less
Submitted 6 February, 2013; v1 submitted 28 November, 2012;
originally announced November 2012.
-
Repulsive polarons in two-dimensional Fermi gases
Authors:
V. Ngampruetikorn,
J. Levinsen,
Meera M. Parish
Abstract:
We consider a single spin-down impurity atom interacting via an attractive, short-range potential with a spin-up Fermi sea in two dimensions (2D). Similarly to 3D, we show how the impurity can form a metastable state (the "repulsive polaron") with energy greater than that of the non-interacting impurity. Moreover, we find that the repulsive polaron can acquire a finite momentum for sufficiently we…
▽ More
We consider a single spin-down impurity atom interacting via an attractive, short-range potential with a spin-up Fermi sea in two dimensions (2D). Similarly to 3D, we show how the impurity can form a metastable state (the "repulsive polaron") with energy greater than that of the non-interacting impurity. Moreover, we find that the repulsive polaron can acquire a finite momentum for sufficiently weak attractive interactions. Even though the energy of the repulsive polaron can become sizeable, we argue that saturated ferromagnetism is unfavorable in 2D because of the polaron's finite lifetime and small quasiparticle weight.
△ Less
Submitted 10 April, 2012; v1 submitted 28 October, 2011;
originally announced October 2011.
