-
Expressivity of Neural Networks with Random Weights and Learned Biases
Authors:
Ezekiel Williams,
Avery Hee-Woon Ryoo,
Thomas Jiralerspong,
Alexandre Payeur,
Matthew G. Perich,
Luca Mazzucato,
Guillaume Lajoie
Abstract:
Landmark universal function approximation results for neural networks with trained weights and biases provided impetus for the ubiquitous use of neural networks as learning models in Artificial Intelligence (AI) and neuroscience. Recent work has pushed the bounds of universal approximation by showing that arbitrary functions can similarly be learned by tuning smaller subsets of parameters, for exa…
▽ More
Landmark universal function approximation results for neural networks with trained weights and biases provided impetus for the ubiquitous use of neural networks as learning models in Artificial Intelligence (AI) and neuroscience. Recent work has pushed the bounds of universal approximation by showing that arbitrary functions can similarly be learned by tuning smaller subsets of parameters, for example the output weights, within randomly initialized networks. Motivated by the fact that biases can be interpreted as biologically plausible mechanisms for adjusting unit outputs in neural networks, such as tonic inputs or activation thresholds, we investigate the expressivity of neural networks with random weights where only biases are optimized. We provide theoretical and numerical evidence demonstrating that feedforward neural networks with fixed random weights can be trained to perform multiple tasks by learning biases only. We further show that an equivalent result holds for recurrent neural networks predicting dynamical system trajectories. Our results are relevant to neuroscience, where they demonstrate the potential for behaviourally relevant changes in dynamics without modifying synaptic weights, as well as for AI, where they shed light on multi-task methods such as bias fine-tuning and unit masking.
△ Less
Submitted 2 July, 2024; v1 submitted 1 July, 2024;
originally announced July 2024.
-
Reply to "Comment on `Low-temperature phonon thermal conductivity of single-crystalline Nd$_2$CuO$_4$: Effects of sample size and surface roughness' " by Sun et al
Authors:
S. Y. Li,
J. -B. Bonnemaison,
A. Payeur,
P. Fournier,
C. H. Wang,
X. H. Chen,
Louis Taillefer
Abstract:
In their comment \cite{Sun} on our study of phonon heat transport in Nd$_2$CuO$_4$ \cite{Li}, Sun and Ando estimate that the phonon mean free path at low temperature is roughly half the width of the single crystal used in our study, from which they argue that phonon scattering cannot be dominated by sample boundaries. Here we show that their use of specific heat data on Nd$_2$CuO$_4$, which cont…
▽ More
In their comment \cite{Sun} on our study of phonon heat transport in Nd$_2$CuO$_4$ \cite{Li}, Sun and Ando estimate that the phonon mean free path at low temperature is roughly half the width of the single crystal used in our study, from which they argue that phonon scattering cannot be dominated by sample boundaries. Here we show that their use of specific heat data on Nd$_2$CuO$_4$, which contains a large magnetic contribution at low temperature that is difficult to reliably extract, leads to an underestimate of the mean free path by a factor two compared to an estimate based on the specific heat data of the non-magnetic isostructural analog Pr$_2$CuO$_4$. This removes the apparent contradiction raised by Sun and Ando.
△ Less
Submitted 13 April, 2009;
originally announced April 2009.
-
Low-temperature phonon thermal conductivity of cuprate single crystals
Authors:
S. Y. Li,
J. -B. Bonnemaison,
A. Payeur,
P. Fournier,
C. H. Wang,
X. H. Chen,
Louis Taillefer
Abstract:
The effect of sample size and surface roughness on the phonon thermal conductivity $κ_p$ of Nd$_2$CuO$_4$ single crystals was studied down to 50 mK. At 0.5 K, $κ_p$ is proportional to $\sqrt{A}$, where $A$ is the cross-sectional area of the sample. This demonstrates that $κ_p$ is dominated by boundary scattering below 0.5 K or so. However, the expected $T^3$ dependence of $κ_p$ is not observed d…
▽ More
The effect of sample size and surface roughness on the phonon thermal conductivity $κ_p$ of Nd$_2$CuO$_4$ single crystals was studied down to 50 mK. At 0.5 K, $κ_p$ is proportional to $\sqrt{A}$, where $A$ is the cross-sectional area of the sample. This demonstrates that $κ_p$ is dominated by boundary scattering below 0.5 K or so. However, the expected $T^3$ dependence of $κ_p$ is not observed down to 50 mK. Upon roughing the surfaces, the $T^3$ dependence is restored, showing that departures from $T^3$ are due to specular reflection of phonons off the mirror-like sample surfaces. We propose an empirical power law fit, to $κ_p \sim T^α$ (where $α< 3$) in cuprate single crystals. Using this method, we show that recent thermal conductivity studies of Zn do** in YBa$_2$Cu$_3$O$_y$ re-affirm the universal heat conductivity of d-wave quasiparticles at $T \to 0$.
△ Less
Submitted 19 September, 2007;
originally announced September 2007.
