-
Constraints on the design of neuromorphic circuits set by the properties of neural population codes
Authors:
Stefano Panzeri,
Ella Janotte,
Alejandro Pequeño-Zurro,
Jacopo Bonato,
Chiara Bartolozzi
Abstract:
In the brain, information is encoded, transmitted and used to inform behaviour at the level of timing of action potentials distributed over population of neurons. To implement neural-like systems in silico, to emulate neural function, and to interface successfully with the brain, neuromorphic circuits need to encode information in a way compatible to that used by populations of neuron in the brain…
▽ More
In the brain, information is encoded, transmitted and used to inform behaviour at the level of timing of action potentials distributed over population of neurons. To implement neural-like systems in silico, to emulate neural function, and to interface successfully with the brain, neuromorphic circuits need to encode information in a way compatible to that used by populations of neuron in the brain. To facilitate the cross-talk between neuromorphic engineering and neuroscience, in this Review we first critically examine and summarize emerging recent findings about how population of neurons encode and transmit information. We examine the effects on encoding and readout of information for different features of neural population activity, namely the sparseness of neural representations, the heterogeneity of neural properties, the correlations among neurons, and the time scales (from short to long) at which neurons encode information and maintain it consistently over time. Finally, we critically elaborate on how these facts constrain the design of information coding in neuromorphic circuits. We focus primarily on the implications for designing neuromorphic circuits that communicate with the brain, as in this case it is essential that artificial and biological neurons use compatible neural codes. However, we also discuss implications for the design of neuromorphic systems for implementation or emulation of neural computation.
△ Less
Submitted 8 December, 2022;
originally announced December 2022.
-
Biologically plausible solutions for spiking networks with efficient coding
Authors:
Veronika Koren,
Stefano Panzeri
Abstract:
Understanding how the dynamics of neural networks is shaped by the computations they perform is a fundamental question in neuroscience. Recently, the framework of efficient coding proposed a theory of how spiking neural networks can compute low-dimensional stimulus signals with high efficiency. Efficient spiking networks are based on time-dependent minimization of a loss function related to inform…
▽ More
Understanding how the dynamics of neural networks is shaped by the computations they perform is a fundamental question in neuroscience. Recently, the framework of efficient coding proposed a theory of how spiking neural networks can compute low-dimensional stimulus signals with high efficiency. Efficient spiking networks are based on time-dependent minimization of a loss function related to information coding with spikes. To inform the understanding of the function and dynamics of biological networks in the brain, however, the mathematical models have to be informed by biology and obey the same constraints as biological networks. Currently, spiking network models of efficient coding have been extended to include some features of biological plausibility, such as architectures with excitatory and inhibitory neurons. However, biological realism of efficient coding theories is still limited to simple cases and does not include single neuron and network properties that are known to be key in biological circuits. Here, we revisit the theory of efficient coding with spikes to develop spiking neural networks that are closer to biological circuits. Namely, we find a biologically plausible spiking model realizing efficient coding in the case of a generalized leaky integrate-and-fire network with excitatory and inhibitory units, equipped with fast and slow synaptic currents, local homeostatic currents such as spike-triggered adaptation, hyperpolarization-activated rebound current, heterogeneous firing thresholds and resets, heterogeneous postsynaptic potentials, and structured, low-rank connectivity. We show how the complexity of E-E connectivity matrix shapes network responses.
△ Less
Submitted 24 October, 2022; v1 submitted 13 October, 2022;
originally announced October 2022.
-
Causal learning with sufficient statistics: an information bottleneck approach
Authors:
Daniel Chicharro,
Michel Besserve,
Stefano Panzeri
Abstract:
The inference of causal relationships using observational data from partially observed multivariate systems with hidden variables is a fundamental question in many scientific domains. Methods extracting causal information from conditional independencies between variables of a system are common tools for this purpose, but are limited in the lack of independencies. To surmount this limitation, we ca…
▽ More
The inference of causal relationships using observational data from partially observed multivariate systems with hidden variables is a fundamental question in many scientific domains. Methods extracting causal information from conditional independencies between variables of a system are common tools for this purpose, but are limited in the lack of independencies. To surmount this limitation, we capitalize on the fact that the laws governing the generative mechanisms of a system often result in substructures embodied in the generative functional equation of a variable, which act as sufficient statistics for the influence that other variables have on it. These functional sufficient statistics constitute intermediate hidden variables providing new conditional independencies to be tested. We propose to use the Information Bottleneck method, a technique commonly applied for dimensionality reduction, to find underlying sufficient sets of statistics. Using these statistics we formulate new additional rules of causal orientation that provide causal information not obtainable from standard structure learning algorithms, which exploit only conditional independencies between observable variables. We validate the use of sufficient statistics for structure learning both with simulated systems built to contain specific sufficient statistics and with benchmark data from regulatory rules previously and independently proposed to model biological signal transduction networks.
△ Less
Submitted 11 October, 2020;
originally announced October 2020.
-
Conditionally-additive-noise Models for Structure Learning
Authors:
Daniel Chicharro,
Stefano Panzeri,
Ilya Shpitser
Abstract:
Constraint-based structure learning algorithms infer the causal structure of multivariate systems from observational data by determining an equivalent class of causal structures compatible with the conditional independencies in the data. Methods based on additive-noise (AN) models have been proposed to further discriminate between causal structures that are equivalent in terms of conditional indep…
▽ More
Constraint-based structure learning algorithms infer the causal structure of multivariate systems from observational data by determining an equivalent class of causal structures compatible with the conditional independencies in the data. Methods based on additive-noise (AN) models have been proposed to further discriminate between causal structures that are equivalent in terms of conditional independencies. These methods rely on a particular form of the generative functional equations, with an additive noise structure, which allows inferring the directionality of causation by testing the independence between the residuals of a nonlinear regression and the predictors (nrr-independencies). Full causal structure identifiability has been proven for systems that contain only additive-noise equations and have no hidden variables. We extend the AN framework in several ways. We introduce alternative regression-free tests of independence based on conditional variances (cv-independencies). We consider conditionally-additive-noise (CAN) models, in which the equations may have the AN form only after conditioning. We exploit asymmetries in nrr-independencies or cv-independencies resulting from the CAN form to derive a criterion that infers the causal relation between a pair of variables in a multivariate system without any assumption about the form of the equations or the presence of hidden variables.
△ Less
Submitted 20 May, 2019;
originally announced May 2019.
-
Mathematical studies of the dynamics of finite-size binary neural networks: A review of recent progress
Authors:
Diego Fasoli,
Stefano Panzeri
Abstract:
Traditional mathematical approaches to studying analytically the dynamics of neural networks rely on the mean-field approximation, which is rigorously applicable only to networks of infinite size. However, all existing real biological networks have finite size, and many of them, such as microscopic circuits in invertebrates, are composed only of a few tens of neurons. Thus, it is important to be a…
▽ More
Traditional mathematical approaches to studying analytically the dynamics of neural networks rely on the mean-field approximation, which is rigorously applicable only to networks of infinite size. However, all existing real biological networks have finite size, and many of them, such as microscopic circuits in invertebrates, are composed only of a few tens of neurons. Thus, it is important to be able to extend to small-size networks our ability to study analytically neural dynamics. Analytical solutions of the dynamics of finite-size neural networks have remained elusive for many decades, because the powerful methods of statistical analysis, such as the central limit theorem and the law of large numbers, do not apply to small networks. In this article, we critically review recent progress on the study of the dynamics of small networks composed of binary neurons. In particular, we review the mathematical techniques we developed for studying the bifurcations of the network dynamics, the dualism between neural activity and membrane potentials, cross-neuron correlations, and pattern storage in stochastic networks. Finally, we highlight key challenges that remain open, future directions for further progress, and possible implications of our results for neuroscience.
△ Less
Submitted 29 April, 2019;
originally announced April 2019.
-
Stationary-State Statistics of a Binary Neural Network Model with Quenched Disorder
Authors:
Diego Fasoli,
Stefano Panzeri
Abstract:
We study the statistical properties of the stationary firing-rate states of a neural network model with quenched disorder. The model has arbitrary size, discrete-time evolution equations and binary firing rates, while the topology and the strength of the synaptic connections are randomly generated from known, generally arbitrary, probability distributions. We derived semi-analytical expressions of…
▽ More
We study the statistical properties of the stationary firing-rate states of a neural network model with quenched disorder. The model has arbitrary size, discrete-time evolution equations and binary firing rates, while the topology and the strength of the synaptic connections are randomly generated from known, generally arbitrary, probability distributions. We derived semi-analytical expressions of the occurrence probability of the stationary states and the mean multistability diagram of the model, in terms of the distribution of the synaptic connections and of the external stimuli to the network. Our calculations rely on the probability distribution of the bifurcation points of the stationary states with respect to the external stimuli, which can be calculated in terms of the permanent of special matrices, according to extreme value theory. While our semi-analytical expressions are exact for any size of the network and for any distribution of the synaptic connections, we also specialized our calculations to the case of statistically-homogeneous multi-population networks. In the specific case of this network topology, we calculated analytically the permanent, obtaining a compact formula that outperforms of several orders of magnitude the Balasubramanian-Bax-Franklin-Glynn algorithm. To conclude, by applying the Fisher-Tippett-Gnedenko theorem, we derived asymptotic expressions of the stationary-state statistics of multi-population networks in the large-network-size limit, in terms of the Gumbel (double exponential) distribution. We also provide a Python implementation of our formulas and some examples of the results generated by the code.
△ Less
Submitted 29 November, 2018;
originally announced November 2018.
-
Information estimation using nonparametric copulas
Authors:
Houman Safaai,
Arno Onken,
Christopher D. Harvey,
Stefano Panzeri
Abstract:
Estimation of mutual information between random variables has become crucial in a range of fields, from physics to neuroscience to finance. Estimating information accurately over a wide range of conditions relies on the development of flexible methods to describe statistical dependencies among variables, without imposing potentially invalid assumptions on the data. Such methods are needed in cases…
▽ More
Estimation of mutual information between random variables has become crucial in a range of fields, from physics to neuroscience to finance. Estimating information accurately over a wide range of conditions relies on the development of flexible methods to describe statistical dependencies among variables, without imposing potentially invalid assumptions on the data. Such methods are needed in cases that lack prior knowledge of their statistical properties and that have limited sample numbers. Here we propose a powerful and generally applicable information estimator based on non-parametric copulas. This estimator, called the non-parametric copula-based estimator (NPC), is tailored to take into account detailed stochastic relationships in the data independently of the data's marginal distributions. The NPC estimator can be used both for continuous and discrete numerical variables and thus provides a single framework for the mutual information estimation of both continuous and discrete data. By extensive validation on artificial samples drawn from various statistical distributions, we found that the NPC estimator compares well against commonly used alternatives. Unlike methods not based on copulas, it allows an estimation of information that is robust to changes of the details of the marginal distributions. Unlike parametric copula methods, it remains accurate regardless of the precise form of the interactions between the variables. In addition, the NPC estimator had accurate information estimates even at low sample numbers, in comparison to alternative estimators. The NPC estimator therefore provides a good balance between general applicability to arbitrarily shaped statistical dependencies in the data and shows accurate and robust performance when working with small sample sizes.
△ Less
Submitted 25 October, 2018; v1 submitted 20 July, 2018;
originally announced July 2018.
-
Synthesizing realistic neural population activity patterns using Generative Adversarial Networks
Authors:
Manuel Molano-Mazon,
Arno Onken,
Eugenio Piasini,
Stefano Panzeri
Abstract:
The ability to synthesize realistic patterns of neural activity is crucial for studying neural information processing. Here we used the Generative Adversarial Networks (GANs) framework to simulate the concerted activity of a population of neurons. We adapted the Wasserstein-GAN variant to facilitate the generation of unconstrained neural population activity patterns while still benefiting from par…
▽ More
The ability to synthesize realistic patterns of neural activity is crucial for studying neural information processing. Here we used the Generative Adversarial Networks (GANs) framework to simulate the concerted activity of a population of neurons. We adapted the Wasserstein-GAN variant to facilitate the generation of unconstrained neural population activity patterns while still benefiting from parameter sharing in the temporal domain. We demonstrate that our proposed GAN, which we termed Spike-GAN, generates spike trains that match accurately the first- and second-order statistics of datasets of tens of neurons and also approximates well their higher-order statistics. We applied Spike-GAN to a real dataset recorded from salamander retina and showed that it performs as well as state-of-the-art approaches based on the maximum entropy and the dichotomized Gaussian frameworks. Importantly, Spike-GAN does not require to specify a priori the statistics to be matched by the model, and so constitutes a more flexible method than these alternative approaches. Finally, we show how to exploit a trained Spike-GAN to construct 'importance maps' to detect the most relevant statistical structures present in a spike train. Spike-GAN provides a powerful, easy-to-use technique for generating realistic spiking neural activity and for describing the most relevant features of the large-scale neural population recordings studied in modern systems neuroscience.
△ Less
Submitted 17 April, 2018; v1 submitted 1 March, 2018;
originally announced March 2018.
-
Quantifying how much sensory information in a neural code is relevant for behavior
Authors:
Giuseppe Pica,
Eugenio Piasini,
Houman Safaai,
Caroline A. Runyan,
Mathew E. Diamond,
Tommaso Fellin,
Christoph Kayser,
Christopher D. Harvey,
Stefano Panzeri
Abstract:
Determining how much of the sensory information carried by a neural code contributes to behavioral performance is key to understand sensory function and neural information flow. However, there are as yet no analytical tools to compute this information that lies at the intersection between sensory coding and behavioral readout. Here we develop a novel measure, termed the information-theoretic inter…
▽ More
Determining how much of the sensory information carried by a neural code contributes to behavioral performance is key to understand sensory function and neural information flow. However, there are as yet no analytical tools to compute this information that lies at the intersection between sensory coding and behavioral readout. Here we develop a novel measure, termed the information-theoretic intersection information $I_{II}(S;R;C)$, that quantifies how much of the sensory information carried by a neural response R is used for behavior during perceptual discrimination tasks. Building on the Partial Information Decomposition framework, we define $I_{II}(S;R;C)$ as the part of the mutual information between the stimulus S and the response R that also informs the consequent behavioral choice C. We compute $I_{II}(S;R;C)$ in the analysis of two experimental cortical datasets, to show how this measure can be used to compare quantitatively the contributions of spike timing and spike rates to task performance, and to identify brain areas or neural populations that specifically transform sensory information into choice.
△ Less
Submitted 6 December, 2017;
originally announced December 2017.
-
The identity of information: how deterministic dependencies constrain information synergy and redundancy
Authors:
Daniel Chicharro,
Giuseppe Pica,
Stefano Panzeri
Abstract:
Understanding how different information sources together transmit information is crucial in many domains. For example, understanding the neural code requires characterizing how different neurons contribute unique, redundant, or synergistic pieces of information about sensory or behavioral variables. Williams and Beer (2010) proposed a partial information decomposition (PID) which separates the mut…
▽ More
Understanding how different information sources together transmit information is crucial in many domains. For example, understanding the neural code requires characterizing how different neurons contribute unique, redundant, or synergistic pieces of information about sensory or behavioral variables. Williams and Beer (2010) proposed a partial information decomposition (PID) which separates the mutual information that a set of sources contains about a set of targets into nonnegative terms interpretable as these pieces. Quantifying redundancy requires assigning an identity to different information pieces, to assess when information is common across sources. Harder et al. (2013) proposed an identity axiom stating that there cannot be redundancy between two independent sources about a copy of themselves. However, Bertschinger et al. (2012) showed that with a deterministically related sources-target copy this axiom is incompatible with ensuring PID nonnegativity. Here we study systematically the effect of deterministic target-sources dependencies. We introduce two synergy stochasticity axioms that generalize the identity axiom, and we derive general expressions separating stochastic and deterministic PID components. Our analysis identifies how negative terms can originate from deterministic dependencies and shows how different assumptions on information identity, implicit in the stochasticity and identity axioms, determine the PID structure. The implications for studying neural coding are discussed.
△ Less
Submitted 13 November, 2017;
originally announced November 2017.
-
Invariant components of synergy, redundancy, and unique information among three variables
Authors:
Giuseppe Pica,
Eugenio Piasini,
Daniel Chicharro,
Stefano Panzeri
Abstract:
In a system of three stochastic variables, the Partial Information Decomposition (PID) of Williams and Beer dissects the information that two variables (sources) carry about a third variable (target) into nonnegative information atoms that describe redundant, unique, and synergistic modes of dependencies among the variables. However, the classification of the three variables into two sources and o…
▽ More
In a system of three stochastic variables, the Partial Information Decomposition (PID) of Williams and Beer dissects the information that two variables (sources) carry about a third variable (target) into nonnegative information atoms that describe redundant, unique, and synergistic modes of dependencies among the variables. However, the classification of the three variables into two sources and one target limits the dependency modes that can be quantitatively resolved, and does not naturally suit all systems. Here, we extend the PID to describe trivariate modes of dependencies in full generality, without introducing additional decomposition axioms or making assumptions about the target/source nature of the variables. By comparing different PID lattices of the same system, we unveil a finer PID structure made of seven nonnegative information subatoms that are invariant to different target/source classifications and that are sufficient to construct any PID lattice. This finer structure naturally splits redundant information into two nonnegative components: the source redundancy, which arises from the pairwise correlations between the source variables, and the non-source redundancy, which does not, and relates to the synergistic information the sources carry about the target. The invariant structure is also sufficient to construct the system's entropy, hence it characterizes completely all the interdependencies in the system.
△ Less
Submitted 27 June, 2017;
originally announced June 2017.
-
Optimized brute-force algorithms for the bifurcation analysis of a spin-glass-like neural network model
Authors:
Diego Fasoli,
Stefano Panzeri
Abstract:
Bifurcation theory is a powerful tool for studying how the dynamics of a neural network model depends on its underlying neurophysiological parameters. However, bifurcation theory has been developed mostly for smooth dynamical systems and for continuous-time non-smooth models, which prevents us from understanding the changes of dynamics in some widely used classes of artificial neural network model…
▽ More
Bifurcation theory is a powerful tool for studying how the dynamics of a neural network model depends on its underlying neurophysiological parameters. However, bifurcation theory has been developed mostly for smooth dynamical systems and for continuous-time non-smooth models, which prevents us from understanding the changes of dynamics in some widely used classes of artificial neural network models. This article is an attempt to fill this gap, through the introduction of algorithms that perform a semi-analytical bifurcation analysis of a spin-glass-like neural network model with binary firing rates and discrete-time evolution. Our approach is based on a numerical brute-force search of the stationary and oscillatory solutions of the spin-glass model, from which we derive analytical expressions of its bifurcation structure by means of the state-to-state transition probability matrix. The algorithms determine how the network parameters affect the degree of multistability, the emergence and the period of the neural oscillations, and the formation of symmetry-breaking in the neural populations. While this technique can be applied to networks with arbitrary (generally asymmetric) connectivity matrices, in particular we introduce a highly efficient algorithm for the bifurcation analysis of sparse networks. We also provide some examples of the obtained bifurcation diagrams and a Python implementation of the algorithms.
△ Less
Submitted 16 May, 2017;
originally announced May 2017.
-
Pattern Storage, Bifurcations and Higher-Order Correlation Structure of an Exactly Solvable Asymmetric Neural Network Model
Authors:
Diego Fasoli,
Anna Cattani,
Stefano Panzeri
Abstract:
Exactly solvable neural network models with asymmetric weights are rare, and exact solutions are available only in some mean-field approaches. In this article we find exact analytical solutions of an asymmetric spin-glass-like model of arbitrary size and we perform a complete study of its dynamical and statistical properties. The network has discrete-time evolution equations, binary firing rates a…
▽ More
Exactly solvable neural network models with asymmetric weights are rare, and exact solutions are available only in some mean-field approaches. In this article we find exact analytical solutions of an asymmetric spin-glass-like model of arbitrary size and we perform a complete study of its dynamical and statistical properties. The network has discrete-time evolution equations, binary firing rates and can be driven by noise with any distribution. We find analytical expressions of the conditional and stationary joint probability distributions of the membrane potentials and the firing rates. The conditional probability distribution of the firing rates allows us to introduce a new learning rule to store safely, under the presence of noise, point and cyclic attractors, with important applications in the field of content-addressable memories. Furthermore, we study the neuronal dynamics in terms of the bifurcation structure of the network. We derive analytically examples of the codimension one and codimension two bifurcation diagrams of the network, which describe how the neuronal dynamics changes with the external stimuli. In particular, we find that the network may undergo transitions among multistable regimes, oscillatory behavior elicited by asymmetric synaptic connections, and various forms of spontaneous symmetry-breaking. On the other hand, the joint probability distributions allow us to calculate analytically the higher-order correlation structure of the network, which reveals neuronal regimes where, statistically, the membrane potentials and the firing rates are either synchronous or asynchronous. Our results are valid for networks composed of an arbitrary number of neurons, but for completeness we also derive the network equations in the mean-field limit and we study analytically their local bifurcations. All the analytical results are extensively validated by numerical simulations.
△ Less
Submitted 15 February, 2017; v1 submitted 10 February, 2017;
originally announced February 2017.
-
Redundancy and synergy in dual decompositions of mutual information gain and information loss
Authors:
Daniel Chicharro,
Stefano Panzeri
Abstract:
Williams and Beer (2010) proposed a nonnegative mutual information decomposition, based on the construction of information gain lattices, which allows separating the information that a set of variables contains about another into components interpretable as the unique information of one variable, or redundant and synergy components. In this work we extend the framework of Williams and Beer (2010)…
▽ More
Williams and Beer (2010) proposed a nonnegative mutual information decomposition, based on the construction of information gain lattices, which allows separating the information that a set of variables contains about another into components interpretable as the unique information of one variable, or redundant and synergy components. In this work we extend the framework of Williams and Beer (2010) focusing on the lattices that underpin the decomposition. We generalize the type of constructible lattices and examine the relations between the terms in different lattices, for example relating bivariate and trivariate decompositions. We point out that, in information gain lattices, redundancy components are invariant across decompositions, but unique and synergy components are decomposition-dependent. Exploiting the connection between different lattices we propose a procedure to construct, in the general multivariate case, information decompositions from measures of synergy or unique information. We introduce an alternative type of mutual information decompositions based on information loss lattices, with the role and invariance properties of redundancy and synergy components exchanged with respect to gain lattices. We study the correspondence between information gain and information loss lattices and we define dual decompositions that allow overcoming the intrinsic asymmetry between invariant and decomposition-dependent components, which hinders the consistent joint characterization of synergy and redundancy.
△ Less
Submitted 30 December, 2016;
originally announced December 2016.
-
From Local Chaos to Critical Slowing Down: A Theory of the Functional Connectivity of Small Neural Circuits
Authors:
Diego Fasoli,
Anna Cattani,
Stefano Panzeri
Abstract:
Functional connectivity is a fundamental property of neural networks that quantifies the segregation and integration of information between cortical areas. Due to mathematical complexity, a theory that could explain how the parameters of mesoscopic networks composed of a few tens of neurons affect the functional connectivity is still to be formulated. Yet, many interesting problems in neuroscience…
▽ More
Functional connectivity is a fundamental property of neural networks that quantifies the segregation and integration of information between cortical areas. Due to mathematical complexity, a theory that could explain how the parameters of mesoscopic networks composed of a few tens of neurons affect the functional connectivity is still to be formulated. Yet, many interesting problems in neuroscience involve the study of networks composed of a small number of neurons. Based on a recent study of the dynamics of small neural circuits, we combine the analysis of local bifurcations of multi-population neural networks of arbitrary size with the analytical calculation of the functional connectivity. We study the functional connectivity in different regimes, showing that external stimuli cause the network to switch from asynchronous states characterized by weak correlation and low variability (local chaos), to synchronous states characterized by strong correlations and wide temporal fluctuations (critical slowing down). Local chaos typically occurs in large networks, but here we show that it can also be generated by strong stimuli in small neural circuits. On the other side, critical slowing down is expected to occur when the stimulus moves the network close to a local bifurcation. In particular, strongly positive correlations occur at the saddle-node and Andronov-Hopf bifurcations of the network, while strongly negative correlations occur when the network undergoes a spontaneous symmetry-breaking at the branching-point bifurcations. These results prove that the functional connectivity of firing-rate network models is strongly affected by the external stimuli even if the anatomical connections are fixed, and suggest an effective mechanism through which biological networks can dynamically modulate the encoding and integration of sensory information.
△ Less
Submitted 26 May, 2016; v1 submitted 24 May, 2016;
originally announced May 2016.
-
The complexity of dynamics in small neural circuits
Authors:
Diego Fasoli,
Anna Cattani,
Stefano Panzeri
Abstract:
Mean-field theory is a powerful tool for studying large neural networks. However, when the system is composed of a few neurons, macroscopic differences between the mean-field approximation and the real behavior of the network can arise. Here we introduce a study of the dynamics of a small firing-rate network with excitatory and inhibitory populations, in terms of local and global bifurcations of t…
▽ More
Mean-field theory is a powerful tool for studying large neural networks. However, when the system is composed of a few neurons, macroscopic differences between the mean-field approximation and the real behavior of the network can arise. Here we introduce a study of the dynamics of a small firing-rate network with excitatory and inhibitory populations, in terms of local and global bifurcations of the neural activity. Our approach is analytically tractable in many respects, and sheds new light on the finite-size effects of the system. In particular, we focus on the formation of multiple branching solutions of the neural equations through spontaneous symmetry-breaking, since this phenomenon increases considerably the complexity of the dynamical behavior of the network. For these reasons, branching points may reveal important mechanisms through which neurons interact and process information, which are not accounted for by the mean-field approximation.
△ Less
Submitted 22 July, 2015; v1 submitted 30 June, 2015;
originally announced June 2015.
-
Phase-of-firing code
Authors:
Anna Cattani,
Gaute T. Einevoll,
Stefano Panzeri
Abstract:
Definition. The phase-of-firing code is a neural coding scheme whereby neurons encode information using the time at which they fire spikes within a cycle of the ongoing oscillatory pattern of network activity. This coding scheme may allow neurons to use their temporal pattern of spikes to encode information that is not encoded in their firing rate.
Definition. The phase-of-firing code is a neural coding scheme whereby neurons encode information using the time at which they fire spikes within a cycle of the ongoing oscillatory pattern of network activity. This coding scheme may allow neurons to use their temporal pattern of spikes to encode information that is not encoded in their firing rate.
△ Less
Submitted 15 April, 2015;
originally announced April 2015.
-
Estimating Information-Theoretic Quantities
Authors:
Robin A. A. Ince,
Simon R. Schultz,
Stefano Panzeri
Abstract:
Information theory is a practical and theoretical framework developed for the study of communication over noisy channels. Its probabilistic basis and capacity to relate statistical structure to function make it ideally suited for studying information flow in the nervous system. It has a number of useful properties: it is a general measure sensitive to any relationship, not only linear effects; it…
▽ More
Information theory is a practical and theoretical framework developed for the study of communication over noisy channels. Its probabilistic basis and capacity to relate statistical structure to function make it ideally suited for studying information flow in the nervous system. It has a number of useful properties: it is a general measure sensitive to any relationship, not only linear effects; it has meaningful units which in many cases allow direct comparison between different experiments; and it can be used to study how much information can be gained by observing neural responses in single trials, rather than in averages over multiple trials. A variety of information theoretic quantities are in common use in neuroscience - (see entry "Summary of Information-Theoretic Quantities"). Estimating these quantities in an accurate and unbiased way from real neurophysiological data frequently presents challenges, which are explained in this entry.
△ Less
Submitted 8 January, 2015;
originally announced January 2015.
-
Applications of Information Theory to Analysis of Neural Data
Authors:
Simon R. Schultz,
Robin A. A. Ince,
Stefano Panzeri
Abstract:
Information theory is a practical and theoretical framework developed for the study of communication over noisy channels. Its probabilistic basis and capacity to relate statistical structure to function make it ideally suited for studying information flow in the nervous system. It has a number of useful properties: it is a general measure sensitive to any relationship, not only linear effects; it…
▽ More
Information theory is a practical and theoretical framework developed for the study of communication over noisy channels. Its probabilistic basis and capacity to relate statistical structure to function make it ideally suited for studying information flow in the nervous system. It has a number of useful properties: it is a general measure sensitive to any relationship, not only linear effects; it has meaningful units which in many cases allow direct comparison between different experiments; and it can be used to study how much information can be gained by observing neural responses in single trials, rather than in averages over multiple trials. A variety of information theoretic quantities are commonly used in neuroscience - (see entry "Definitions of Information-Theoretic Quantities"). In this entry we review some applications of information theory in neuroscience to study encoding of information in both single neurons and neuronal populations.
△ Less
Submitted 8 January, 2015;
originally announced January 2015.
-
Summary of Information Theoretic Quantities
Authors:
Robin A. A. Ince,
Stefano Panzeri,
Simon R. Schultz
Abstract:
Information theory is a practical and theoretical framework developed for the study of communication over noisy channels. Its probabilistic basis and capacity to relate statistical structure to function make it ideally suited for studying information flow in the nervous system. As a framework it has a number of useful properties: it provides a general measure sensitive to any relationship, not onl…
▽ More
Information theory is a practical and theoretical framework developed for the study of communication over noisy channels. Its probabilistic basis and capacity to relate statistical structure to function make it ideally suited for studying information flow in the nervous system. As a framework it has a number of useful properties: it provides a general measure sensitive to any relationship, not only linear effects; its quantities have meaningful units which in many cases allow direct comparison between different experiments; and it can be used to study how much information can be gained by observing neural responses in single experimental trials, rather than in averages over multiple trials. A variety of information theoretic quantities are in common use in neuroscience - including the Shannon entropy, Kullback-Leibler divergence, and mutual information. In this entry, we introduce and define these quantities. Further details on how these quantities can be estimated in practice are provided in the entry "Estimation of Information-Theoretic Quantities" and examples of application of these techniques in neuroscience can be found in the entry "Applications of Information-Theoretic Quantities in Neuroscience".
△ Less
Submitted 8 January, 2015;
originally announced January 2015.
-
The information content of Local Field Potentials: experiments and models
Authors:
Alberto Mazzoni,
Nikos K. Logothetis,
Stefano Panzeri
Abstract:
The LFPs is a broadband signal that captures variations of neural population activity over a wide range of time scales. The range of time scales available in LFPs is particularly interesting from the neural coding point of view because it opens up the possibility to investigate whether there are privileged time scales for information processing, a question that has been hotly debated over the last…
▽ More
The LFPs is a broadband signal that captures variations of neural population activity over a wide range of time scales. The range of time scales available in LFPs is particularly interesting from the neural coding point of view because it opens up the possibility to investigate whether there are privileged time scales for information processing, a question that has been hotly debated over the last one or two decades.It is possible that information is represented by only a small number of specific frequency ranges, each carrying a separate contribution to the information representation. To shed light on this issue, it is important to quantify the information content of each frequency range of neural activity, and understand which ranges carry complementary or similar information.
△ Less
Submitted 4 June, 2012;
originally announced June 2012.
-
Synchronisation, binding and the role of correlated firing in fast information transmission
Authors:
Simon R. Schultz,
Huw D. R. Golledge,
Stefano Panzeri
Abstract:
Does synchronization between action potentials from different neurons in the visual system play a substantial role in solving the binding problem? The binding problem can be studied quantitatively in the broader framework of the information contained in neural spike trains about some external correlate, which in this case is object configurations in the visual field. We approach this problem by…
▽ More
Does synchronization between action potentials from different neurons in the visual system play a substantial role in solving the binding problem? The binding problem can be studied quantitatively in the broader framework of the information contained in neural spike trains about some external correlate, which in this case is object configurations in the visual field. We approach this problem by using a mathematical formalism that quantifies the impact of correlated firing in short time scales. Using a power series expansion, the mutual information an ensemble of neurons conveys about external stimuli is broken down into firing rate and correlation components. This leads to a new quantification procedure directly applicable to simultaneous multiple neuron recordings. It theoretically constrains the neural code, showing that correlations contribute less significantly than firing rates to rapid information processing. By using this approach to study the limits upon the amount of information that an ideal observer is able to extract from a synchrony code, it may be possible to determine whether the available amount of information is sufficient to support computational processes such as feature binding.
△ Less
Submitted 30 June, 2000;
originally announced June 2000.
-
Temporal correlations and neural spike train entropy
Authors:
Simon R. Schultz,
Stefano Panzeri
Abstract:
Sampling considerations limit the experimental conditions under which information theoretic analyses of neurophysiological data yield reliable results. We develop a procedure for computing the full temporal entropy and information of ensembles of neural spike trains, which performs reliably for limited samples of data. This approach also yields insight upon the role of correlations between spike…
▽ More
Sampling considerations limit the experimental conditions under which information theoretic analyses of neurophysiological data yield reliable results. We develop a procedure for computing the full temporal entropy and information of ensembles of neural spike trains, which performs reliably for limited samples of data. This approach also yields insight upon the role of correlations between spikes in temporal coding mechanisms. The method, when applied to recordings from complex cells of the monkey primary visual cortex, results in lower RMS error information estimates in comparison to a `brute force' approach.
△ Less
Submitted 11 April, 2001; v1 submitted 4 January, 2000;
originally announced January 2000.
-
A unified approach to the study of temporal, correlational and rate coding
Authors:
S. Panzeri,
S. R. Schultz
Abstract:
We demonstrate that the information contained in the spike occurrence times of a population of neurons can be broken up into a series of terms, each of which reflect something about potential coding mechanisms. This is possible in the coding r{é}gime in which few spikes are emitted in the relevant time window.
This approach allows us to study the additional information contributed by spike tim…
▽ More
We demonstrate that the information contained in the spike occurrence times of a population of neurons can be broken up into a series of terms, each of which reflect something about potential coding mechanisms. This is possible in the coding r{é}gime in which few spikes are emitted in the relevant time window.
This approach allows us to study the additional information contributed by spike timing beyond that present in the spike counts; to examine the contributions to the whole information of different statistical properties of spike trains, such as firing rates and correlation functions; and forms the basis for a new quantitative procedure for the analysis of simultaneous multiple neuron recordings. It also provides theoretical constraints upon neural coding strategies. We find a transition between two coding r{é}gimes, depending upon the size of the relevant observation timescale. For time windows shorter than the timescale of the stimulus-induced response fluctuations, there exists a spike count coding phase, where the purely temporal information is of third order in time. For time windows much longer than the characteristic timescale, there can be additional timing information of first order, leading to a temporal coding phase in which timing information may affect the instantaneous information rate.
We study the relative contributions of the dynamic firing rate and correlation variables to the full temporal information; the interaction of signal and noise correlations in temporal coding; synergy between spikes and between cells; and the effect of refractoriness. We illustrate the utility of the technique by analysis of a few cells from the rat barrel cortex.
△ Less
Submitted 23 August, 2000; v1 submitted 13 August, 1999;
originally announced August 1999.
-
Quantitative analysis of a Schaffer collateral model
Authors:
S Schultz,
S Panzeri,
ET Rolls,
A Treves
Abstract:
Advances in techniques for the formal analysis of neural networks have introduced the possibility of detailed quantitative analyses of brain circuitry. This paper applies a method for calculating mutual information to the analysis of the Schaffer collateral connections between regions CA3 and CA1 of the hippocampus. Attention is given to the introduction of further details of anatomy and physiol…
▽ More
Advances in techniques for the formal analysis of neural networks have introduced the possibility of detailed quantitative analyses of brain circuitry. This paper applies a method for calculating mutual information to the analysis of the Schaffer collateral connections between regions CA3 and CA1 of the hippocampus. Attention is given to the introduction of further details of anatomy and physiology to the calculation: in particular, the distribution of the number of connections that CA1 neurons receive from CA3, and the graded nature of the firing-rate distribution in region CA3.
△ Less
Submitted 19 September, 1997; v1 submitted 28 April, 1997;
originally announced April 1997.
-
Finite Temperature Lattice QCD in the Large N Limit
Authors:
M. Billo',
M. Caselle,
A. D'Adda,
S. Panzeri
Abstract:
Our aim is to give a self-contained review of recent advances in the analytic description of the deconfinement transition and determination of the deconfinement temperature in lattice QCD at large N. We also include some new results, as for instance in the comparison of the analytic results with Montecarlo simulations. We first review the general set-up of finite temperature lattice gauge theori…
▽ More
Our aim is to give a self-contained review of recent advances in the analytic description of the deconfinement transition and determination of the deconfinement temperature in lattice QCD at large N. We also include some new results, as for instance in the comparison of the analytic results with Montecarlo simulations. We first review the general set-up of finite temperature lattice gauge theories, using asymmetric lattices, and develop a consistent perturbative expansion in the coupling $β_s$ of the space-like plaquettes. We study in detail the effective models for the Polyakov loop obtained, in the zeroth order approximation in $β_s$, both from the Wilson action (symmetric lattice) and from the heat kernel action (completely asymmetric lattice). The distinctive feature of the heat kernel model is its relation with two-dimensional QCD on a cylinder; the Wilson model, on the other hand, can be exactly reduced to a twisted one-plaquette model via a procedure of the Eguchi-Kawai type. In the weak coupling regime both models can be related to exactly solvable Kazakov-Migdal matrix models. The instability of the weak coupling solution is due in both cases to a condensation of instantons; in the heat kernel case, it is directly related to the Douglas-Kazakov transition of QCD2. A detailed analysis of these results provides rather accurate predictions of the deconfinement temperature. In spite of the zeroth order approximation they are in good agreement with the Montecarlo simulations in 2+1 dimensions, while in 3+1 dimensions they only agree with the Montecarlo results away from the continuum limit.
△ Less
Submitted 18 October, 1996;
originally announced October 1996.
-
Effective actions for finite temperature Lattice Gauge Theories
Authors:
M. Billo',
M. Caselle,
A. D'Adda,
S. Panzeri
Abstract:
We consider a lattice gauge theory at finite temperature in ($d$+1) dimensions with the Wilson action and different couplings $β_t$ and $β_s$ for timelike and spacelike plaquettes. By using the character expansion and Schwinger-Dyson type equations we construct, order by order in $β_s$, an effective action for the Polyakov loops which is exact to all orders in $β_t$. As an example we construct t…
▽ More
We consider a lattice gauge theory at finite temperature in ($d$+1) dimensions with the Wilson action and different couplings $β_t$ and $β_s$ for timelike and spacelike plaquettes. By using the character expansion and Schwinger-Dyson type equations we construct, order by order in $β_s$, an effective action for the Polyakov loops which is exact to all orders in $β_t$. As an example we construct the first non-trivial order in $β_s$ for the (3+1) dimensional SU(2) model and use this effective action to extract the deconfinement temperature of the model.
△ Less
Submitted 12 September, 1996;
originally announced September 1996.
-
Toward an analytic determination of the deconfinement temperature in SU(2) L.G.T.
Authors:
M. Billo',
M. Caselle,
A. D'Adda,
S. Panzeri
Abstract:
We consider the SU(2) lattice gauge theory at finite temperature in (d+1) dimensions, with different couplings $β_t$ and $β_s$ for timelike and spacelike plaquettes. By using the character expansion of the Wilson action and performing the integrals over space-like link variables, we find an effective action for the Polyakov loops which is exact to all orders in $β_t$ and to the first non-trivial…
▽ More
We consider the SU(2) lattice gauge theory at finite temperature in (d+1) dimensions, with different couplings $β_t$ and $β_s$ for timelike and spacelike plaquettes. By using the character expansion of the Wilson action and performing the integrals over space-like link variables, we find an effective action for the Polyakov loops which is exact to all orders in $β_t$ and to the first non-trivial order in $β_s$. The critical coupling for the deconfinement transition is determined in the (3+1) dimensional case, by the mean field method, for different values of the lattice size $N_t$ in the compactified time direction and of the asymmetry parameter $ρ= \sqrt{β_t/β_s}$. We find good agreement with Montecarlo simulations in the range $1\leq N_t \leq 5$, and good qualitative agreement in the same range with the logarithmic scaling law of QCD. Moreover the dependence of the results from the parameter $ρ$ is in excellent agreement with previous theoretical predictions.
△ Less
Submitted 29 January, 1996;
originally announced January 1996.
-
Deconfinement Transition in Large N Lattice Gauge Theory
Authors:
M. Billo',
M. Caselle,
A. D'Adda,
L. Magnea,
S. Panzeri
Abstract:
We study analytically the phase diagram of the pure $SU(N)$ lattice gauge theory at finite temperature, and we attempt to estimate the critical deconfinement temperature. We apply large $N$ techniques to the Wilson and to the Heat Kernel action, and we study the resulting models both in the strong coupling and in the weak coupling limits. Using the Heat Kernel action, we establish an interesting…
▽ More
We study analytically the phase diagram of the pure $SU(N)$ lattice gauge theory at finite temperature, and we attempt to estimate the critical deconfinement temperature. We apply large $N$ techniques to the Wilson and to the Heat Kernel action, and we study the resulting models both in the strong coupling and in the weak coupling limits. Using the Heat Kernel action, we establish an interesting connection between the Douglas-Kazakov phase transition of two-dimensional QCD and the deconfining transition in $d$ dimensions. The analytic results obtained for the critical temperature compare well with Montecarlo simulations of the full theory in $(2+1)$ and in $(3+1)$ dimensions.
△ Less
Submitted 26 July, 1994;
originally announced July 1994.
-
Two-dimensional QCD on the sphere and on the cylinder
Authors:
M. Caselle,
A. D'Adda,
L. Magnea,
S. Panzeri
Abstract:
The partition functions of QCD2 on simple surfaces admit representations in terms of exponentials of the inverse coupling, that are modular transforms of the usual character expansions. We review the construction of such a representation in the case of the cylinder, and show how it leads to a formulation of QCD2 as a $c=1$ matrix model of the Kazakov-Migdal type. The eigenvalues describe the pos…
▽ More
The partition functions of QCD2 on simple surfaces admit representations in terms of exponentials of the inverse coupling, that are modular transforms of the usual character expansions. We review the construction of such a representation in the case of the cylinder, and show how it leads to a formulation of QCD2 as a $c=1$ matrix model of the Kazakov-Migdal type. The eigenvalues describe the positions of $N$ Sutherland fermions on a circle, while their discretized momenta label the representations in the corresponding character expansion. Using this language, we derive some new results: we give an alternative description of the Douglas-Kazakov phase transition on the sphere, and we argue that an analogous phase transition exists on the cylinder. We calculate the large $N$ limit of the partition function on the cylinder with boundary conditions given by semicircular distributions of eigenvalues, and we find an explicit expression for the large $N$ limit of the Itzykson-Zuber integral with the same boundary conditions. (Talk given at the ``Workshop on high energy physics and cosmology'' at Trieste, July 1993.)
△ Less
Submitted 20 September, 1993;
originally announced September 1993.
-
The C = 1 Matrix Model Formulation of Two Dimensional Yang-Mills Theories
Authors:
Stefano Panzeri
Abstract:
We find the exact matrix model description of two dimensional Yang-Mills theories on a cylinder or on a torus and with an arbitrary compact gauge group. This matrix model is the singlet sector of a $c =1$ matrix model where the matrix field is in the fundamental representation of the gauge group. We also prove that the basic constituents of the theory are Sutherland fermions in the zero coupling…
▽ More
We find the exact matrix model description of two dimensional Yang-Mills theories on a cylinder or on a torus and with an arbitrary compact gauge group. This matrix model is the singlet sector of a $c =1$ matrix model where the matrix field is in the fundamental representation of the gauge group. We also prove that the basic constituents of the theory are Sutherland fermions in the zero coupling limit, and this leads to an interesting connection between two dimensional gauge theories and one dimensional integrable systems. In particular we derive for all the classical groups the exact grand canonical partition function of the free fermion system corresponding to a two dimensional gauge theory on a torus.
△ Less
Submitted 28 July, 1993;
originally announced July 1993.
-
Two dimensional QCD is a one dimensional Kazakov-Migdal model
Authors:
M. Caselle,
A. D'Adda,
L. Magnea,
S. Panzeri
Abstract:
We calculate the partition functions of QCD in two dimensions on a cylinder and on a torus in the gauge $\partial_{0} A_{0} = 0$ by integrating explicitly over the non zero modes of the Fourier expansion in the periodic time variable. The result is a one dimensional Kazakov-Migdal matrix model with eigenvalues on a circle rather than on a line. We prove that our result coincides with the standar…
▽ More
We calculate the partition functions of QCD in two dimensions on a cylinder and on a torus in the gauge $\partial_{0} A_{0} = 0$ by integrating explicitly over the non zero modes of the Fourier expansion in the periodic time variable. The result is a one dimensional Kazakov-Migdal matrix model with eigenvalues on a circle rather than on a line. We prove that our result coincides with the standard expansion in representations of the gauge group. This involves a non trivial modular transformation from an expansion in exponentials of $g^2$ to one in exponentials of $1/g^2$. Finally we argue that the states of the $U(N)$ or $SU(N)$ partition function can be interpreted as a gas of N free fermions, and the grand canonical partition function of such ensemble is given explicitly as an infinite product.
△ Less
Submitted 9 April, 1993; v1 submitted 5 April, 1993;
originally announced April 1993.
-
The Kazakov-Migdal Model as a High Temperature Lattice Gauge Theory
Authors:
M. Caselle,
A. D'Adda,
S. Panzeri
Abstract:
We show that the Kazakov-Migdal (K-M) induced gauge model in $d$ dimensions describes the high temperature limit of ordinary lattice gauge theories in $d+1$ dimensions. The matter fields are related to the Polyakov loops, while the spatial gauge variables become the gauge fields of the K-M model. This interpretation of the K-M model is in agreement with some recent results in high temperature la…
▽ More
We show that the Kazakov-Migdal (K-M) induced gauge model in $d$ dimensions describes the high temperature limit of ordinary lattice gauge theories in $d+1$ dimensions. The matter fields are related to the Polyakov loops, while the spatial gauge variables become the gauge fields of the K-M model. This interpretation of the K-M model is in agreement with some recent results in high temperature lattice QCD.
△ Less
Submitted 11 December, 1992;
originally announced December 1992.
-
Kazakov-Migdal Induced Gauge Theory and the Coupling of 2D Quantum Gravity to D=1 Matter
Authors:
M. Caselle,
A. D'Adda,
S. Panzeri
Abstract:
Recently Kazakov and Migdal proposed a new approach to the large $N$ limit of SU(N) gauge theories which could hopefully describe the asymptotically free fixed point of QCD in 4 dimensions. In this contribution we review the exact solution of their model in the case of a $d=1$ compactified lattice and its connection with the partition function of the vortex-free sector of the d=1 compactified bo…
▽ More
Recently Kazakov and Migdal proposed a new approach to the large $N$ limit of SU(N) gauge theories which could hopefully describe the asymptotically free fixed point of QCD in 4 dimensions. In this contribution we review the exact solution of their model in the case of a $d=1$ compactified lattice and its connection with the partition function of the vortex-free sector of the d=1 compactified bosonic string. Some new results on the addition of a linear term in the K-M model and on the SU(N) restriction of the general U(N) model are also presented.
△ Less
Submitted 2 December, 1992;
originally announced December 1992.
-
Exact solution of d=1 Kazakov-Migdal induced gauge theory
Authors:
M. Caselle,
A. D'Adda,
S. Panzeri
Abstract:
We give the exact solution of the Kazakov-Migdal induced gauge model in the case of a D=1 compactified lattice with a generic number $S$ of sites and for any value of N. Due to the peculiar features of the model, the partition function that we obtain also describes the vortex-free sector of the D=1 compactified bosonic string, and it coincides in the continuum limit with the one obtained by Boul…
▽ More
We give the exact solution of the Kazakov-Migdal induced gauge model in the case of a D=1 compactified lattice with a generic number $S$ of sites and for any value of N. Due to the peculiar features of the model, the partition function that we obtain also describes the vortex-free sector of the D=1 compactified bosonic string, and it coincides in the continuum limit with the one obtained by Boulatov and Kazakov in this context.
△ Less
Submitted 3 August, 1992; v1 submitted 27 July, 1992;
originally announced July 1992.