-
Quantifying synergy and redundancy in multiplex networks
Authors:
Andrea I. Luppi,
Eckehard Olbrich,
Conor Finn,
Laura E. Suárez,
Fernando E. Rosas,
Pedro A. M. Mediano,
Jürgen Jost
Abstract:
Understanding how different networks relate to each other is key for obtaining a greater insight into complex systems. Here, we introduce an intuitive yet powerful framework to characterise the relationship between two networks comprising the same nodes. We showcase our framework by decomposing the shortest paths between nodes as being contributed uniquely by one or the other source network, or re…
▽ More
Understanding how different networks relate to each other is key for obtaining a greater insight into complex systems. Here, we introduce an intuitive yet powerful framework to characterise the relationship between two networks comprising the same nodes. We showcase our framework by decomposing the shortest paths between nodes as being contributed uniquely by one or the other source network, or redundantly by either, or synergistically by the two together. Our approach takes into account the networks' full topology, and it also provides insights at multiple levels of resolution: from global statistics, to individual paths of different length. We show that this approach is widely applicable, from brains to the London public transport system. In humans and across 123 other mammalian species, we demonstrate that reliance on unique contributions by long-range white matter fibers is a conserved feature of mammalian structural brain networks. Across species, we also find that efficient communication relies on significantly greater synergy between long-range and short-range fibers than expected by chance, and significantly less redundancy. Our framework may find applications to help decide how to trade-off different desiderata when designing network systems, or to evaluate their relative presence in existing systems, whether biological or artificial.
△ Less
Submitted 8 August, 2023; v1 submitted 2 June, 2023;
originally announced June 2023.
-
Order-disorder transition in the zero-temperature Ising model on random graphs
Authors:
Armin Pournaki,
Eckehard Olbrich,
Sven Banisch,
Konstantin Klemm
Abstract:
The zero-temperature Ising model is known to reach a fully ordered ground state in sufficiently dense random graphs. In sparse random graphs, the dynamics gets absorbed in disordered local minima at magnetization close to zero. Here, we find that the non-equilibrium transition between the ordered and the disordered regime occurs at an average degree that slowly grows with the graph size. The syste…
▽ More
The zero-temperature Ising model is known to reach a fully ordered ground state in sufficiently dense random graphs. In sparse random graphs, the dynamics gets absorbed in disordered local minima at magnetization close to zero. Here, we find that the non-equilibrium transition between the ordered and the disordered regime occurs at an average degree that slowly grows with the graph size. The system shows bistability: The distribution of the absolute magnetization in the reached absorbing state is bimodal, with peaks only at zero and unity. For a fixed system size, the average time to absorption behaves nonmonotonically as a function of average degree. The peak value of the average absorption time grows as a power law of the system size. These findings have relevance for community detection, opinion dynamics, and games on networks.
△ Less
Submitted 30 May, 2023; v1 submitted 19 September, 2022;
originally announced September 2022.
-
Continuity and Additivity Properties of Information Decompositions
Authors:
Johannes Rauh,
Pradeep Kr. Banerjee,
Eckehard Olbrich,
Guido Montúfar,
Jürgen Jost
Abstract:
Information decompositions quantify how the Shannon information about a given random variable is distributed among several other random variables. Various requirements have been proposed that such a decomposition should satisfy, leading to different candidate solutions. Curiously, however, only two of the original requirements that determined the Shannon information have been considered, namely mo…
▽ More
Information decompositions quantify how the Shannon information about a given random variable is distributed among several other random variables. Various requirements have been proposed that such a decomposition should satisfy, leading to different candidate solutions. Curiously, however, only two of the original requirements that determined the Shannon information have been considered, namely monotonicity and normalization. Two other important properties, continuity and additivity, have not been considered. In this contribution, we focus on the mutual information of two finite variables $Y,Z$ about a third finite variable $S$ and check which of the decompositions satisfy these two properties. While most of them satisfy continuity, only one of them is both continuous and additive.
△ Less
Submitted 9 July, 2023; v1 submitted 22 April, 2022;
originally announced April 2022.
-
Grounding force-directed network layouts with latent space models
Authors:
Felix Gaisbauer,
Armin Pournaki,
Sven Banisch,
Eckehard Olbrich
Abstract:
Force-directed layout algorithms are ubiquitously-used tools for network visualisation across a multitude of scientific disciplines. However, they lack theoretical grounding which allows to interpret their outcomes rigorously and can guide the choice of specific algorithms for certain data sets. We propose an approach building on latent space models, which assume that the probability of nodes form…
▽ More
Force-directed layout algorithms are ubiquitously-used tools for network visualisation across a multitude of scientific disciplines. However, they lack theoretical grounding which allows to interpret their outcomes rigorously and can guide the choice of specific algorithms for certain data sets. We propose an approach building on latent space models, which assume that the probability of nodes forming a tie depends on their distance in an unobserved latent space. From such latent space models, we derive force equations for a force-directed layout algorithm. Since the forces infer positions which maximise the likelihood of the given network under the latent space model, the force-directed layout becomes interpretable. We implement these forces for unweighted and weighted networks and spatialise different real-world networks. Comparison to existing layout algorithms (not grounded in an interpretable model) reveals that node groups are placed in similar configurations, while said algorithms show a stronger intra-cluster separation of nodes, as well as a tendency to separate clusters more strongly in retweet networks. We also explore the possibility of visualising data traditionally not seen as network data, such as survey data.
△ Less
Submitted 1 February, 2022; v1 submitted 22 October, 2021;
originally announced October 2021.
-
The rise of populism and the reconfiguration of the German political space
Authors:
Eckehard Olbrich,
Sven Banisch
Abstract:
The paper explores the notion of a reconfiguration of political space in the context of the rise of populism and its effects on the political system. We focus on Germany and the appearance of the new right wing party "Alternative for Germany" (AfD). Many scholars of politics discuss the rise of the new populism in Western Europe and the US with respect to a new political cleavage related to global…
▽ More
The paper explores the notion of a reconfiguration of political space in the context of the rise of populism and its effects on the political system. We focus on Germany and the appearance of the new right wing party "Alternative for Germany" (AfD). Many scholars of politics discuss the rise of the new populism in Western Europe and the US with respect to a new political cleavage related to globalization, which is assumed to mainly affect the cultural dimension of the political space. As such, it might replace the older economic cleavage based on class divisions in defining the dominant dimension of political conflict. An explanation along these lines suggests a reconfiguration of the political space in the sense that (1) the main cleavage within the political space changes its direction from the economic axis towards the cultural axis, but (2) also the semantics of the cultural axis itself is changing towards globalization related topics. Using the electoral manifestos from the Manifesto project database, we empirically address this reconfiguration of the political space by comparing political spaces for Germany built using topic modeling with the spaces based on the content analysis of the Manifesto project and the corresponding categories of political goals. We find that both spaces have a similar structure and that the AfD appears on a new dimension. In order to characterize this new dimension we employ a novel technique, inter-issue consistency networks (IICN) that allow to analyze the evolution of the correlations between the political positions on different issues over several elections. We find that the new dimension introduced by the AfD can be related to the split off of a new "cultural right" issue bundle from the previously existing center-right bundle.
△ Less
Submitted 25 April, 2022; v1 submitted 29 June, 2021;
originally announced June 2021.
-
Ideological differences in engagement in public debate on Twitter
Authors:
Felix Gaisbauer,
Armin Pournaki,
Sven Banisch,
Eckehard Olbrich
Abstract:
This article analyses public debate on Twitter via network representations of retweets and replies. We argue that tweets observable on Twitter have both a direct and mediated effect on the perception of public opinion. Through the interplay of the two networks, it is possible to identify potentially misleading representations of public opinion on the platform. The method is employed to observe pub…
▽ More
This article analyses public debate on Twitter via network representations of retweets and replies. We argue that tweets observable on Twitter have both a direct and mediated effect on the perception of public opinion. Through the interplay of the two networks, it is possible to identify potentially misleading representations of public opinion on the platform. The method is employed to observe public debate about two events: The Saxon state elections and violent riots in the city of Leipzig in 2019. We show that in both cases, (i) different opinion groups exhibit different propensities to get involved in debate, and therefore have unequal impact on public opinion. Users retweeting far-right parties and politicians are significantly more active, hence their positions are disproportionately visible. (ii) Said users act significantly more confrontational in the sense that they reply mostly to users from different groups, while the contrary is not the case.
△ Less
Submitted 30 March, 2021; v1 submitted 3 September, 2020;
originally announced September 2020.
-
How social feedback processing in the brain shapes collective opinion processes in the era of social media
Authors:
Sven Banisch,
Felix Gaisbauer,
Eckehard Olbrich
Abstract:
What are the mechanisms by which groups with certain opinions gain public voice and force others holding a different view into silence? And how does social media play into this? Drawing on recent neuro-scientific insights into the processing of social feedback, we develop a theoretical model that allows to address these questions. The model captures phenomena described by spiral of silence theory…
▽ More
What are the mechanisms by which groups with certain opinions gain public voice and force others holding a different view into silence? And how does social media play into this? Drawing on recent neuro-scientific insights into the processing of social feedback, we develop a theoretical model that allows to address these questions. The model captures phenomena described by spiral of silence theory of public opinion, provides a mechanism-based foundation for it, and allows in this way more general insight into how different group structures relate to different regimes of collective opinion expression. Even strong majorities can be forced into silence if a minority acts as a cohesive whole. The proposed framework of social feedback theory (SFT) highlights the need for sociological theorising to understand the societal-level implications of findings in social and cognitive neuroscience.
△ Less
Submitted 18 March, 2020;
originally announced March 2020.
-
The Twitter Explorer: a Framework for Observing Twitter through Interactive Networks
Authors:
Armin Pournaki,
Felix Gaisbauer,
Sven Banisch,
Eckehard Olbrich
Abstract:
We present an open-source interface for scientists to explore Twitter data through interactive network visualizations. Combining data collection, transformation and visualization in one easily accessible framework, the twitter explorer connects distant and close reading of Twitter data through the interactive exploration of interaction networks and semantic networks. By lowering the technological…
▽ More
We present an open-source interface for scientists to explore Twitter data through interactive network visualizations. Combining data collection, transformation and visualization in one easily accessible framework, the twitter explorer connects distant and close reading of Twitter data through the interactive exploration of interaction networks and semantic networks. By lowering the technological barriers of data-driven research, it aims to attract researchers from various disciplinary backgrounds and facilitates new perspectives in the thriving field of computational social science.
△ Less
Submitted 7 April, 2021; v1 submitted 7 March, 2020;
originally announced March 2020.
-
The dynamics of opinion expression
Authors:
Felix Gaisbauer,
Eckehard Olbrich,
Sven Banisch
Abstract:
Modelling efforts in opinion dynamics have to a large extent ignored that opinion exchange between individuals can also have an effect on how willing they are to express their opinion publicly. Here, we introduce a model of public opinion expression. Two groups of agents with different opinion on an issue interact with each other, changing the willingness to express their opinion according to whet…
▽ More
Modelling efforts in opinion dynamics have to a large extent ignored that opinion exchange between individuals can also have an effect on how willing they are to express their opinion publicly. Here, we introduce a model of public opinion expression. Two groups of agents with different opinion on an issue interact with each other, changing the willingness to express their opinion according to whether they perceive themselves as part of the majority or minority opinion. We formulate the model as a multi-group majority game and investigate the Nash equilibria. We also provide a dynamical systems perspective: Using the reinforcement learning algorithm of $Q$-learning, we reduce the $N$-agent system in a mean-field approach to two dimensions which represent the two opinion groups. This two-dimensional system is analyzed in a comprehensive bifurcation analysis of its parameters. The model identifies social-structural conditions for public opinion predominance of different groups. Among other findings, we show under which circumstances a minority can dominate public discourse.
△ Less
Submitted 5 February, 2020; v1 submitted 29 December, 2019;
originally announced December 2019.
-
Unique Information and Secret Key Decompositions
Authors:
Johannes Rauh,
Pradeep Kr. Banerjee,
Eckehard Olbrich,
Jürgen Jost
Abstract:
The unique information ($UI$) is an information measure that quantifies a deviation from the Blackwell order. We have recently shown that this quantity is an upper bound on the one-way secret key rate. In this paper, we prove a triangle inequality for the $UI$, which implies that the $UI$ is never greater than one of the best known upper bounds on the two-way secret key rate. We conjecture that th…
▽ More
The unique information ($UI$) is an information measure that quantifies a deviation from the Blackwell order. We have recently shown that this quantity is an upper bound on the one-way secret key rate. In this paper, we prove a triangle inequality for the $UI$, which implies that the $UI$ is never greater than one of the best known upper bounds on the two-way secret key rate. We conjecture that the $UI$ lower bounds the two-way rate and discuss implications of the conjecture.
△ Less
Submitted 23 January, 2019;
originally announced January 2019.
-
An Argument Communication Model of Polarization and Ideological Alignment
Authors:
Sven Banisch,
Eckehard Olbrich
Abstract:
A multi-level model of opinion formation is presented which takes into account that attitudes on different issues are usually not independent. In the model, agents exchange beliefs regarding a series of facts. A cognitive structure of evaluative associations links different (partially overlap**) sets of facts to different political issues and determines an agents' attitudinal positions in a way…
▽ More
A multi-level model of opinion formation is presented which takes into account that attitudes on different issues are usually not independent. In the model, agents exchange beliefs regarding a series of facts. A cognitive structure of evaluative associations links different (partially overlap**) sets of facts to different political issues and determines an agents' attitudinal positions in a way borrowed from expectancy value theory. If agents preferentially interact with other agents that hold similar attitudes on one or several issues, this leads to biased argument pools and polarization in the sense that groups of agents selectively belief in distinct subsets of facts. Besides the emergence of a bi-modal distribution of opinions on single issues that most previous opinion polarization models address, our model also accounts for the alignment of attitudes across several issues along ideological dimensions.
△ Less
Submitted 3 December, 2019; v1 submitted 17 September, 2018;
originally announced September 2018.
-
Unique Informations and Deficiencies
Authors:
Pradeep Kr. Banerjee,
Eckehard Olbrich,
Jürgen Jost,
Johannes Rauh
Abstract:
Given two channels that convey information about the same random variable, we introduce two measures of the unique information of one channel with respect to the other. The two quantities are based on the notion of generalized weighted Le Cam deficiencies and differ on whether one channel can approximate the other by a randomization at either its input or output. We relate the proposed quantities…
▽ More
Given two channels that convey information about the same random variable, we introduce two measures of the unique information of one channel with respect to the other. The two quantities are based on the notion of generalized weighted Le Cam deficiencies and differ on whether one channel can approximate the other by a randomization at either its input or output. We relate the proposed quantities to an existing measure of unique information which we call the minimum-synergy unique information. We give an operational interpretation of the latter in terms of an upper bound on the one-way secret key rate and discuss the role of the unique informations in the context of nonnegative mutual information decompositions into unique, redundant and synergistic components.
△ Less
Submitted 9 December, 2019; v1 submitted 13 July, 2018;
originally announced July 2018.
-
Opinion Polarization by Learning from Social Feedback
Authors:
Sven Banisch,
Eckehard Olbrich
Abstract:
We explore a new mechanism to explain polarization phenomena in opinion dynamics in which agents evaluate alternative views on the basis of the social feedback obtained on expressing them. High support of the favored opinion in the social environment, is treated as a positive feedback which reinforces the value associated to this opinion. In connected networks of sufficiently high modularity, diff…
▽ More
We explore a new mechanism to explain polarization phenomena in opinion dynamics in which agents evaluate alternative views on the basis of the social feedback obtained on expressing them. High support of the favored opinion in the social environment, is treated as a positive feedback which reinforces the value associated to this opinion. In connected networks of sufficiently high modularity, different groups of agents can form strong convictions of competing opinions. Linking the social feedback process to standard equilibrium concepts we analytically characterize sufficient conditions for the stability of bi-polarization. While previous models have emphasized the polarization effects of deliberative argument-based communication, our model highlights an affective experience-based route to polarization, without assumptions about negative influence or bounded confidence.
△ Less
Submitted 19 October, 2018; v1 submitted 7 April, 2017;
originally announced April 2017.
-
The approach towards equilibrium in a reversible Ising dynamics model -- an information-theoretic analysis based on an exact solution
Authors:
Kristian Lindgren,
Eckehard Olbrich
Abstract:
We study the approach towards equilibrium in a dynamic Ising model, the Q2R cellular automaton, with microscopic reversibility and conserved energy for an infinite one-dimensional system. Starting from a low-entropy state with positive magnetisation, we investigate how the system approaches equilibrium characteristics given by statistical mechanics. We show that the magnetisation converges to zero…
▽ More
We study the approach towards equilibrium in a dynamic Ising model, the Q2R cellular automaton, with microscopic reversibility and conserved energy for an infinite one-dimensional system. Starting from a low-entropy state with positive magnetisation, we investigate how the system approaches equilibrium characteristics given by statistical mechanics. We show that the magnetisation converges to zero exponentially. The reversibility of the dynamics implies that the entropy density of the microstates is conserved in the time evolution. Still, it appears as if equilibrium, with a higher entropy density is approached. In order to understand this process, we solve the dynamics by formally proving how the information-theoretic characteristics of the microstates develop over time. With this approach we can show that an estimate of the entropy density based on finite length statistics within microstates converges to the equilibrium entropy density. The process behind this apparent entropy increase is a dissipation of correlation information over increasing distances. It is shown that the average information-theoretic correlation length increases linearly in time, being equivalent to a corresponding increase in excess entropy.
△ Less
Submitted 11 April, 2017; v1 submitted 30 March, 2017;
originally announced March 2017.
-
On extractable shared information
Authors:
Johannes Rauh,
Pradeep Kr. Banerjee,
Eckehard Olbrich,
Jürgen Jost,
Nils Bertschinger
Abstract:
We consider the problem of quantifying the information shared by a pair of random variables $X_{1},X_{2}$ about another variable $S$. We propose a new measure of shared information, called extractable shared information, that is left monotonic; that is, the information shared about $S$ is bounded from below by the information shared about $f(S)$ for any function $f$. We show that our measure leads…
▽ More
We consider the problem of quantifying the information shared by a pair of random variables $X_{1},X_{2}$ about another variable $S$. We propose a new measure of shared information, called extractable shared information, that is left monotonic; that is, the information shared about $S$ is bounded from below by the information shared about $f(S)$ for any function $f$. We show that our measure leads to a new nonnegative decomposition of the mutual information $I(S;X_1X_2)$ into shared, complementary and unique components. We study properties of this decomposition and show that a left monotonic shared information is not compatible with a Blackwell interpretation of unique information. We also discuss whether it is possible to have a decomposition in which both shared and unique information are left monotonic.
△ Less
Submitted 10 November, 2017; v1 submitted 26 January, 2017;
originally announced January 2017.
-
Coarse-graining and the Blackwell order
Authors:
Johannes Rauh,
Pradeep Kr. Banerjee,
Eckehard Olbrich,
Jürgen Jost,
Nils Bertschinger,
David Wolpert
Abstract:
Suppose we have a pair of information channels, $κ_{1},κ_{2}$, with a common input. The Blackwell order is a partial order over channels that compares $κ_{1}$ and $κ_{2}$ by the maximal expected utility an agent can obtain when decisions are based on the channel outputs. Equivalently, $κ_{1}$ is said to be Blackwell-inferior to $κ_{2}$ if and only if $κ_{1}$ can be constructed by garbling the outp…
▽ More
Suppose we have a pair of information channels, $κ_{1},κ_{2}$, with a common input. The Blackwell order is a partial order over channels that compares $κ_{1}$ and $κ_{2}$ by the maximal expected utility an agent can obtain when decisions are based on the channel outputs. Equivalently, $κ_{1}$ is said to be Blackwell-inferior to $κ_{2}$ if and only if $κ_{1}$ can be constructed by garbling the output of $κ_{2}$. A related partial order stipulates that $κ_{2}$ is more capable than $κ_{1}$ if the mutual information between the input and output is larger for $κ_{2}$ than for $κ_{1}$ for any distribution over inputs. A Blackwell-inferior channel is necessarily less capable. However, examples are known where $κ_{1}$ is less capable than $κ_{2}$ but not Blackwell-inferior. We show that this may even happen when $κ_{1}$ is constructed by coarse-graining the inputs of $κ_{2}$. Such a coarse-graining is a special kind of "pre-garbling" of the channel inputs. This example directly establishes that the expected value of the shared utility function for the coarse-grained channel is larger than it is for the non-coarse-grained channel. This contradicts the intuition that coarse-graining can only destroy information and lead to inferior channels. We also discuss our results in the context of information decompositions.
△ Less
Submitted 10 November, 2017; v1 submitted 26 January, 2017;
originally announced January 2017.
-
The Coconut Model with Heterogeneous Strategies and Learning
Authors:
Sven Banisch,
Eckehard Olbrich
Abstract:
In this paper, we develop an agent-based version of the Diamond search equilibrium model - also called Coconut Model. In this model, agents are faced with production decisions that have to be evaluated based on their expectations about the future utility of the produced entity which in turn depends on the global production level via a trading mechanism. While the original dynamical systems formula…
▽ More
In this paper, we develop an agent-based version of the Diamond search equilibrium model - also called Coconut Model. In this model, agents are faced with production decisions that have to be evaluated based on their expectations about the future utility of the produced entity which in turn depends on the global production level via a trading mechanism. While the original dynamical systems formulation assumes an infinite number of homogeneously adapting agents obeying strong rationality conditions, the agent-based setting allows to discuss the effects of heterogeneous and adaptive expectations and enables the analysis of non-equilibrium trajectories. Starting from a baseline implementation that matches the asymptotic behavior of the original model, we show how agent heterogeneity can be accounted for in the aggregate dynamical equations. We then show that when agents adapt their strategies by a simple temporal difference learning scheme, the system converges to one of the fixed points of the original system. Systematic simulations reveal that this is the only stable equilibrium solution.
△ Less
Submitted 1 December, 2016;
originally announced December 2016.
-
Perspectives on Multi-Level Dynamics
Authors:
Fatihcan M. Atay,
Sven Banisch,
Philippe Blanchard,
Bruno Cessac,
Eckehard Olbrich
Abstract:
As Physics did in previous centuries, there is currently a common dream of extracting generic laws of nature in economics, sociology, neuroscience, by focalising the description of phenomena to a minimal set of variables and parameters, linked together by causal equations of evolution whose structure may reveal hidden principles. This requires a huge reduction of dimensionality (number of degrees…
▽ More
As Physics did in previous centuries, there is currently a common dream of extracting generic laws of nature in economics, sociology, neuroscience, by focalising the description of phenomena to a minimal set of variables and parameters, linked together by causal equations of evolution whose structure may reveal hidden principles. This requires a huge reduction of dimensionality (number of degrees of freedom) and a change in the level of description. Beyond the mere necessity of develo** accurate techniques affording this reduction, there is the question of the correspondence between the initial system and the reduced one. In this paper, we offer a perspective towards a common framework for discussing and understanding multi-level systems exhibiting structures at various spatial and temporal levels. We propose a common foundation and illustrate it with examples from different fields. We also point out the difficulties in constructing such a general setting and its limitations.
△ Less
Submitted 14 July, 2016; v1 submitted 29 April, 2016;
originally announced June 2016.
-
Quantifying Emergent Behavior of Autonomous Robots
Authors:
Georg Martius,
Eckehard Olbrich
Abstract:
Quantifying behaviors of robots which were generated autonomously from task-independent objective functions is an important prerequisite for objective comparisons of algorithms and movements of animals. The temporal sequence of such a behavior can be considered as a time series and hence complexity measures developed for time series are natural candidates for its quantification. The predictive inf…
▽ More
Quantifying behaviors of robots which were generated autonomously from task-independent objective functions is an important prerequisite for objective comparisons of algorithms and movements of animals. The temporal sequence of such a behavior can be considered as a time series and hence complexity measures developed for time series are natural candidates for its quantification. The predictive information and the excess entropy are such complexity measures. They measure the amount of information the past contains about the future and thus quantify the nonrandom structure in the temporal sequence. However, when using these measures for systems with continuous states one has to deal with the fact that their values will depend on the resolution with which the systems states are observed. For deterministic systems both measures will diverge with increasing resolution. We therefore propose a new decomposition of the excess entropy in resolution dependent and resolution independent parts and discuss how they depend on the dimensionality of the dynamics, correlations and the noise level. For the practical estimation we propose to use estimates based on the correlation integral instead of the direct estimation of the mutual information using the algorithm by Kraskov et al. (2004) which is based on next neighbor statistics because the latter allows less control of the scale dependencies. Using our algorithm we are able to show how autonomous learning generates behavior of increasing complexity with increasing learning duration.
△ Less
Submitted 6 October, 2015;
originally announced October 2015.
-
The Information Theory of Individuality
Authors:
David Krakauer,
Nils Bertschinger,
Eckehard Olbrich,
Nihat Ay,
Jessica C. Flack
Abstract:
We consider biological individuality in terms of information theoretic and graphical principles. Our purpose is to extract through an algorithmic decomposition system-environment boundaries supporting individuality. We infer or detect evolved individuals rather than assume that they exist. Given a set of consistent measurements over time, we discover a coarse-grained or quantized description on a…
▽ More
We consider biological individuality in terms of information theoretic and graphical principles. Our purpose is to extract through an algorithmic decomposition system-environment boundaries supporting individuality. We infer or detect evolved individuals rather than assume that they exist. Given a set of consistent measurements over time, we discover a coarse-grained or quantized description on a system, inducing partitions (which can be nested). Legitimate individual partitions will propagate information from the past into the future, whereas spurious aggregations will not. Individuals are therefore defined in terms of ongoing, bounded information processing units rather than lists of static features or conventional replication-based definitions which tend to fail in the case of cultural change. One virtue of this approach is that it could expand the scope of what we consider adaptive or biological phenomena, particularly in the microscopic and macroscopic regimes of molecular and social phenomena.
△ Less
Submitted 7 December, 2014;
originally announced December 2014.
-
Reconsidering unique information: Towards a multivariate information decomposition
Authors:
Johannes Rauh,
Nils Bertschinger,
Eckehard Olbrich,
Jürgen Jost
Abstract:
The information that two random variables $Y$, $Z$ contain about a third random variable $X$ can have aspects of shared information (contained in both $Y$ and $Z$), of complementary information (only available from $(Y,Z)$ together) and of unique information (contained exclusively in either $Y$ or $Z$). Here, we study measures $\widetilde{SI}$ of shared, $\widetilde{UI}$ unique and…
▽ More
The information that two random variables $Y$, $Z$ contain about a third random variable $X$ can have aspects of shared information (contained in both $Y$ and $Z$), of complementary information (only available from $(Y,Z)$ together) and of unique information (contained exclusively in either $Y$ or $Z$). Here, we study measures $\widetilde{SI}$ of shared, $\widetilde{UI}$ unique and $\widetilde{CI}$ complementary information introduced by Bertschinger et al., which are motivated from a decision theoretic perspective. We find that in most cases the intuitive rule that more variables contain more information applies, with the exception that $\widetilde{SI}$ and $\widetilde{CI}$ information are not monotone in the target variable $X$. Additionally, we show that it is not possible to extend the bivariate information decomposition into $\widetilde{SI}$, $\widetilde{UI}$ and $\widetilde{CI}$ to a non-negative decomposition on the partial information lattice of Williams and Beer. Nevertheless, the quantities $\widetilde{UI}$, $\widetilde{SI}$ and $\widetilde{CI}$ have a well-defined interpretation, even in the multivariate setting.
△ Less
Submitted 11 April, 2014;
originally announced April 2014.
-
Value of information in noncooperative games
Authors:
Nils Bertschinger,
David H. Wolpert,
Eckehard Olbrich,
Juergen Jost
Abstract:
In some games, additional information hurts a player, e.g., in games with first-mover advantage, the second-mover is hurt by seeing the first-mover's move. What properties of a game determine whether it has such negative "value of information" for a particular player? Can a game have negative value of information for all players? To answer such questions, we generalize the definition of marginal u…
▽ More
In some games, additional information hurts a player, e.g., in games with first-mover advantage, the second-mover is hurt by seeing the first-mover's move. What properties of a game determine whether it has such negative "value of information" for a particular player? Can a game have negative value of information for all players? To answer such questions, we generalize the definition of marginal utility of a good to define the marginal utility of a parameter vector specifying a game. So rather than analyze the global structure of the relationship between a game's parameter vector and player behavior, as in previous work, we focus on the local structure of that relationship. This allows us to prove that generically, every game can have negative marginal value of information, unless one imposes a priori constraints on allowed changes to the game's parameter vector. We demonstrate these and related results numerically, and discuss their implications.
△ Less
Submitted 30 January, 2015; v1 submitted 27 December, 2013;
originally announced January 2014.
-
Quantifying unique information
Authors:
Nils Bertschinger,
Johannes Rauh,
Eckehard Olbrich,
Jürgen Jost,
Nihat Ay
Abstract:
We propose new measures of shared information, unique information and synergistic information that can be used to decompose the multi-information of a pair of random variables $(Y,Z)$ with a third random variable $X$. Our measures are motivated by an operational idea of unique information which suggests that shared information and unique information should depend only on the pair marginal distribu…
▽ More
We propose new measures of shared information, unique information and synergistic information that can be used to decompose the multi-information of a pair of random variables $(Y,Z)$ with a third random variable $X$. Our measures are motivated by an operational idea of unique information which suggests that shared information and unique information should depend only on the pair marginal distributions of $(X,Y)$ and $(X,Z)$. Although this invariance property has not been studied before, it is satisfied by other proposed measures of shared information. The invariance property does not uniquely determine our new measures, but it implies that the functions that we define are bounds to any other measures satisfying the same invariance property. We study properties of our measures and compare them to other candidate measures.
△ Less
Submitted 15 January, 2014; v1 submitted 12 November, 2013;
originally announced November 2013.
-
Shared Information -- New Insights and Problems in Decomposing Information in Complex Systems
Authors:
Nils Bertschinger,
Johannes Rauh,
Eckehard Olbrich,
Jürgen Jost
Abstract:
How can the information that a set ${X_{1},...,X_{n}}$ of random variables contains about another random variable $S$ be decomposed? To what extent do different subgroups provide the same, i.e. shared or redundant, information, carry unique information or interact for the emergence of synergistic information?
Recently Williams and Beer proposed such a decomposition based on natural properties fo…
▽ More
How can the information that a set ${X_{1},...,X_{n}}$ of random variables contains about another random variable $S$ be decomposed? To what extent do different subgroups provide the same, i.e. shared or redundant, information, carry unique information or interact for the emergence of synergistic information?
Recently Williams and Beer proposed such a decomposition based on natural properties for shared information. While these properties fix the structure of the decomposition, they do not uniquely specify the values of the different terms. Therefore, we investigate additional properties such as strong symmetry and left monotonicity. We find that strong symmetry is incompatible with the properties proposed by Williams and Beer. Although left monotonicity is a very natural property for an information measure it is not fulfilled by any of the proposed measures.
We also study a geometric framework for information decompositions and ask whether it is possible to represent shared information by a family of posterior distributions.
Finally, we draw connections to the notions of shared knowledge and common knowledge in game theory. While many people believe that independent variables cannot share information, we show that in game theory independent agents can have shared knowledge, but not common knowledge. We conclude that intuition and heuristic arguments do not suffice when arguing about information.
△ Less
Submitted 22 October, 2012;
originally announced October 2012.
-
Hysteresis effects of changing parameters of noncooperative games
Authors:
David H. Wolpert,
Michael Harre,
Eckehard Olbrich,
Nils Bertschinger,
Juergen Jost
Abstract:
We adapt the method used by Jaynes to derive the equilibria of statistical physics to instead derive equilibria of bounded rational game theory. We analyze the dependence of these equilibria on the parameters of the underlying game, focusing on hysteresis effects. In particular, we show that by gradually imposing individual-specific tax rates on the players of the game, and then gradually removing…
▽ More
We adapt the method used by Jaynes to derive the equilibria of statistical physics to instead derive equilibria of bounded rational game theory. We analyze the dependence of these equilibria on the parameters of the underlying game, focusing on hysteresis effects. In particular, we show that by gradually imposing individual-specific tax rates on the players of the game, and then gradually removing those taxes, the players move from a poor equilibrium to one that is better for all of them.
△ Less
Submitted 27 October, 2010;
originally announced October 2010.
-
Knockouts, Robustness and Cell Cycles
Authors:
Gunnar Boldhaus,
Nils Bertschinger,
Johannes Rauh,
Eckehard Olbrich,
Konstantin Klemm
Abstract:
The response to a knockout of a node is a characteristic feature of a networked dynamical system. Knockout resilience in the dynamics of the remaining nodes is a sign of robustness. Here we study the effect of knockouts for binary state sequences and their implementations in terms of Boolean threshold networks. Beside random sequences with biologically plausible constraints, we analyze the cell…
▽ More
The response to a knockout of a node is a characteristic feature of a networked dynamical system. Knockout resilience in the dynamics of the remaining nodes is a sign of robustness. Here we study the effect of knockouts for binary state sequences and their implementations in terms of Boolean threshold networks. Beside random sequences with biologically plausible constraints, we analyze the cell cycle sequence of the species Saccharomyces cerevisiae and the Boolean networks implementing it. Comparing with an appropriate null model we do not find evidence that the yeast wildtype network is optimized for high knockout resilience. Our notion of knockout resilience weakly correlates with the size of the basin of attraction, which has also been considered a measure of robustness.
△ Less
Submitted 27 February, 2010;
originally announced March 2010.
-
Quantifying structure in networks
Authors:
Eckehard Olbrich,
Thomas Kahle,
Nils Bertschinger,
Nihat Ay,
Juergen Jost
Abstract:
We investigate exponential families of random graph distributions as a framework for systematic quantification of structure in networks. In this paper we restrict ourselves to undirected unlabeled graphs. For these graphs, the counts of subgraphs with no more than k links are a sufficient statistics for the exponential families of graphs with interactions between at most k links. In this framewo…
▽ More
We investigate exponential families of random graph distributions as a framework for systematic quantification of structure in networks. In this paper we restrict ourselves to undirected unlabeled graphs. For these graphs, the counts of subgraphs with no more than k links are a sufficient statistics for the exponential families of graphs with interactions between at most k links. In this framework we investigate the dependencies between several observables commonly used to quantify structure in networks, such as the degree distribution, cluster and assortativity coefficients.
△ Less
Submitted 22 December, 2009;
originally announced December 2009.
-
Complexity Measures from Interaction Structures
Authors:
Thomas Kahle,
Eckehard Olbrich,
Juergen Jost,
Nihat Ay
Abstract:
We evaluate new complexity measures on the symbolic dynamics of coupled tent maps and cellular automata. These measures quantify complexity in terms of $k$-th order statistical dependencies that cannot be reduced to interactions between $k-1$ units. We demonstrate that these measures are able to identify complex dynamical regimes.
We evaluate new complexity measures on the symbolic dynamics of coupled tent maps and cellular automata. These measures quantify complexity in terms of $k$-th order statistical dependencies that cannot be reduced to interactions between $k-1$ units. We demonstrate that these measures are able to identify complex dynamical regimes.
△ Less
Submitted 26 November, 2008; v1 submitted 16 June, 2008;
originally announced June 2008.
-
Chaos or Noise - Difficulties of a Distinction
Authors:
M. Cencini,
M. Falcioni,
H. Kantz,
E. Olbrich,
A. Vulpiani
Abstract:
In experiments, the dynamical behavior of systems is reflected in time series. Due to the finiteness of the observational data set it is not possible to reconstruct the invariant measure up to arbitrary fine resolution and arbitrary high embedding dimension. These restrictions limit our ability to distinguish between signals generated by different systems, such as regular, chaotic or stochastic…
▽ More
In experiments, the dynamical behavior of systems is reflected in time series. Due to the finiteness of the observational data set it is not possible to reconstruct the invariant measure up to arbitrary fine resolution and arbitrary high embedding dimension. These restrictions limit our ability to distinguish between signals generated by different systems, such as regular, chaotic or stochastic ones, when analyzed from a time series point of view. We propose to classify the signal behavior, without referring to any specific model, as stochastic or deterministic on a certain scale of the resolution $ε$, according to the dependence of the $(ε,τ)$-entropy, $h(ε, τ)$, and of the finite size Lyapunov exponent, $λ(ε)$, on $ε$.
△ Less
Submitted 20 March, 2000; v1 submitted 14 February, 2000;
originally announced February 2000.
-
Local estimates for entropy densities in coupled map lattices
Authors:
Eckehard Olbrich,
Rainer Hegger,
Holger Kantz
Abstract:
We present a method to derive an upper bound for the entropy density of coupled map lattices with local interactions from local observations. To do this, we use an embedding technique being a combination of time delay and spatial embedding. This embedding allows us to identify the local character of the equations of motion. Based on this method we present an approximate estimate of the entropy d…
▽ More
We present a method to derive an upper bound for the entropy density of coupled map lattices with local interactions from local observations. To do this, we use an embedding technique being a combination of time delay and spatial embedding. This embedding allows us to identify the local character of the equations of motion. Based on this method we present an approximate estimate of the entropy density by the correlation integral.
△ Less
Submitted 6 January, 2000;
originally announced January 2000.
-
Manifestation of classical bifurcation in the spectrum of the integrable quantum dimer
Authors:
S. Aubry,
S. Flach,
K. Kladko,
E. Olbrich
Abstract:
We analyze the classical and quantum properties of the integrable dimer problem. The classical version exhibits exactly one bifurcation in phase space, which gives birth to permutational symmetry broken trajectories and a separatrix. The quantum analysis yields all tunneling rates (splittings) in leading order of perturbation. In the semiclassical regime the eigenvalue spectrum obtained by numer…
▽ More
We analyze the classical and quantum properties of the integrable dimer problem. The classical version exhibits exactly one bifurcation in phase space, which gives birth to permutational symmetry broken trajectories and a separatrix. The quantum analysis yields all tunneling rates (splittings) in leading order of perturbation. In the semiclassical regime the eigenvalue spectrum obtained by numerically exact diagonalization allows to conclude about the presence of a separatrix and a bifurcation in the corresponding classical model.
△ Less
Submitted 29 August, 1998;
originally announced August 1998.
