-
Unique Information and Secret Key Agreement
Authors:
Ryan G. James,
Jeffrey Emenheiser,
James P. Crutchfield
Abstract:
The partial information decomposition (PID) is a promising framework for decomposing a joint random variable into the amount of influence each source variable Xi has on a target variable Y, relative to the other sources. For two sources, influence breaks down into the information that both X0 and X1 redundantly share with Y, what X0 uniquely shares with Y, what X1 uniquely shares with Y, and final…
▽ More
The partial information decomposition (PID) is a promising framework for decomposing a joint random variable into the amount of influence each source variable Xi has on a target variable Y, relative to the other sources. For two sources, influence breaks down into the information that both X0 and X1 redundantly share with Y, what X0 uniquely shares with Y, what X1 uniquely shares with Y, and finally what X0 and X1 synergistically share with Y. Unfortunately, considerable disagreement has arisen as to how these four components should be quantified. Drawing from cryptography, we consider the secret key agreement rate as an operational method of quantifying unique informations. Secret key agreement rate comes in several forms, depending upon which parties are permitted to communicate. We demonstrate that three of these four forms are inconsistent with the PID. The remaining form implies certain interpretations as to the PID's meaning---interpretations not present in PID's definition but that, we argue, need to be explicit. These reveal an inconsistency between third-order connected information, two-way secret key agreement rate, and synergy. Similar difficulties arise with a popular PID measure in light the results here as well as from a maximum entropy viewpoint. We close by reviewing the challenges facing the PID.
△ Less
Submitted 1 November, 2018;
originally announced November 2018.
-
A Perspective on Unique Information: Directionality, Intuitions, and Secret Key Agreement
Authors:
Ryan G. James,
Jeffrey Emenheiser,
James P. Crutchfield
Abstract:
Recently, the partial information decomposition emerged as a promising framework for identifying the meaningful components of the information contained in a joint distribution. Its adoption and practical application, however, have been stymied by the lack of a generally-accepted method of quantifying its components. Here, we briefly discuss the bivariate (two-source) partial information decomposit…
▽ More
Recently, the partial information decomposition emerged as a promising framework for identifying the meaningful components of the information contained in a joint distribution. Its adoption and practical application, however, have been stymied by the lack of a generally-accepted method of quantifying its components. Here, we briefly discuss the bivariate (two-source) partial information decomposition and two implicitly directional interpretations used to intuitively motivate alternative component definitions. Drawing parallels with secret key agreement rates from information-theoretic cryptography, we demonstrate that these intuitions are mutually incompatible and suggest that this underlies the persistence of competing definitions and interpretations. Having highlighted this hitherto unacknowledged issue, we outline several possible solutions.
△ Less
Submitted 26 August, 2018;
originally announced August 2018.
-
Unique Information via Dependency Constraints
Authors:
Ryan G. James,
Jeffrey Emenheiser,
James P. Crutchfield
Abstract:
The partial information decomposition (PID) is perhaps the leading proposal for resolving information shared between a set of sources and a target into redundant, synergistic, and unique constituents. Unfortunately, the PID framework has been hindered by a lack of a generally agreed-upon, multivariate method of quantifying the constituents. Here, we take a step toward rectifying this by develo**…
▽ More
The partial information decomposition (PID) is perhaps the leading proposal for resolving information shared between a set of sources and a target into redundant, synergistic, and unique constituents. Unfortunately, the PID framework has been hindered by a lack of a generally agreed-upon, multivariate method of quantifying the constituents. Here, we take a step toward rectifying this by develo** a decomposition based on a new method that quantifies unique information. We first develop a broadly applicable method---the dependency decomposition---that delineates how statistical dependencies influence the structure of a joint distribution. The dependency decomposition then allows us to define a measure of the information about a target that can be uniquely attributed to a particular source as the least amount which the source-target statistical dependency can influence the information shared between the sources and the target. The result is the first measure that satisfies the core axioms of the PID framework while not satisfying the Blackwell relation, which depends on a particular interpretation of how the variables are related. This makes a key step forward to a practical PID.
△ Less
Submitted 27 October, 2018; v1 submitted 19 September, 2017;
originally announced September 2017.
-
Evolution of opinions on social networks in the presence of competing committed groups
Authors:
J. Xie,
J. Emenheiser,
M. Kirby,
S. Sreenivasan,
B. K. Szymanski,
G. Korniss
Abstract:
Public opinion is often affected by the presence of committed groups of individuals dedicated to competing points of view. Using a model of pairwise social influence, we study how the presence of such groups within social networks affects the outcome and the speed of evolution of the overall opinion on the network. Earlier work indicated that a single committed group within a dense social network…
▽ More
Public opinion is often affected by the presence of committed groups of individuals dedicated to competing points of view. Using a model of pairwise social influence, we study how the presence of such groups within social networks affects the outcome and the speed of evolution of the overall opinion on the network. Earlier work indicated that a single committed group within a dense social network can cause the entire network to quickly adopt the group's opinion (in times scaling logarithmically with the network size), so long as the committed group constitutes more than about 10% of the population (with the findings being qualitatively similar for sparse networks as well). Here we study the more general case of opinion evolution when two groups committed to distinct, competing opinions $A$ and $B$, and constituting fractions $p_A$ and $p_B$ of the total population respectively, are present in the network. We show for stylized social networks (including Erdős-Rényi random graphs and Barabási-Albert scale-free networks) that the phase diagram of this system in parameter space $(p_A,p_B)$ consists of two regions, one where two stable steady-states coexist, and the remaining where only a single stable steady-state exists. These two regions are separated by two fold-bifurcation (spinodal) lines which meet tangentially and terminate at a cusp (critical point). We provide further insights to the phase diagram and to the nature of the underlying phase transitions by investigating the model on infinite (mean-field limit), finite complete graphs and finite sparse networks. For the latter case, we also derive the scaling exponent associated with the exponential growth of switching times as a function of the distance from the critical point.
△ Less
Submitted 22 April, 2012; v1 submitted 29 December, 2011;
originally announced December 2011.
