Showing 1–16 of 16 results for author: Krakauer, D

The Debate Over Understanding in AI's Large Language Models

Authors: Melanie Mitchell, David C. Krakauer

Abstract: We survey a current, heated debate in the AI research community on whether large pre-trained language models can be said to "understand" language -- and the physical and social situations language encodes -- in any important sense. We describe arguments that have been made for and against such understanding, and key questions for the broader sciences of intelligence that have arisen in light of th… ▽ More We survey a current, heated debate in the AI research community on whether large pre-trained language models can be said to "understand" language -- and the physical and social situations language encodes -- in any important sense. We describe arguments that have been made for and against such understanding, and key questions for the broader sciences of intelligence that have arisen in light of these arguments. We contend that a new science of intelligence can be developed that will provide insight into distinct modes of understanding, their strengths and limitations, and the challenge of integrating diverse forms of cognition. △ Less

Submitted 10 February, 2023; v1 submitted 14 October, 2022; originally announced October 2022.

Comments: Under submission as a Perspective article. Updated with additional discussion and citations

Journal ref: Proceedings of the National Academy of Sciences 120 (13), 2023

Outsourcing Memory Through Niche Construction

Authors: Edward D. Lee, Jessica C. Flack, David C. Krakauer

Abstract: Adaptation to changing environments is a universal feature of life and can involve the organism modifying itself in response to the environment as well as actively modifying the environment to control selection pressures. The latter case couples the organism to environment. Then, how quickly should the organism change in response to the environment? We formulate this question in terms of how memor… ▽ More Adaptation to changing environments is a universal feature of life and can involve the organism modifying itself in response to the environment as well as actively modifying the environment to control selection pressures. The latter case couples the organism to environment. Then, how quickly should the organism change in response to the environment? We formulate this question in terms of how memory duration scales with environmental rate of change when there are trade-offs in remembering vs. forgetting. We derive a universal scaling law for optimal memory duration, taking into account memory precision as well as two components of environmental volatility, bias and stability. We find sublinear scaling with any amount of environmental volatility. We use a memory complexity measure to explore the strategic conditions (game dynamics) favoring actively reducing environmental volatility -- outsourcing memory through niche construction -- over investing in neural tissue. We predict stabilizing niche construction will evolve when neural tissue is costly, the environment is variable, and it is beneficial to be able to encode a rich repertoire of environmental states. △ Less

Submitted 7 January, 2023; v1 submitted 1 September, 2022; originally announced September 2022.

Simulation Intelligence: Towards a New Generation of Scientific Methods

Authors: Alexander Lavin, David Krakauer, Hector Zenil, Justin Gottschlich, Tim Mattson, Johann Brehmer, Anima Anandkumar, Sanjay Choudry, Kamil Rocki, Atılım Güneş Baydin, Carina Prunkl, Brooks Paige, Olexandr Isayev, Erik Peterson, Peter L. McMahon, Jakob Macke, Kyle Cranmer, Jiaxin Zhang, Haruko Wainwright, Adi Hanuka, Manuela Veloso, Samuel Assefa, Stephan Zheng, Avi Pfeffer

Abstract: The original "Seven Motifs" set forth a roadmap of essential methods for the field of scientific computing, where a motif is an algorithmic method that captures a pattern of computation and data movement. We present the "Nine Motifs of Simulation Intelligence", a roadmap for the development and integration of the essential algorithms necessary for a merger of scientific computing, scientific simul… ▽ More The original "Seven Motifs" set forth a roadmap of essential methods for the field of scientific computing, where a motif is an algorithmic method that captures a pattern of computation and data movement. We present the "Nine Motifs of Simulation Intelligence", a roadmap for the development and integration of the essential algorithms necessary for a merger of scientific computing, scientific simulation, and artificial intelligence. We call this merger simulation intelligence (SI), for short. We argue the motifs of simulation intelligence are interconnected and interdependent, much like the components within the layers of an operating system. Using this metaphor, we explore the nature of each layer of the simulation intelligence operating system stack (SI-stack) and the motifs therein: (1) Multi-physics and multi-scale modeling; (2) Surrogate modeling and emulation; (3) Simulation-based inference; (4) Causal modeling and inference; (5) Agent-based modeling; (6) Probabilistic programming; (7) Differentiable programming; (8) Open-ended optimization; (9) Machine programming. We believe coordinated efforts between motifs offers immense opportunity to accelerate scientific discovery, from solving inverse problems in synthetic biology and climate science, to directing nuclear energy experiments and predicting emergent behavior in socioeconomic settings. We elaborate on each layer of the SI-stack, detailing the state-of-art methods, presenting examples to highlight challenges and opportunities, and advocating for specific ways to advance the motifs and the synergies from their combinations. Advancing and integrating these technologies can enable a robust and efficient hypothesis-simulation-analysis type of scientific method, which we introduce with several use-cases for human-machine teaming and automated science. △ Less

Submitted 27 November, 2022; v1 submitted 6 December, 2021; originally announced December 2021.

A scaling theory of armed conflict avalanches

Authors: Edward D. Lee, Bryan C. Daniels, Christopher R. Myers, David C. Krakauer, Jessica C. Flack

Abstract: Armed conflict data display scaling and universal dynamics in both social and physical properties like fatalities and geographic extent. We propose a randomly branching, armed-conflict model that relates multiple properties to one another in a way consistent with data. The model incorporates a fractal lattice on which conflict spreads, uniform dynamics driving conflict growth, and regional virulen… ▽ More Armed conflict data display scaling and universal dynamics in both social and physical properties like fatalities and geographic extent. We propose a randomly branching, armed-conflict model that relates multiple properties to one another in a way consistent with data. The model incorporates a fractal lattice on which conflict spreads, uniform dynamics driving conflict growth, and regional virulence that modulates local conflict intensity. The quantitative constraints on scaling and universal dynamics we use to develop our minimal model serve more generally as a set of constraints for other models for armed conflict dynamics. We show how this approach akin to thermodynamics imparts mechanistic intuition and unifies multiple conflict properties, giving insight into causation, prediction, and intervention timing. △ Less

Submitted 29 April, 2020; originally announced April 2020.

Journal ref: Phys. Rev. E 102, 042312 (2020)

Emergent regularities and scaling in armed conflict data

Authors: Edward D. Lee, Bryan C. Daniels, Christopher R. Myers, David C. Krakauer, Jessica C. Flack

Abstract: Armed conflict exhibits regularities beyond known power law distributions of fatalities and duration over varying culture and geography. We systematically cluster conflict reports from a database of $10^5$ events from Africa spanning 20 years into conflict avalanches. Conflict profiles collapse over a range of scales. Duration, diameter, extent, fatalities, and report totals satisfy mutually consi… ▽ More Armed conflict exhibits regularities beyond known power law distributions of fatalities and duration over varying culture and geography. We systematically cluster conflict reports from a database of $10^5$ events from Africa spanning 20 years into conflict avalanches. Conflict profiles collapse over a range of scales. Duration, diameter, extent, fatalities, and report totals satisfy mutually consistent scaling relations captured with a model combining geographic spread and local conflict-site growth. The emergence of such social scaling laws hints at principles guiding conflict evolution. △ Less

Submitted 29 April, 2020; v1 submitted 18 March, 2019; originally announced March 2019.

The Cultural Evolution of National Constitutions

Authors: Daniel N. Rockmore, Chen Fang, Nicholas J. Foti, Tom Ginsburg, David C. Krakauer

Abstract: We explore how ideas from infectious disease and genetics can be used to uncover patterns of cultural inheritance and innovation in a corpus of 591 national constitutions spanning 1789 - 2008. Legal "Ideas" are encoded as "topics" - words statistically linked in documents - derived from topic modeling the corpus of constitutions. Using these topics we derive a diffusion network for borrowing from… ▽ More We explore how ideas from infectious disease and genetics can be used to uncover patterns of cultural inheritance and innovation in a corpus of 591 national constitutions spanning 1789 - 2008. Legal "Ideas" are encoded as "topics" - words statistically linked in documents - derived from topic modeling the corpus of constitutions. Using these topics we derive a diffusion network for borrowing from ancestral constitutions back to the US Constitution of 1789 and reveal that constitutions are complex cultural recombinants. We find systematic variation in patterns of borrowing from ancestral texts and "biological"-like behavior in patterns of inheritance with the distribution of "offspring" arising through a bounded preferential-attachment process. This process leads to a small number of highly innovative (influential) constitutions some of which have yet to have been identified as so in the current literature. Our findings thus shed new light on the critical nodes of the constitution-making network. The constitutional network structure reflects periods of intense constitution creation, and systematic patterns of variation in constitutional life-span and temporal influence. △ Less

Submitted 18 November, 2017; originally announced November 2017.

Comments: 38 pages with supplemental information, 13 figures, 2 tables; Accepted for publication the Journal of the Association for Information Science and Technology

MSC Class: 68.U99 ACM Class: J.4

Control of critical behavior in a small-scale social system

Authors: Bryan C. Daniels, David C. Krakauer, Jessica C. Flack

Abstract: Over the last decade new technologies for making large numbers of fine-grained measurements have led to the surprising discovery that many biological systems sit near a critical point. These systems are potentially more adaptive in that small changes to component behavior can induce large-scale changes in aggregate structure and function. Accounting for criticality remains a challenge as sensitivi… ▽ More Over the last decade new technologies for making large numbers of fine-grained measurements have led to the surprising discovery that many biological systems sit near a critical point. These systems are potentially more adaptive in that small changes to component behavior can induce large-scale changes in aggregate structure and function. Accounting for criticality remains a challenge as sensitivity to perturbation suggests a lack of robustness. Furthermore, change induced by perturbation may not be adaptive. Complicating matters further critical phenomena can result from history-dependent stochastic processes. A question central to distinguishing among these conflicting views of criticality is to what degree criticality can be controlled by the components of the system. We address the control of criticality using data on conflict dynamics and fight sizes from an animal society model system (Macaca nemestrina, n=48). The system is fundamentally finite so we operationalize criticality in information theoretic terms using Fisher information and a measure of instability. We analyze criticality using empirically-grounded equilibrium (maximum entropy) and dynamic (branching process) models of the monkeys' fight-joining behavior. We find that (1) this heterogeneous, socially organized system, like homogeneous, spatial systems (flocks and schools), sits near a critical point, (2) the contributions individuals make to how critical the system is can be quantified and vary, and (3) the distance from the critical point (DFC) can be controlled through biologically plausible mechanisms operating on this heterogeneity. These mechanisms include third-party policing, which dampens fight participation of the individuals with the largest effect on DFC. Control of DFC allows biological systems to balance the tradeoff between robustness and need for rapid change. △ Less

Submitted 13 January, 2016; originally announced January 2016.

Comments: 34 pages, 8 figures

Cryptographic Nature

Authors: David Krakauer

Abstract: I consider the many ways in which evolved information-flows are restricted and metabolic resources protected and hidden -- the thesis of living phenomena as evolutionary cryptosystems. I present the information theory of secrecy systems and discuss mechanisms acquired by evolved lineages that encrypt sensitive heritable information with random keys. I explore the idea that complexity science is a… ▽ More I consider the many ways in which evolved information-flows are restricted and metabolic resources protected and hidden -- the thesis of living phenomena as evolutionary cryptosystems. I present the information theory of secrecy systems and discuss mechanisms acquired by evolved lineages that encrypt sensitive heritable information with random keys. I explore the idea that complexity science is a cryptographic discipline as "frozen accidents", or various forms of regularized randomness, historically encrypt adaptive dynamics. △ Less

Submitted 7 May, 2015; originally announced May 2015.

The Information Theory of Individuality

Authors: David Krakauer, Nils Bertschinger, Eckehard Olbrich, Nihat Ay, Jessica C. Flack

Abstract: We consider biological individuality in terms of information theoretic and graphical principles. Our purpose is to extract through an algorithmic decomposition system-environment boundaries supporting individuality. We infer or detect evolved individuals rather than assume that they exist. Given a set of consistent measurements over time, we discover a coarse-grained or quantized description on a… ▽ More We consider biological individuality in terms of information theoretic and graphical principles. Our purpose is to extract through an algorithmic decomposition system-environment boundaries supporting individuality. We infer or detect evolved individuals rather than assume that they exist. Given a set of consistent measurements over time, we discover a coarse-grained or quantized description on a system, inducing partitions (which can be nested). Legitimate individual partitions will propagate information from the past into the future, whereas spurious aggregations will not. Individuals are therefore defined in terms of ongoing, bounded information processing units rather than lists of static features or conventional replication-based definitions which tend to fail in the case of cultural change. One virtue of this approach is that it could expand the scope of what we consider adaptive or biological phenomena, particularly in the microscopic and macroscopic regimes of molecular and social phenomena. △ Less

Submitted 7 December, 2014; originally announced December 2014.

Capturing collective conflict dynamics with sparse social circuits

Authors: Edward Lee, Bryan Daniels, Jessica Flack, David Krakauer

Abstract: We discuss a set of computational techniques, called Inductive Game Theory, for extracting strategic decision-making rules from time series data and constructing probabilistic social circuits. We construct these circuits by connecting component individuals and groups with strategies in a game and propose an inductive approach to reconstructing the edges. We demonstrate this approach with conflict… ▽ More We discuss a set of computational techniques, called Inductive Game Theory, for extracting strategic decision-making rules from time series data and constructing probabilistic social circuits. We construct these circuits by connecting component individuals and groups with strategies in a game and propose an inductive approach to reconstructing the edges. We demonstrate this approach with conflict behavior in a society of pigtailed macaques by identifying significant patterns in decision-making by individuals. With the constructed circuit, we then capture macroscopic features of the system that were not specified in the construction of the initial circuit, providing a map** between individual level behaviors to collective behaviors over the scale of the group. We extend on previous work in Inductive Game Theory by more efficiently searching the space of possible strategies by grou** individuals into socially relevant sets to produce a more efficient, parsimonious specification of the underlying interactions between components. We discuss how we reduce the dimensionality of these circuits using coarse-graining or compression to build cognitive effective theories for collective behavior. △ Less

Submitted 30 June, 2014; originally announced June 2014.

Report number: ci-2014/108

Dynamics and Processing in Finite Self-Similar Networks

Authors: Simon DeDeo, David C. Krakauer

Abstract: A common feature of biological networks is the geometric property of self-similarity. Molecular regulatory networks through to circulatory systems, nervous systems, social systems and ecological trophic networks, show self-similar connectivity at multiple scales. We analyze the relationship between topology and signaling in contrasting classes of such topologies. We find that networks differ in th… ▽ More A common feature of biological networks is the geometric property of self-similarity. Molecular regulatory networks through to circulatory systems, nervous systems, social systems and ecological trophic networks, show self-similar connectivity at multiple scales. We analyze the relationship between topology and signaling in contrasting classes of such topologies. We find that networks differ in their ability to contain or propagate signals between arbitrary nodes in a network depending on whether they possess branching or loop-like features. Networks also differ in how they respond to noise, such that one allows for greater integration at high noise, and this performance is reversed at low noise. Surprisingly, small-world topologies, with diameters logarithmic in system size, have slower dynamical timescales, and may be less integrated (more modular) than networks with longer path lengths. All of these phenomena are essentially mesoscopic, vanishing in the infinite limit but producing strong effects at sizes and timescales relevant to biology. △ Less

Submitted 20 February, 2012; v1 submitted 12 September, 2011; originally announced September 2011.

Comments: 31 pages, 8 figures, to appear in J. Roy. Soc. Interface

Report number: SFI Working Paper #12-03-003

Non-equilibrium phase transitions in biomolecular signal transduction

Authors: Eric Smith, Supriya Krishnamurthy, Walter Fontana, David Krakauer

Abstract: We study a mechanism for reliable switching in biomolecular signal-transduction cascades. Steady bistable states are created by system-size cooperative effects in populations of proteins, in spite of the fact that the phosphorylation-state transitions of any molecule, by means of which the switch is implemented, are highly stochastic. The emergence of switching is a nonequilibrium phase transition… ▽ More We study a mechanism for reliable switching in biomolecular signal-transduction cascades. Steady bistable states are created by system-size cooperative effects in populations of proteins, in spite of the fact that the phosphorylation-state transitions of any molecule, by means of which the switch is implemented, are highly stochastic. The emergence of switching is a nonequilibrium phase transition in an energetically driven, dissipative system described by a master equation. We use operator and functional integral methods from reaction-diffusion theory to solve for the phase structure, noise spectrum, and escape trajectories and first-passage times of a class of minimal models of switches, showing how all critical properties for switch behavior can be computed within a unified framework. △ Less

Submitted 30 August, 2011; v1 submitted 20 August, 2011; originally announced August 2011.

Evidence of strategic periodicities in collective conflict dynamics

Authors: Simon DeDeo, David C. Krakauer, Jessica C. Flack

Abstract: We analyze the timescales of conflict decision-making in a primate society. We present evidence for multiple, periodic timescales associated with social decision-making and behavioral patterns. We demonstrate the existence of periodicities that are not directly coupled to environmental cycles or known ultraridian mechanisms. Among specific biological and socially-defined demographic classes, perio… ▽ More We analyze the timescales of conflict decision-making in a primate society. We present evidence for multiple, periodic timescales associated with social decision-making and behavioral patterns. We demonstrate the existence of periodicities that are not directly coupled to environmental cycles or known ultraridian mechanisms. Among specific biological and socially-defined demographic classes, periodicities span timescales between hours and days, and many are not driven by exogenous or internal regularities. Our results indicate that they are instead driven by strategic responses to social interaction patterns. Analyses also reveal that a class of individuals, playing a critical functional role, policing, have a signature timescale on the order of one hour. We propose a classification of behavioral timescales analogous to those of the nervous system, with high-frequency, or $α$-scale, behavior occurring on hour-long scales, through to multi-hour, or $β$-scale, behavior, and, finally $γ$ periodicities observed on a timescale of days. △ Less

Submitted 7 January, 2011; originally announced January 2011.

Comments: 22 pages, 7 figures, 1 table. Accepted for publication in Journal of the Royal Society Interface

Report number: SFI Working Paper #11-01-002

Journal ref: J. R. Soc. Interface (2011) vol. 8, no. 62, 1260-1273

Inductive Game Theory and the Dynamics of Animal Conflict

Authors: Simon DeDeo, David C. Krakauer, Jessica C. Flack

Abstract: Conflict destabilizes social interactions and impedes cooperation at multiple scales of biological organization. Of fundamental interest are the causes of turbulent periods of conflict. We analyze conflict dynamics in a monkey society model system. We develop a technique, Inductive Game Theory, to extract directly from time-series data the decision-making strategies used by individuals and groups.… ▽ More Conflict destabilizes social interactions and impedes cooperation at multiple scales of biological organization. Of fundamental interest are the causes of turbulent periods of conflict. We analyze conflict dynamics in a monkey society model system. We develop a technique, Inductive Game Theory, to extract directly from time-series data the decision-making strategies used by individuals and groups. This technique uses Monte Carlo simulation to test alternative causal models of conflict dynamics. We find individuals base their decision to fight on memory of social factors, not on short timescale ecological resource competition. Furthermore, the social assessments on which these decisions are based are triadic (self in relation to another pair of individuals), not pairwise. We show that this triadic decision making causes long conflict cascades and that there is a high population cost of the large fights associated with these cascades. These results suggest that individual agency has been over-emphasized in the social evolution of complex aggregates, and that pair-wise formalisms are inadequate. An appreciation of the empirical foundations of the collective dynamics of conflict is a crucial step towards its effective management. △ Less

Submitted 28 June, 2010; originally announced June 2010.

Comments: 37 pages, includes Supplementary Information, matches PLoS version

Journal ref: PLoS Comput Biol 6(5): e1000782 (May, 2010)

The stochastic behavior of a molecular switching circuit with feedback

Authors: Supriya Krishnamurthy, Eric Smith, David Krakauer, Walter Fontana

Abstract: Background: Using a statistical physics approach, we study the stochastic switching behavior of a model circuit of multisite phosphorylation and dephosphorylation with feedback. The circuit consists of a kinase and phosphatase acting on multiple sites of a substrate that, contingent on its modification state, catalyzes its own phosphorylation and, in a symmetric scenario, dephosphorylation. The… ▽ More Background: Using a statistical physics approach, we study the stochastic switching behavior of a model circuit of multisite phosphorylation and dephosphorylation with feedback. The circuit consists of a kinase and phosphatase acting on multiple sites of a substrate that, contingent on its modification state, catalyzes its own phosphorylation and, in a symmetric scenario, dephosphorylation. The symmetric case is viewed as a cartoon of conflicting feedback that could result from antagonistic pathways im**ing on the state of a shared component. Results: Multisite phosphorylation is sufficient for bistable behavior under feedback even when catalysis is linear in substrate concentration, which is the case we consider. We compute the phase diagram, fluctuation spectrum and large-deviation properties related to switch memory within a statistical mechanics framework. Bistability occurs as either a first-order or second-order non-equilibrium phase transition, depending on the network symmetries and the ratio of phosphatase to kinase numbers. In the second-order case, the circuit never leaves the bistable regime upon increasing the number of substrate molecules at constant kinase to phosphatase ratio. Conclusions: The number of substrate molecules is a key parameter controlling both the onset of the bistable regime, fluctuation intensity, and the residence time in a switched state. The relevance of the concept of memory depends on the degree of switch symmetry, as memory presupposes information to be remembered, which is highest for equal residence times in the switched states. Reviewers: This article was reviewed by Artem Novozhilov (nominated by Eugene Koonin), Sergei Maslov, and Ned Wingreen. △ Less

Submitted 1 October, 2007; v1 submitted 15 December, 2003; originally announced December 2003.

Comments: Version published in Biology Direct including reviewer comments and author responses, 28 pages, 7 figures

Journal ref: Biology Direct, 2:13 (2007)

Structure Function Subgroup Summary

Authors: D. Harris, F. I. Olness, S. Ritz, M. Albrow, E. Berger, T. Bolton, A. Caldwell, A. El-Khadra, B. Ermolaev, J. W. Gary, E. Hughes, J. Huston, D. Jansen, Y. -K. Kim, D. Krakauer, S. Kuhlmann, H. Lai, D. Naples, J. Qiu, D. Reeder, M. H. Reno, R. J. Scalise, B. Schumm, P. Spentzouris, C. Taylor , et al. (8 additional authors not shown)

Abstract: We summarize the studies and discussions of the Structure Function subgroup of the QCD working group of the Snowmass 1996 Workshop: New Directions for High Energy Physics. We summarize the studies and discussions of the Structure Function subgroup of the QCD working group of the Snowmass 1996 Workshop: New Directions for High Energy Physics. △ Less

Submitted 22 December, 1997; v1 submitted 23 June, 1997; originally announced June 1997.

Comments: 14 pages, LaTeX, 23 PostScript figures, uses epsf.sty and snowhack.cls. Contribution to Snowmass 96 Proceedings. Typographical errors corrected and minor changes made to text

Journal ref: ECONF C960625:QCD178,1996