-
Reinforcement Learning in Categorical Cybernetics
Authors:
Jules Hedges,
Riu Rodríguez Sakamoto
Abstract:
We show that several major algorithms of reinforcement learning (RL) fit into the framework of categorical cybernetics, that is to say, parametrised bidirectional processes. We build on our previous work in which we show that value iteration can be represented by precomposition with a certain optic. The outline of the main construction in this paper is: (1) We extend the Bellman operators to param…
▽ More
We show that several major algorithms of reinforcement learning (RL) fit into the framework of categorical cybernetics, that is to say, parametrised bidirectional processes. We build on our previous work in which we show that value iteration can be represented by precomposition with a certain optic. The outline of the main construction in this paper is: (1) We extend the Bellman operators to parametrised optics that apply to action-value functions and depend on a sample. (2) We apply a representable contravariant functor, obtaining a parametrised function that applies the Bellman iteration. (3) This parametrised function becomes the backward pass of another parametrised optic that represents the model, which interacts with an environment via an agent. Thus, parametrised optics appear in two different ways in our construction, with one becoming part of the other. As we show, many of the major classes of algorithms in RL can be seen as different extremal cases of this general setup: dynamic programming, Monte Carlo methods, temporal difference learning, and deep RL. We see this as strong evidence that this approach is a natural one and believe that it will be a fruitful way to think about RL in the future.
△ Less
Submitted 3 April, 2024;
originally announced April 2024.
-
Rendering string diagrams recursively
Authors:
Celia Rubio-Madrigal,
Jules Hedges
Abstract:
String diagrams are a graphical language used to represent processes that can be composed sequentially or in parallel, which correspond graphically to horizontal or vertical juxtaposition. In this paper we demonstrate how to compute the layout of a string diagram by folding over its algebraic representation in terms of sequential and parallel composition operators. The algebraic representation can…
▽ More
String diagrams are a graphical language used to represent processes that can be composed sequentially or in parallel, which correspond graphically to horizontal or vertical juxtaposition. In this paper we demonstrate how to compute the layout of a string diagram by folding over its algebraic representation in terms of sequential and parallel composition operators. The algebraic representation can be seen as a term of a free monoidal category or a proof tree for a small fragment of linear logic. This contrasts to existing non-compositional approaches that use graph layout techniques. The key innovation is storing the diagrams in binary space-partition trees, maintaining a right-trapezoidal shape for the diagram's outline as an invariant.
We provide an implementation in Haskell, using an existing denotational graphics library called Diagrams. Our renderer also supports adding semantics to diagrams to serve as a compiler, with matrix algebra used as an example.
△ Less
Submitted 3 April, 2024;
originally announced April 2024.
-
The Compositional Structure of Bayesian Inference
Authors:
Dylan Braithwaite,
Jules Hedges,
Toby St Clere Smithe
Abstract:
Bayes' rule tells us how to invert a causal process in order to update our beliefs in light of new evidence. If the process is believed to have a complex compositional structure, we may observe that the inversion of the whole can be computed piecewise in terms of the component processes. We study the structure of this compositional rule, noting that it relates to the lens pattern in functional pro…
▽ More
Bayes' rule tells us how to invert a causal process in order to update our beliefs in light of new evidence. If the process is believed to have a complex compositional structure, we may observe that the inversion of the whole can be computed piecewise in terms of the component processes. We study the structure of this compositional rule, noting that it relates to the lens pattern in functional programming. Working in a suitably general axiomatic presentation of a category of Markov kernels, we see how we can think of Bayesian inversion as a particular instance of a state-dependent morphism in a fibred category. We discuss the compositional nature of this, formulated as a functor on the underlying category and explore how this can used for a more type-driven approach to statistical inference.
△ Less
Submitted 20 July, 2023; v1 submitted 10 May, 2023;
originally announced May 2023.
-
Dependent Bayesian Lenses: Categories of Bidirectional Markov Kernels with Canonical Bayesian Inversion
Authors:
Dylan Braithwaite,
Jules Hedges
Abstract:
We generalise an existing construction of Bayesian Lenses to admit lenses between pairs of objects where the backwards object is dependent on states on the forwards object (interpreted as probability distributions). This gives a natural setting for studying stochastic maps with Bayesian inverses restricted to the points supported by a given prior. In order to state this formally we develop a propo…
▽ More
We generalise an existing construction of Bayesian Lenses to admit lenses between pairs of objects where the backwards object is dependent on states on the forwards object (interpreted as probability distributions). This gives a natural setting for studying stochastic maps with Bayesian inverses restricted to the points supported by a given prior. In order to state this formally we develop a proposed definition by Fritz of a support object in a Markov category and show that these give rise to a section into the category of dependent Bayesian lenses encoding a more canonical notion of Bayesian inversion.
△ Less
Submitted 29 September, 2022;
originally announced September 2022.
-
Value Iteration is Optic Composition
Authors:
Jules Hedges,
Riu Rodríguez Sakamoto
Abstract:
Dynamic programming is a class of algorithms used to compute optimal control policies for Markov decision processes. Dynamic programming is ubiquitous in control theory, and is also the foundation of reinforcement learning. In this paper, we show that value improvement, one of the main steps of dynamic programming, can be naturally seen as composition in a category of optics, and intuitively, the…
▽ More
Dynamic programming is a class of algorithms used to compute optimal control policies for Markov decision processes. Dynamic programming is ubiquitous in control theory, and is also the foundation of reinforcement learning. In this paper, we show that value improvement, one of the main steps of dynamic programming, can be naturally seen as composition in a category of optics, and intuitively, the optimal value function is the limit of a chain of optic compositions. We illustrate this with three classic examples: the gridworld, the inverted pendulum and the savings problem. This is a first step towards a complete account of reinforcement learning in terms of parametrised optics.
△ Less
Submitted 31 July, 2023; v1 submitted 9 June, 2022;
originally announced June 2022.
-
Fibre optics
Authors:
Dylan Braithwaite,
Matteo Capucci,
Bruno Gavranović,
Jules Hedges,
Eigil Fjeldgren Rischel
Abstract:
Lenses, optics and dependent lenses (or equivalently morphisms of containers, or equivalently natural transformations of polynomial functors) are all widely used in applied category theory as models of bidirectional processes. From the definition of lenses over a finite product category, optics weaken the required structure to actions of monoidal categories, and dependent lenses make use of the ad…
▽ More
Lenses, optics and dependent lenses (or equivalently morphisms of containers, or equivalently natural transformations of polynomial functors) are all widely used in applied category theory as models of bidirectional processes. From the definition of lenses over a finite product category, optics weaken the required structure to actions of monoidal categories, and dependent lenses make use of the additional property of finite completeness (or, in case of polynomials, even local cartesian closure). This has caused a split in the applied category theory literature between those using optics and those using dependent lenses. The goal of this paper is to unify optics with dependent lenses, by finding a definition of fibre optics admitting both as special cases.
△ Less
Submitted 21 December, 2021;
originally announced December 2021.
-
Composing games into complex institutions
Authors:
Seth Frey,
Jules Hedges,
Joshua Tan,
Philipp Zahn
Abstract:
Game theory is used by all behavioral sciences, but its development has long centered around tools for relatively simple games and toy systems, such as the economic interpretation of equilibrium outcomes. Our contribution, compositional game theory, permits another approach of equally general appeal: the high-level design of large games for expressing complex architectures and representing real-wo…
▽ More
Game theory is used by all behavioral sciences, but its development has long centered around tools for relatively simple games and toy systems, such as the economic interpretation of equilibrium outcomes. Our contribution, compositional game theory, permits another approach of equally general appeal: the high-level design of large games for expressing complex architectures and representing real-world institutions faithfully. Compositional game theory, grounded in the mathematics underlying programming languages, and introduced here as a general computational framework, increases the parsimony of game representations with abstraction and modularity, accelerates search and design, and helps theorists across disciplines express real-world institutional complexity in well-defined ways. Relative to existing approaches in game theory, compositional game theory is especially promising for solving game systems with long-range dependencies, for comparing large numbers of structurally related games, and for nesting games into the larger logical or strategic flows typical of real world policy or institutional systems.
△ Less
Submitted 9 March, 2023; v1 submitted 11 August, 2021;
originally announced August 2021.
-
Towards Foundations of Categorical Cybernetics
Authors:
Matteo Capucci,
Bruno Gavranović,
Jules Hedges,
Eigil Fjeldgren Rischel
Abstract:
We propose a categorical framework for processes which interact bidirectionally with both an environment and a 'controller'. Examples include open learners, in which the controller is an optimiser such as gradient descent, and an approach to compositional game theory closely related to open games, in which the controller is a composite of game-theoretic agents. We believe that 'cybernetic' is an a…
▽ More
We propose a categorical framework for processes which interact bidirectionally with both an environment and a 'controller'. Examples include open learners, in which the controller is an optimiser such as gradient descent, and an approach to compositional game theory closely related to open games, in which the controller is a composite of game-theoretic agents. We believe that 'cybernetic' is an appropriate name for the processes that can be described in this framework.
△ Less
Submitted 3 November, 2022; v1 submitted 13 May, 2021;
originally announced May 2021.
-
Sheet diagrams for bimonoidal categories
Authors:
Cole Comfort,
Antonin Delpeuch,
Jules Hedges
Abstract:
Bimonoidal categories (also known as rig categories) are categories with two monoidal structures, one of which distributes over the other. We formally define sheet diagrams, a graphical calculus for bimonoidal categories that was informally introduced by Staton. Sheet diagrams are string diagrams drawn on a branching surface, which is itself an extruded string diagram. Our main result is a soundne…
▽ More
Bimonoidal categories (also known as rig categories) are categories with two monoidal structures, one of which distributes over the other. We formally define sheet diagrams, a graphical calculus for bimonoidal categories that was informally introduced by Staton. Sheet diagrams are string diagrams drawn on a branching surface, which is itself an extruded string diagram. Our main result is a soundness and completeness theorem of the usual form for graphical calculi: we show that sheet diagrams form the free bimonoidal category on a signature.
△ Less
Submitted 19 December, 2020; v1 submitted 26 October, 2020;
originally announced October 2020.
-
Compositional modelling of network games
Authors:
Elena Di Lavore,
Jules Hedges,
Paweł Sobociński
Abstract:
The analysis of games played on graph-like structures is of increasing importance due to the prevalence of social networks, both virtual and physical, in our daily life. As well as being relevant in computer science, mathematical analysis and computer simulations of such distributed games are vital methodologies in economics, politics and epidemiology, amongst other fields. Our contribution is to…
▽ More
The analysis of games played on graph-like structures is of increasing importance due to the prevalence of social networks, both virtual and physical, in our daily life. As well as being relevant in computer science, mathematical analysis and computer simulations of such distributed games are vital methodologies in economics, politics and epidemiology, amongst other fields. Our contribution is to give compositional semantics of a family of such games as a well-behaved map**, a strict monoidal functor, from a category of open graphs (syntax) to a category of open games (semantics). As well as introducing the theoretical framework, we identify some applications of compositionality.
△ Less
Submitted 16 May, 2022; v1 submitted 5 June, 2020;
originally announced June 2020.
-
Bayesian open games
Authors:
Joe Bolt,
Jules Hedges,
Philipp Zahn
Abstract:
This paper generalises the treatment of compositional game theory as introduced by Ghani et al. in 2018, where games are modelled as morphisms of a symmetric monoidal category. From an economic modelling perspective, the notion of a game in the work by Ghani et al. is not expressive enough for many applications. This includes stochastic environments, stochastic choices by players, as well as incom…
▽ More
This paper generalises the treatment of compositional game theory as introduced by Ghani et al. in 2018, where games are modelled as morphisms of a symmetric monoidal category. From an economic modelling perspective, the notion of a game in the work by Ghani et al. is not expressive enough for many applications. This includes stochastic environments, stochastic choices by players, as well as incomplete information regarding the game being played. The current paper addresses these three issues all at once.
△ Less
Submitted 28 September, 2023; v1 submitted 8 October, 2019;
originally announced October 2019.
-
Foundations of brick diagrams
Authors:
Jules Hedges,
Jelle Herold
Abstract:
We discuss the foundations of 2-dimensional graphical languages, with a view towards their computer implementation in a 'compiler' for monoidal categories. In particular, we discuss the close relationship between string diagrams, pasting diagrams, linear logic proof trees and k-d trees, the last being a data structure from computational geometry and computer graphics that recursively partitions a…
▽ More
We discuss the foundations of 2-dimensional graphical languages, with a view towards their computer implementation in a 'compiler' for monoidal categories. In particular, we discuss the close relationship between string diagrams, pasting diagrams, linear logic proof trees and k-d trees, the last being a data structure from computational geometry and computer graphics that recursively partitions a space. We introduce a minor variant of cubical pasting diagrams, which we call 'brick diagrams', which are used in the Statebox visual programming language. This paper is intended as a discussion and literature review, and contains little mathematics.
△ Less
Submitted 28 August, 2019;
originally announced August 2019.
-
The game semantics of game theory
Authors:
Jules Hedges
Abstract:
We use a reformulation of compositional game theory to reunite game theory with game semantics, by viewing an open game as the System and its choice of contexts as the Environment. Specifically, the system is jointly controlled by $n \geq 0$ noncooperative players, each independently optimising a real-valued payoff. The goal of the system is to play a Nash equilibrium, and the goal of the environm…
▽ More
We use a reformulation of compositional game theory to reunite game theory with game semantics, by viewing an open game as the System and its choice of contexts as the Environment. Specifically, the system is jointly controlled by $n \geq 0$ noncooperative players, each independently optimising a real-valued payoff. The goal of the system is to play a Nash equilibrium, and the goal of the environment is to prevent it. The key to this is the realisation that lenses (from functional programming) form a dialectica category, which have an existing game-semantic interpretation.
In the second half of this paper, we apply these ideas to build a compact closed category of `computable open games' by replacing the underlying dialectica category with a wave-style geometry of interaction category, specifically the Int-construction applied to the cartesian monoidal category of directed-complete partial orders.
△ Less
Submitted 15 September, 2020; v1 submitted 25 April, 2019;
originally announced April 2019.
-
From open learners to open games
Authors:
Jules Hedges
Abstract:
The categories of open learners (due to Fong, Spivak and Tuyéras) and open games (due to the present author, Ghani, Winschel and Zahn) bear a very striking and unexpected similarity. The purpose of this short note is to prove that there is a faithful symmetric monoidal functor from the former to the latter, which means that any supervised neural network (without feedback or other complicating feat…
▽ More
The categories of open learners (due to Fong, Spivak and Tuyéras) and open games (due to the present author, Ghani, Winschel and Zahn) bear a very striking and unexpected similarity. The purpose of this short note is to prove that there is a faithful symmetric monoidal functor from the former to the latter, which means that any supervised neural network (without feedback or other complicating features) can be seen as an open game in a canonical way. Roughly, each parameter is controlled by a different player, and the game's best response relation encodes the dynamics of gradient descent. We suggest paths for further work exploiting the link.
△ Less
Submitted 22 February, 2019;
originally announced February 2019.
-
Sequential games and nondeterministic selection functions
Authors:
Joe Bolt,
Jules Hedges,
Philipp Zahn
Abstract:
This paper analyses Escardó and Oliva's generalisation of selection functions over a strong monad from a game-theoretic perspective. We focus on the case of the nondeterminism (finite nonempty powerset) monad $\mathcal{P}$. We use these nondeterministic selection functions of type $\mathcal{J}^{\mathcal{P}}_R X = (X \rightarrow R) \rightarrow \mathcal{P} (X)$ to study sequential games, extending p…
▽ More
This paper analyses Escardó and Oliva's generalisation of selection functions over a strong monad from a game-theoretic perspective. We focus on the case of the nondeterminism (finite nonempty powerset) monad $\mathcal{P}$. We use these nondeterministic selection functions of type $\mathcal{J}^{\mathcal{P}}_R X = (X \rightarrow R) \rightarrow \mathcal{P} (X)$ to study sequential games, extending previous work linking (deterministic) selection functions to game theory. Similar to deterministic selection functions, which compute a subgame perfect Nash equilibrium play of a game, we characterise those non-deterministic selection functions which have a clear game-theoretic interpretation. We show, surprisingly, no non-deterministic selection function exists which computes the set of all subgame perfect Nash equilibrium plays. Instead we show that there are selection functions corresponding to sequential versions of the iterated removal of strictly dominated strategies.
△ Less
Submitted 14 April, 2019; v1 submitted 16 November, 2018;
originally announced November 2018.
-
Proceedings of the 2018 Workshop on Compositional Approaches in Physics, NLP, and Social Sciences
Authors:
Martha Lewis,
Bob Coecke,
Jules Hedges,
Dimitri Kartsaklis,
Dan Marsden
Abstract:
The ability to compose parts to form a more complex whole, and to analyze a whole as a combination of elements, is desirable across disciplines. This workshop bring together researchers applying compositional approaches to physics, NLP, cognitive science, and game theory. Within NLP, a long-standing aim is to represent how words can combine to form phrases and sentences. Within the framework of di…
▽ More
The ability to compose parts to form a more complex whole, and to analyze a whole as a combination of elements, is desirable across disciplines. This workshop bring together researchers applying compositional approaches to physics, NLP, cognitive science, and game theory. Within NLP, a long-standing aim is to represent how words can combine to form phrases and sentences. Within the framework of distributional semantics, words are represented as vectors in vector spaces. The categorical model of Coecke et al. [2010], inspired by quantum protocols, has provided a convincing account of compositionality in vector space models of NLP. There is furthermore a history of vector space models in cognitive science. Theories of categorization such as those developed by Nosofsky [1986] and Smith et al. [1988] utilise notions of distance between feature vectors. More recently Gärdenfors [2004, 2014] has developed a model of concepts in which conceptual spaces provide geometric structures, and information is represented by points, vectors and regions in vector spaces. The same compositional approach has been applied to this formalism, giving conceptual spaces theory a richer model of compositionality than previously [Bolt et al., 2018]. Compositional approaches have also been applied in the study of strategic games and Nash equilibria. In contrast to classical game theory, where games are studied monolithically as one global object, compositional game theory works bottom-up by building large and complex games from smaller components. Such an approach is inherently difficult since the interaction between games has to be considered. Research into categorical compositional methods for this field have recently begun [Ghani et al., 2018]. Moreover, the interaction between the three disciplines of cognitive science, linguistics and game theory is a fertile ground for research. Game theory in cognitive science is a well-established area [Camerer, 2011]. Similarly game theoretic approaches have been applied in linguistics [Jäger, 2008]. Lastly, the study of linguistics and cognitive science is intimately intertwined [Smolensky and Legendre, 2006, Jackendoff, 2007]. Physics supplies compositional approaches via vector spaces and categorical quantum theory, allowing the interplay between the three disciplines to be examined.
△ Less
Submitted 6 November, 2018;
originally announced November 2018.
-
Limits of bimorphic lenses
Authors:
Jules Hedges
Abstract:
Bimorphic lenses are a simplification of polymorphic lenses that (like polymorphic lenses) have a type defined by 4 parameters, but which are defined in a monomorphic type system (i.e. an ordinary category with finite products). We show that the category of bimorphic lenses is complete when the base category is complete, cocomplete and cartesian closed, and so symmetric bimorphic lenses can be def…
▽ More
Bimorphic lenses are a simplification of polymorphic lenses that (like polymorphic lenses) have a type defined by 4 parameters, but which are defined in a monomorphic type system (i.e. an ordinary category with finite products). We show that the category of bimorphic lenses is complete when the base category is complete, cocomplete and cartesian closed, and so symmetric bimorphic lenses can be defined as spans of ordinary bimorphic lenses. This is in contrast to monomorphic lenses, which do not have pullbacks, and for which the category of spans can be defined in an ad-hoc way only when the lenses satisfy a certain axiom (the put-get law). This is a step towards a theory of symmetric polymorphic lenses. Bimorphic lenses additionally play an essential role in compositional game theory, and spans of bimorphic lenses are a step towards a compact closed category of open games.
△ Less
Submitted 25 August, 2019; v1 submitted 16 August, 2018;
originally announced August 2018.
-
Towards Functorial Language-Games
Authors:
Jules Hedges,
Martha Lewis
Abstract:
In categorical compositional semantics of natural language one studies functors from a category of grammatical derivations (such as a Lambek pregroup) to a semantic category (such as real vector spaces). We compositionally build game-theoretic semantics of sentences by taking the semantic category to be the category whose morphisms are open games. This requires some modifications to the grammar ca…
▽ More
In categorical compositional semantics of natural language one studies functors from a category of grammatical derivations (such as a Lambek pregroup) to a semantic category (such as real vector spaces). We compositionally build game-theoretic semantics of sentences by taking the semantic category to be the category whose morphisms are open games. This requires some modifications to the grammar category to compensate for the failure of open games to form a compact closed category. We illustrate the theory using simple examples of Wittgenstein's language-games.
△ Less
Submitted 8 November, 2018; v1 submitted 20 July, 2018;
originally announced July 2018.
-
Backward Induction for Repeated Games
Authors:
Jules Hedges
Abstract:
We present a method of backward induction for computing approximate subgame perfect Nash equilibria of infinitely repeated games with discounted payoffs. This uses the selection monad transformer, combined with the searchable set monad viewed as a notion of 'topologically compact' nondeterminism, and a simple model of computable real numbers. This is the first application of Escardó and Oliva's th…
▽ More
We present a method of backward induction for computing approximate subgame perfect Nash equilibria of infinitely repeated games with discounted payoffs. This uses the selection monad transformer, combined with the searchable set monad viewed as a notion of 'topologically compact' nondeterminism, and a simple model of computable real numbers. This is the first application of Escardó and Oliva's theory of higher-order sequential games to games of imperfect information, in which (as well as its mathematical elegance) lazy evaluation does nontrivial work for us compared with a traditional game-theoretic analysis. Since a full theoretical understanding of this method is lacking (and appears to be very hard), we consider this an 'experimental' paper heavily inspired by theoretical ideas. We use the famous Iterated Prisoner's Dilemma as a worked example.
△ Less
Submitted 11 July, 2018; v1 submitted 19 April, 2018;
originally announced April 2018.
-
The algebra of predicting agents
Authors:
Joe Bolt,
Jules Hedges,
Viktor Winschel
Abstract:
The category of open games, which provides a strongly compositional foundation of economic game theory, is intermediate between symmetric monoidal and compact closed. More precisely it has counits with no corresponding units, and a partially defined duality. There exist open games with the same types as unit maps, given by agents with the strategic goal of predicting a future value. Such agents ap…
▽ More
The category of open games, which provides a strongly compositional foundation of economic game theory, is intermediate between symmetric monoidal and compact closed. More precisely it has counits with no corresponding units, and a partially defined duality. There exist open games with the same types as unit maps, given by agents with the strategic goal of predicting a future value. Such agents appear in earlier work on selection functions. We explore the algebraic properties of these agents via the symmetric monoidal bicategory whose 2-cells are morphisms between open games, and show how the resulting structure approximates a compact closed category with a family of lax commutative bialgebras.
△ Less
Submitted 27 March, 2018;
originally announced March 2018.
-
Morphisms of open games
Authors:
Jules Hedges
Abstract:
We define a notion of morphisms between open games, exploiting a surprising connection between lenses in computer science and compositional game theory. This extends the more intuitively obvious definition of globular morphisms as map**s between strategy profiles that preserve best responses, and hence in particular preserve Nash equilibria. We construct a symmetric monoidal double category in w…
▽ More
We define a notion of morphisms between open games, exploiting a surprising connection between lenses in computer science and compositional game theory. This extends the more intuitively obvious definition of globular morphisms as map**s between strategy profiles that preserve best responses, and hence in particular preserve Nash equilibria. We construct a symmetric monoidal double category in which the horizontal 1-cells are open games, vertical 1-morphisms are lenses, and 2-cells are morphisms of open games. States (morphisms out of the monoidal unit) in the vertical category give a flexible solution concept that includes both Nash and subgame perfect equilibria. Products in the vertical category give an external choice operator that is reminiscent of products in game semantics, and is useful in practical examples. We illustrate the above two features with a simple worked example from microeconomics, the market entry game.
△ Less
Submitted 23 November, 2017; v1 submitted 19 November, 2017;
originally announced November 2017.
-
Coherence for lenses and open games
Authors:
Jules Hedges
Abstract:
Categories of polymorphic lenses in computer science, and of open games in compositional game theory, have a curious structure that is reminiscent of compact closed categories, but differs in some crucial ways. Specifically they have a family of morphisms that behave like the counits of a compact closed category, but have no corresponding units; and they have a `partial' duality that behaves like…
▽ More
Categories of polymorphic lenses in computer science, and of open games in compositional game theory, have a curious structure that is reminiscent of compact closed categories, but differs in some crucial ways. Specifically they have a family of morphisms that behave like the counits of a compact closed category, but have no corresponding units; and they have a `partial' duality that behaves like transposition in a compact closed category when it is defined. We axiomatise this structure, which we refer to as a `teleological category'. We precisely define a diagrammatic language suitable for these categories, and prove a coherence theorem for them. This underpins the use of diagrammatic reasoning in compositional game theory, which has previously been used only informally.
△ Less
Submitted 15 September, 2017; v1 submitted 7 April, 2017;
originally announced April 2017.
-
Compositionality and String Diagrams for Game Theory
Authors:
Jules Hedges,
Evguenia Shprits,
Viktor Winschel,
Philipp Zahn
Abstract:
We introduce string diagrams as a formal mathematical, graphical language to represent, compose, program and reason about games. The language is well established in quantum physics, quantum computing and quantum linguistic with the semantics given by category theory. We apply this language to the game theoretical setting and show examples how to use it for some economic games where we highlight th…
▽ More
We introduce string diagrams as a formal mathematical, graphical language to represent, compose, program and reason about games. The language is well established in quantum physics, quantum computing and quantum linguistic with the semantics given by category theory. We apply this language to the game theoretical setting and show examples how to use it for some economic games where we highlight the compositional nature of our higher-order game theory.
△ Less
Submitted 20 April, 2016;
originally announced April 2016.
-
Compositional game theory
Authors:
Neil Ghani,
Jules Hedges,
Viktor Winschel,
Philipp Zahn
Abstract:
We introduce open games as a compositional foundation of economic game theory. A compositional approach potentially allows methods of game theory and theoretical computer science to be applied to large-scale economic models for which standard economic tools are not practical. An open game represents a game played relative to an arbitrary environment and to this end we introduce the concept of cout…
▽ More
We introduce open games as a compositional foundation of economic game theory. A compositional approach potentially allows methods of game theory and theoretical computer science to be applied to large-scale economic models for which standard economic tools are not practical. An open game represents a game played relative to an arbitrary environment and to this end we introduce the concept of coutility, which is the utility generated by an open game and returned to its environment. Open games are the morphisms of a symmetric monoidal category and can therefore be composed by categorical composition into sequential move games and by monoidal products into simultaneous move games. Open games can be represented by string diagrams which provide an intuitive but formal visualisation of the information flows. We show that a variety of games can be faithfully represented as open games in the sense of having the same Nash equilibria and off-equilibrium best responses.
△ Less
Submitted 15 February, 2018; v1 submitted 15 March, 2016;
originally announced March 2016.
-
A Generalised Quantifier Theory of Natural Language in Categorical Compositional Distributional Semantics with Bialgebras
Authors:
Jules Hedges,
Mehrnoosh Sadrzadeh
Abstract:
Categorical compositional distributional semantics is a model of natural language; it combines the statistical vector space models of words with the compositional models of grammar. We formalise in this model the generalised quantifier theory of natural language, due to Barwise and Cooper. The underlying setting is a compact closed category with bialgebras. We start from a generative grammar forma…
▽ More
Categorical compositional distributional semantics is a model of natural language; it combines the statistical vector space models of words with the compositional models of grammar. We formalise in this model the generalised quantifier theory of natural language, due to Barwise and Cooper. The underlying setting is a compact closed category with bialgebras. We start from a generative grammar formalisation and develop an abstract categorical compositional semantics for it, then instantiate the abstract setting to sets and relations and to finite dimensional vector spaces and linear maps. We prove the equivalence of the relational instantiation to the truth theoretic semantics of generalised quantifiers. The vector space instantiation formalises the statistical usages of words and enables us to, for the first time, reason about quantified phrases and sentences compositionally in distributional semantics.
△ Less
Submitted 2 September, 2017; v1 submitted 4 February, 2016;
originally announced February 2016.
-
Higher-Order Decision Theory
Authors:
Jules Hedges,
Paulo Oliva,
Evguenia Sprits,
Viktor Winschel,
Philipp Zahn
Abstract:
Classical decision theory models behaviour in terms of utility maximisation where utilities represent rational preference relations over outcomes. However, empirical evidence and theoretical considerations suggest that we need to go beyond this framework. We propose to represent goals by higher-order functions or operators that take other functions as arguments where the max and argmax operators a…
▽ More
Classical decision theory models behaviour in terms of utility maximisation where utilities represent rational preference relations over outcomes. However, empirical evidence and theoretical considerations suggest that we need to go beyond this framework. We propose to represent goals by higher-order functions or operators that take other functions as arguments where the max and argmax operators are special cases. Our higher-order functions take a context function as their argument where a context represents a process from actions to outcomes. By that we can define goals being dependent on the actions and the process in addition to outcomes only. This formulation generalises outcome based preferences to context-dependent goals. We show how to uniformly represent within our higher-order framework classical utility maximisation but also various other extensions that have been debated in economics.
△ Less
Submitted 3 June, 2015; v1 submitted 2 June, 2015;
originally announced June 2015.
-
Higher-Order Game Theory
Authors:
Jules Hedges,
Paulo Oliva,
Evguenia Sprits,
Viktor Winschel,
Philipp Zahn
Abstract:
In applied game theory the motivation of players is a key element. It is encoded in the payoffs of the game form and often based on utility functions. But there are cases were formal descriptions in the form of a utility function do not exist. In this paper we introduce a representation of games where players' goals are modeled based on so-called higher-order functions. Our representation provides…
▽ More
In applied game theory the motivation of players is a key element. It is encoded in the payoffs of the game form and often based on utility functions. But there are cases were formal descriptions in the form of a utility function do not exist. In this paper we introduce a representation of games where players' goals are modeled based on so-called higher-order functions. Our representation provides a general and powerful way to mathematically summarize players' intentions. In our framework utility functions as well as preference relations are special cases to describe players' goals. We show that in higher-order functions formal descriptions of players may still exist where utility functions do not using a classical example, a variant of Keynes' beauty contest. We also show that equilibrium conditions based on Nash can be easily adapted to our framework. Lastly, this framework serves as a step** stone to powerful tools from computer science that can be usefully applied to economic game theory in the future such as computational and computability aspects.
△ Less
Submitted 3 June, 2015; v1 submitted 2 June, 2015;
originally announced June 2015.
-
String diagrams for game theory
Authors:
Jules Hedges
Abstract:
This paper presents a monoidal category whose morphisms are games (in the sense of game theory, not game semantics) and an associated diagrammatic language. The two basic operations of a monoidal category, namely categorical composition and tensor product, correspond roughly to sequential and simultaneous composition of games. This leads to a compositional theory in which we can reason about prope…
▽ More
This paper presents a monoidal category whose morphisms are games (in the sense of game theory, not game semantics) and an associated diagrammatic language. The two basic operations of a monoidal category, namely categorical composition and tensor product, correspond roughly to sequential and simultaneous composition of games. This leads to a compositional theory in which we can reason about properties of games in terms of corresponding properties of the component parts. In particular, we give a definition of Nash equilibrium which is recursive on the causal structure of the game.
The key technical idea in this paper is the use of continuation passing style for reasoning about the future consequences of players' choices, closely based on applications of selection functions in game theory. Additionally, the clean categorical foundation gives many opportunities for generalisation, for example to learning agents.
△ Less
Submitted 20 March, 2015;
originally announced March 2015.
-
The selection monad as a CPS transformation
Authors:
Jules Hedges
Abstract:
A computation in the continuation monad returns a final result given a continuation, ie. it is a function with type $(X \to R) \to R$. If we instead return the intermediate result at $X$ then our computation is called a selection function. Selection functions appear in diverse areas of mathematics and computer science (especially game theory, proof theory and topology) but the existing literature…
▽ More
A computation in the continuation monad returns a final result given a continuation, ie. it is a function with type $(X \to R) \to R$. If we instead return the intermediate result at $X$ then our computation is called a selection function. Selection functions appear in diverse areas of mathematics and computer science (especially game theory, proof theory and topology) but the existing literature does not heavily emphasise the fact that the selection monad is a CPS translation. In particular it has so far gone unnoticed that the selection monad has a call/cc-like operator with interesting similarities and differences to the usual call/cc, which we explore experimentally using Haskell.
Selection functions can be used whenever we find the intermediate result more interesting than the final result. For example a SAT solver computes an assignment to a boolean function, and then its continuation decides whether it is a satisfying assignment, and we find the assignment itself more interesting than the fact that it is or is not satisfying. In game theory we find the move chosen by a player more interesting than the outcome that results from that move. The author and collaborators are develo** a theory of games in which selection functions are viewed as generalised notions of rationality, used to model players. By realising that strategic contexts in game theory are examples of continuations we can see that classical game theory narrowly misses being in CPS, and that a small change of viewpoint yields a theory of games that is better behaved, and especially more compositional.
△ Less
Submitted 20 March, 2015;
originally announced March 2015.
-
A Higher-order Framework for Decision Problems and Games
Authors:
Jules Hedges,
Paulo Oliva,
Evguenia Winschel,
Viktor Winschel,
Philipp Zahn
Abstract:
We introduce a new unified framework for modelling both decision problems and finite games based on quantifiers and selection functions. We show that the canonical utility maximisation is one special case of a quantifier and that our more abstract framework provides several additional degrees of freedom in modelling. In particular, incomplete preferences, non-maximising heuristics, and context-dep…
▽ More
We introduce a new unified framework for modelling both decision problems and finite games based on quantifiers and selection functions. We show that the canonical utility maximisation is one special case of a quantifier and that our more abstract framework provides several additional degrees of freedom in modelling. In particular, incomplete preferences, non-maximising heuristics, and context-dependent motives can be taken into account when describing an agent's goal. We introduce a suitable generalisation of Nash equilibrium for games in terms of quantifiers and selection functions. Moreover, we introduce a refinement of Nash that captures context-dependency of goals. Modelling in our framework is compositional as the parts of the game are modular and can be easily exchanged. We provide an extended example where we illustrate concepts and highlight the benefits of our alternative modelling approach.
△ Less
Submitted 25 September, 2014;
originally announced September 2014.
-
Monad Transformers for Backtracking Search
Authors:
Jules Hedges
Abstract:
This paper extends Escardo and Oliva's selection monad to the selection monad transformer, a general monadic framework for expressing backtracking search algorithms in Haskell. The use of the closely related continuation monad transformer for similar purposes is also discussed, including an implementation of a DPLL-like SAT solver with no explicit recursion. Continuing a line of work exploring con…
▽ More
This paper extends Escardo and Oliva's selection monad to the selection monad transformer, a general monadic framework for expressing backtracking search algorithms in Haskell. The use of the closely related continuation monad transformer for similar purposes is also discussed, including an implementation of a DPLL-like SAT solver with no explicit recursion. Continuing a line of work exploring connections between selection functions and game theory, we use the selection monad transformer with the nondeterminism monad to obtain an intuitive notion of backward induction for a certain class of nondeterministic games.
△ Less
Submitted 8 June, 2014;
originally announced June 2014.
-
Dialectica models of additive-free linear logic
Authors:
Jules Hedges
Abstract:
This paper presents a construction which transforms categorical models of additive-free propositional linear logic, closely based on de Paiva's dialectica categories and Oliva's functional interpretations of classical linear logic. The construction is defined using dependent type theory, which proves to be a useful tool for reasoning about dialectica categories. Abstractly, we have a closure opera…
▽ More
This paper presents a construction which transforms categorical models of additive-free propositional linear logic, closely based on de Paiva's dialectica categories and Oliva's functional interpretations of classical linear logic. The construction is defined using dependent type theory, which proves to be a useful tool for reasoning about dialectica categories. Abstractly, we have a closure operator on the class of models: it preserves soundness and completeness and has a monad-like structure. When applied to categories of games we obtain `games with bidding', which are hybrids of dialectica and game models, and we prove completeness theorems for two specific such models.
△ Less
Submitted 25 September, 2014; v1 submitted 18 January, 2014;
originally announced January 2014.
-
Reconstructing the Population Genetic History of the Caribbean
Authors:
Andres Moreno-Estrada,
Simon Gravel,
Fouad Zakharia,
Jacob L. McCauley,
Jake K. Byrnes,
Christopher R. Gignoux,
Patricia A. Ortiz-Tello,
Ricardo J. Martinez,
Dale J. Hedges,
Richard W. Morris,
Celeste Eng,
Karla Sandoval,
Suehelay Acevedo-Acevedo,
Juan Carlos Martinez-Cruzado,
Paul J. Norman,
Zulay Layrisse,
Peter Parham,
Esteban Gonzalez Burchard,
Michael L. Cuccaro,
Eden R. Martin,
Carlos D. Bustamante
Abstract:
The Caribbean basin is home to some of the most complex interactions in recent history among previously diverged human populations. Here, by making use of genome-wide SNP array data, we characterize ancestral components of Caribbean populations on a sub-continental level and unveil fine-scale patterns of population structure distinguishing insular from mainland Caribbean populations as well as fro…
▽ More
The Caribbean basin is home to some of the most complex interactions in recent history among previously diverged human populations. Here, by making use of genome-wide SNP array data, we characterize ancestral components of Caribbean populations on a sub-continental level and unveil fine-scale patterns of population structure distinguishing insular from mainland Caribbean populations as well as from other Hispanic/Latino groups. We provide genetic evidence for an inland South American origin of the Native American component in island populations and for extensive pre-Columbian gene flow across the Caribbean basin. The Caribbean-derived European component shows significant differentiation from parental Iberian populations, presumably as a result of founder effects during the colonization of the New World. Based on demographic models, we reconstruct the complex population history of the Caribbean since the onset of continental admixture. We find that insular populations are best modeled as mixtures absorbing two pulses of African migrants, coinciding with early and maximum activity stages of the transatlantic slave trade. These two pulses appear to have originated in different regions within West Africa, imprinting two distinguishable signatures in present day Afro-Caribbean genomes and shedding light on the genetic impact of the dynamics occurring during the slave trade in the Caribbean.
△ Less
Submitted 3 June, 2013;
originally announced June 2013.
-
A generalisation of Nash's theorem with higher-order functionals
Authors:
Julian Hedges
Abstract:
The recent theory of sequential games and selection functions by Mar- tin Escardo and Paulo Oliva is extended to games in which players move simultaneously. The Nash existence theorem for mixed-strategy equilibria of finite games is generalised to games defined by selection functions. A normal form construction is given which generalises the game-theoretic normal form, and its soundness is proven.…
▽ More
The recent theory of sequential games and selection functions by Mar- tin Escardo and Paulo Oliva is extended to games in which players move simultaneously. The Nash existence theorem for mixed-strategy equilibria of finite games is generalised to games defined by selection functions. A normal form construction is given which generalises the game-theoretic normal form, and its soundness is proven. Minimax strategies also gener- alise to the new class of games and are computed by the Berardi-Bezem- Coquand functional, studied in proof theory as an interpretation of the axiom of countable choice.
△ Less
Submitted 21 January, 2013;
originally announced January 2013.
