Showing 1–29 of 29 results for author: Parr, T

Active Inference and Intentional Behaviour

Authors: Karl J. Friston, Tommaso Salvatori, Takuya Isomura, Alexander Tschantz, Alex Kiefer, Tim Verbelen, Magnus Koudahl, Aswin Paul, Thomas Parr, Adeel Razi, Brett Kagan, Christopher L. Buckley, Maxwell J. D. Ramstead

Abstract: Recent advances in theoretical biology suggest that basal cognition and sentient behaviour are emergent properties of in vitro cell cultures and neuronal networks, respectively. Such neuronal networks spontaneously learn structured behaviours in the absence of reward or reinforcement. In this paper, we characterise this kind of self-organisation through the lens of the free energy principle, i.e.,… ▽ More Recent advances in theoretical biology suggest that basal cognition and sentient behaviour are emergent properties of in vitro cell cultures and neuronal networks, respectively. Such neuronal networks spontaneously learn structured behaviours in the absence of reward or reinforcement. In this paper, we characterise this kind of self-organisation through the lens of the free energy principle, i.e., as self-evidencing. We do this by first discussing the definitions of reactive and sentient behaviour in the setting of active inference, which describes the behaviour of agents that model the consequences of their actions. We then introduce a formal account of intentional behaviour, that describes agents as driven by a preferred endpoint or goal in latent state-spaces. We then investigate these forms of (reactive, sentient, and intentional) behaviour using simulations. First, we simulate the aforementioned in vitro experiments, in which neuronal cultures spontaneously learn to play Pong, by implementing nested, free energy minimising processes. The simulations are then used to deconstruct the ensuing predictive behaviour, leading to the distinction between merely reactive, sentient, and intentional behaviour, with the latter formalised in terms of inductive planning. This distinction is further studied using simple machine learning benchmarks (navigation in a grid world and the Tower of Hanoi problem), that show how quickly and efficiently adaptive behaviour emerges under an inductive form of active inference. △ Less

Submitted 16 December, 2023; v1 submitted 6 December, 2023; originally announced December 2023.

Comments: 33 pages, 9 figures

Supervised structure learning

Authors: Karl J. Friston, Lancelot Da Costa, Alexander Tschantz, Alex Kiefer, Tommaso Salvatori, Victorita Neacsu, Magnus Koudahl, Conor Heins, Noor Sajid, Dimitrije Markovic, Thomas Parr, Tim Verbelen, Christopher L Buckley

Abstract: This paper concerns structure learning or discovery of discrete generative models. It focuses on Bayesian model selection and the assimilation of training data or content, with a special emphasis on the order in which data are ingested. A key move - in the ensuing schemes - is to place priors on the selection of models, based upon expected free energy. In this setting, expected free energy reduces… ▽ More This paper concerns structure learning or discovery of discrete generative models. It focuses on Bayesian model selection and the assimilation of training data or content, with a special emphasis on the order in which data are ingested. A key move - in the ensuing schemes - is to place priors on the selection of models, based upon expected free energy. In this setting, expected free energy reduces to a constrained mutual information, where the constraints inherit from priors over outcomes (i.e., preferred outcomes). The resulting scheme is first used to perform image classification on the MNIST dataset to illustrate the basic idea, and then tested on a more challenging problem of discovering models with dynamics, using a simple sprite-based visual disentanglement paradigm and the Tower of Hanoi (cf., blocks world) problem. In these examples, generative models are constructed autodidactically to recover (i.e., disentangle) the factorial structure of latent states - and their characteristic paths or dynamics. △ Less

Submitted 16 November, 2023; originally announced November 2023.

A variational synthesis of evolutionary and developmental dynamics

Authors: Karl Friston, Daniel Ari Friedman, Axel Constant, V. Bleu Knight, Thomas Parr, John O. Campbell

Abstract: This paper introduces a variational formulation of natural selection, paying special attention to the nature of "things" and the way that different "kinds" of "things" are individuated from - and influence - each other. We use the Bayesian mechanics of particular partitions to understand how slow phylogenetic processes constrain - and are constrained by - fast, phenotypic processes. The main resul… ▽ More This paper introduces a variational formulation of natural selection, paying special attention to the nature of "things" and the way that different "kinds" of "things" are individuated from - and influence - each other. We use the Bayesian mechanics of particular partitions to understand how slow phylogenetic processes constrain - and are constrained by - fast, phenotypic processes. The main result is a formulation of adaptive fitness as a path integral of phenotypic fitness. Paths of least action, at the phenotypic and phylogenetic scales, can then be read as inference and learning processes, respectively. In this view, a phenotype actively infers the state of its econiche under a generative model, whose parameters are learned via natural (bayesian model selection). The ensuing variational synthesis features some unexpected aspects. Perhaps the most notable is that it is not possible to describe or model a population of conspecifics per se. Rather, it is necessary to consider populations - and nested meta-populations - of different natural kinds that influence each other. This paper is limited to a description of the mathematical apparatus and accompanying ideas. Subsequent work will use these methods for simulations and numerical analyses - and identify points of contact with related mathematical formulations of evolution. △ Less

Submitted 8 March, 2023; originally announced March 2023.

Bistable perception, precision and neuromodulation

Authors: Filip Novicky, Thomas Parr, Karl Friston, M. Berk Mirza, Noor Sajid

Abstract: Bistable perception follows from observing a static, ambiguous, (visual) stimulus with two possible interpretations. Here, we present an active (Bayesian) inference account of bistable perception and posit that perceptual transitions between different interpretations (i.e., inferences) of the same stimulus ensue from specific eye movements that shift the focus to a different visual feature. Formal… ▽ More Bistable perception follows from observing a static, ambiguous, (visual) stimulus with two possible interpretations. Here, we present an active (Bayesian) inference account of bistable perception and posit that perceptual transitions between different interpretations (i.e., inferences) of the same stimulus ensue from specific eye movements that shift the focus to a different visual feature. Formally, these inferences are a consequence of precision control that determines how confident beliefs are and change the frequency with which one can perceive - and alternate between - two distinct percepts. We hypothesised that there are multiple, but distinct, ways in which precision modulation can interact to give rise to a similar frequency of bistable perception. We validated this using numerical simulations of the Necker's cube paradigm and demonstrate the multiple routes that underwrite the frequency of perceptual alternation. Our results provide an (enactive) computational account of the intricate precision balance underwriting bistable perception. Importantly, these precision parameters can be considered the computational homologues of particular neurotransmitters - i.e., acetylcholine, noradrenaline, dopamine - that have been previously implicated in controlling bistable perception, providing a computational link between the neurochemistry and perception. △ Less

Submitted 19 December, 2022; originally announced December 2022.

Path integrals, particular kinds, and strange things

Authors: Karl Friston, Lancelot Da Costa, Dalton A. R. Sakthivadivel, Conor Heins, Grigorios A. Pavliotis, Maxwell Ramstead, Thomas Parr

Abstract: This paper describes a path integral formulation of the free energy principle. The ensuing account expresses the paths or trajectories that a particle takes as it evolves over time. The main results are a method or principle of least action that can be used to emulate the behaviour of particles in open exchange with their external milieu. Particles are defined by a particular partition, in which i… ▽ More This paper describes a path integral formulation of the free energy principle. The ensuing account expresses the paths or trajectories that a particle takes as it evolves over time. The main results are a method or principle of least action that can be used to emulate the behaviour of particles in open exchange with their external milieu. Particles are defined by a particular partition, in which internal states are individuated from external states by active and sensory blanket states. The variational principle at hand allows one to interpret internal dynamics - of certain kinds of particles - as inferring external states that are hidden behind blanket states. We consider different kinds of particles, and to what extent they can be imbued with an elementary form of inference or sentience. Specifically, we consider the distinction between dissipative and conservative particles, inert and active particles and, finally, ordinary and strange particles. Strange particles can be described as inferring their own actions, endowing them with apparent autonomy or agency. In short - of the kinds of particles afforded by a particular partition - strange kinds may be apt for describing sentient behaviour. △ Less

Submitted 13 June, 2023; v1 submitted 23 October, 2022; originally announced October 2022.

Comments: 33 pages (excluding references), 6 figures

Journal ref: Physics of Life Reviews, vol 47, 2023

Reclaiming saliency: rhythmic precision-modulated action and perception

Authors: Ajith Anil Meera, Filip Novicky, Thomas Parr, Karl Friston, Pablo Lanillos, Noor Sajid

Abstract: Computational models of visual attention in artificial intelligence and robotics have been inspired by the concept of a saliency map. These models account for the mutual information between the (current) visual information and its estimated causes. However, they fail to consider the circular causality between perception and action. In other words, they do not consider where to sample next, given c… ▽ More Computational models of visual attention in artificial intelligence and robotics have been inspired by the concept of a saliency map. These models account for the mutual information between the (current) visual information and its estimated causes. However, they fail to consider the circular causality between perception and action. In other words, they do not consider where to sample next, given current beliefs. Here, we reclaim salience as an active inference process that relies on two basic principles: uncertainty minimisation and rhythmic scheduling. For this, we make a distinction between attention and salience. Briefly, we associate attention with precision control, i.e., the confidence with which beliefs can be updated given sampled sensory data, and salience with uncertainty minimisation that underwrites the selection of future sensory data. Using this, we propose a new account of attention based on rhythmic precision-modulation and discuss its potential in robotics, providing numerical experiments that showcase advantages of precision-modulation for state and noise estimation, system identification and action selection for informative path planning. △ Less

Submitted 23 March, 2022; originally announced March 2022.

The free energy principle made simpler but not too simple

Authors: Karl Friston, Lancelot Da Costa, Noor Sajid, Conor Heins, Kai Ueltzhöffer, Grigorios A. Pavliotis, Thomas Parr

Abstract: This paper provides a concise description of the free energy principle, starting from a formulation of random dynamical systems in terms of a Langevin equation and ending with a Bayesian mechanics that can be read as a physics of sentience. It rehearses the key steps using standard results from statistical physics. These steps entail (i) establishing a particular partition of states based upon con… ▽ More This paper provides a concise description of the free energy principle, starting from a formulation of random dynamical systems in terms of a Langevin equation and ending with a Bayesian mechanics that can be read as a physics of sentience. It rehearses the key steps using standard results from statistical physics. These steps entail (i) establishing a particular partition of states based upon conditional independencies that inherit from sparsely coupled dynamics, (ii) unpacking the implications of this partition in terms of Bayesian inference and (iii) describing the paths of particular states with a variational principle of least action. Teleologically, the free energy principle offers a normative account of self-organisation in terms of optimal Bayesian design and decision-making, in the sense of maximising marginal likelihood or Bayesian model evidence. In summary, starting from a description of the world in terms of random dynamical systems, we end up with a description of self-organisation as sentient behaviour that can be interpreted as self-evidencing; namely, self-assembly, autopoiesis or active inference. △ Less

Submitted 30 May, 2023; v1 submitted 17 January, 2022; originally announced January 2022.

Comments: 44 pages, 9 Figures; 64 pages including graphical abstract, highlights and references

Active inference, Bayesian optimal design, and expected utility

Authors: Noor Sajid, Lancelot Da Costa, Thomas Parr, Karl Friston

Abstract: Active inference, a corollary of the free energy principle, is a formal way of describing the behavior of certain kinds of random dynamical systems that have the appearance of sentience. In this chapter, we describe how active inference combines Bayesian decision theory and optimal Bayesian design principles under a single imperative to minimize expected free energy. It is this aspect of active in… ▽ More Active inference, a corollary of the free energy principle, is a formal way of describing the behavior of certain kinds of random dynamical systems that have the appearance of sentience. In this chapter, we describe how active inference combines Bayesian decision theory and optimal Bayesian design principles under a single imperative to minimize expected free energy. It is this aspect of active inference that allows for the natural emergence of information-seeking behavior. When removing prior outcomes preferences from expected free energy, active inference reduces to optimal Bayesian design, i.e., information gain maximization. Conversely, active inference reduces to Bayesian decision theory in the absence of ambiguity and relative risk, i.e., expected utility maximization. Using these limiting cases, we illustrate how behaviors differ when agents select actions that optimize expected utility, expected information gain, and expected free energy. Our T-maze simulations show optimizing expected free energy produces goal-directed information-seeking behavior while optimizing expected utility induces purely exploitive behavior and maximizing information gain engenders intrinsically motivated behavior. △ Less

Submitted 21 September, 2021; originally announced October 2021.

Comments: 19 pages; 3 figures

Bayesian brains and the Rényi divergence

Authors: Noor Sajid, Francesco Faccio, Lancelot Da Costa, Thomas Parr, Jürgen Schmidhuber, Karl Friston

Abstract: Under the Bayesian brain hypothesis, behavioural variations can be attributed to different priors over generative model parameters. This provides a formal explanation for why individuals exhibit inconsistent behavioural preferences when confronted with similar choices. For example, greedy preferences are a consequence of confident (or precise) beliefs over certain outcomes. Here, we offer an alter… ▽ More Under the Bayesian brain hypothesis, behavioural variations can be attributed to different priors over generative model parameters. This provides a formal explanation for why individuals exhibit inconsistent behavioural preferences when confronted with similar choices. For example, greedy preferences are a consequence of confident (or precise) beliefs over certain outcomes. Here, we offer an alternative account of behavioural variability using Rényi divergences and their associated variational bounds. Rényi bounds are analogous to the variational free energy (or evidence lower bound) and can be derived under the same assumptions. Importantly, these bounds provide a formal way to establish behavioural differences through an $α$ parameter, given fixed priors. This rests on changes in $α$ that alter the bound (on a continuous scale), inducing different posterior estimates and consequent variations in behaviour. Thus, it looks as if individuals have different priors, and have reached different conclusions. More specifically, $α\to 0^{+}$ optimisation leads to mass-covering variational estimates and increased variability in choice behaviour. Furthermore, $α\to + \infty$ optimisation leads to mass-seeking variational posteriors and greedy preferences. We exemplify this formulation through simulations of the multi-armed bandit task. We note that these $α$ parameterisations may be especially relevant, i.e., shape preferences, when the true posterior is not in the same family of distributions as the assumed (simpler) approximate density, which may be the case in many real-world scenarios. The ensuing departure from vanilla variational inference provides a potentially useful explanation for differences in behavioural preferences of biological (or artificial) agents under the assumption that the brain performs variational Bayesian inference. △ Less

Submitted 12 July, 2021; originally announced July 2021.

Comments: 23 pages, 5 figures

Anticipation in architectural experience: a computational neurophenomenology for architecture?

Authors: Zakaria Djebbara, Thomas Parr, Karl Friston

Abstract: The perceptual experience of architecture is enacted by the sensory and motor system. When we act, we change the perceived environment according to a set of expectations that depend on our body and the built environment. The continuous process of collecting sensory information is thus based on bodily affordances. Affordances characterize the fit between the physical structure of the body and capac… ▽ More The perceptual experience of architecture is enacted by the sensory and motor system. When we act, we change the perceived environment according to a set of expectations that depend on our body and the built environment. The continuous process of collecting sensory information is thus based on bodily affordances. Affordances characterize the fit between the physical structure of the body and capacities for movement in the built environment. Since little has been done regarding the role of architectural design in the emergence of perceptual experience on a neuronal level, this paper offers a first step towards the role of architectural design in perceptual experience. An approach to synthesize concepts from computational neuroscience with architectural phenomenology into a computational neurophenomenology is considered. The outcome is a framework under which studies of architecture and cognitive neuroscience can be cast. △ Less

Submitted 7 November, 2020; originally announced November 2020.

Comments: 1 title-page, 23 pages, 5 figures

Dynamic causal modelling of immune heterogeneity

Authors: Thomas Parr, Anjali Bhat, Peter Zeidman, Aimee Goel, Alexander J. Billig, Rosalyn Moran, Karl J. Friston

Abstract: An interesting inference drawn by some Covid-19 epidemiological models is that there exists a proportion of the population who are not susceptible to infection -- even at the start of the current pandemic. This paper introduces a model of the immune response to a virus. This is based upon the same sort of mean-field dynamics as used in epidemiology. However, in place of the location, clinical stat… ▽ More An interesting inference drawn by some Covid-19 epidemiological models is that there exists a proportion of the population who are not susceptible to infection -- even at the start of the current pandemic. This paper introduces a model of the immune response to a virus. This is based upon the same sort of mean-field dynamics as used in epidemiology. However, in place of the location, clinical status, and other attributes of people in an epidemiological model, we consider the state of a virus, B and T-lymphocytes, and the antibodies they generate. Our aim is to formalise some key hypotheses as to the mechanism of resistance. We present a series of simple simulations illustrating changes to the dynamics of the immune response under these hypotheses. These include attenuated viral cell entry, pre-existing cross-reactive humoral (antibody-mediated) immunity, and enhanced T-cell dependent immunity. Finally, we illustrate the potential application of this sort of model by illustrating variational inversion (using simulated data) of this model to illustrate its use in testing hypotheses. In principle, this furnishes a fast and efficient immunological assay--based on sequential serology--that provides a (i) quantitative measure of latent immunological responses and (ii) a Bayes optimal classification of the different kinds of immunological response (c.f., glucose tolerance tests used to test for insulin resistance). This may be especially useful in assessing SARS-CoV-2 vaccines. △ Less

Submitted 17 September, 2020; originally announced September 2020.

Reward Maximisation through Discrete Active Inference

Authors: Lancelot Da Costa, Noor Sajid, Thomas Parr, Karl Friston, Ryan Smith

Abstract: Active inference is a probabilistic framework for modelling the behaviour of biological and artificial agents, which derives from the principle of minimising free energy. In recent years, this framework has successfully been applied to a variety of situations where the goal was to maximise reward, offering comparable and sometimes superior performance to alternative approaches. In this paper, we c… ▽ More Active inference is a probabilistic framework for modelling the behaviour of biological and artificial agents, which derives from the principle of minimising free energy. In recent years, this framework has successfully been applied to a variety of situations where the goal was to maximise reward, offering comparable and sometimes superior performance to alternative approaches. In this paper, we clarify the connection between reward maximisation and active inference by demonstrating how and when active inference agents perform actions that are optimal for maximising reward. Precisely, we show the conditions under which active inference produces the optimal solution to the Bellman equation--a formulation that underlies several approaches to model-based reinforcement learning and control. On partially observed Markov decision processes, the standard active inference scheme can produce Bellman optimal actions for planning horizons of 1, but not beyond. In contrast, a recently developed recursive active inference scheme (sophisticated inference) can produce Bellman optimal actions on any finite temporal horizon. We append the analysis with a discussion of the broader relationship between active inference and reinforcement learning. △ Less

Submitted 11 July, 2022; v1 submitted 17 September, 2020; originally announced September 2020.

Comments: 18 pages, 3 figures (main text); 37 pages including references and appendix

Action and Perception as Divergence Minimization

Authors: Danijar Hafner, Pedro A. Ortega, Jimmy Ba, Thomas Parr, Karl Friston, Nicolas Heess

Abstract: To learn directed behaviors in complex environments, intelligent agents need to optimize objective functions. Various objectives are known for designing artificial agents, including task rewards and intrinsic motivation. However, it is unclear how the known objectives relate to each other, which objectives remain yet to be discovered, and which objectives better describe the behavior of humans. We… ▽ More To learn directed behaviors in complex environments, intelligent agents need to optimize objective functions. Various objectives are known for designing artificial agents, including task rewards and intrinsic motivation. However, it is unclear how the known objectives relate to each other, which objectives remain yet to be discovered, and which objectives better describe the behavior of humans. We introduce the Action Perception Divergence (APD), an approach for categorizing the space of possible objective functions for embodied agents. We show a spectrum that reaches from narrow to general objectives. While the narrow objectives correspond to domain-specific rewards as typical in reinforcement learning, the general objectives maximize information with the environment through latent variable models of input sequences. Intuitively, these agents use perception to align their beliefs with the world and use actions to align the world with their beliefs. They infer representations that are informative of past inputs, explore future inputs that are informative of their representations, and select actions or skills that maximally influence future inputs. This explains a wide range of unsupervised objectives from a single principle, including representation learning, information gain, empowerment, and skill discovery. Our findings suggest leveraging powerful world models for unsupervised exploration as a path toward highly adaptive agents that seek out large niches in their environments, rendering task rewards optional. △ Less

Submitted 12 February, 2022; v1 submitted 3 September, 2020; originally announced September 2020.

Comments: Website: https://danijar.com/apd

Parcels and particles: Markov blankets in the brain

Authors: Karl J. Friston, Erik D. Fagerholm, Tahereh S. Zarghami, Thomas Parr, Inês Hipólito, Loïc Magrou, Adeel Razi

Abstract: At the inception of human brain map**, two principles of functional anatomy underwrote most conceptions - and analyses - of distributed brain responses: namely functional segregation and integration. There are currently two main approaches to characterising functional integration. The first is a mechanistic modelling of connectomics in terms of directed effective connectivity that mediates neuro… ▽ More At the inception of human brain map**, two principles of functional anatomy underwrote most conceptions - and analyses - of distributed brain responses: namely functional segregation and integration. There are currently two main approaches to characterising functional integration. The first is a mechanistic modelling of connectomics in terms of directed effective connectivity that mediates neuronal message passing and dynamics on neuronal circuits. The second phenomenological approach usually characterises undirected functional connectivity (i.e., measurable correlations), in terms of intrinsic brain networks, self-organised criticality, dynamical instability, etc. This paper describes a treatment of effective connectivity that speaks to the emergence of intrinsic brain networks and critical dynamics. It is predicated on the notion of Markov blankets that play a fundamental role in the self-organisation of far from equilibrium systems. Using the apparatus of the renormalisation group, we show that much of the phenomenology found in network neuroscience is an emergent property of a particular partition of neuronal states, over progressively larger scales. As such, it offers a way of linking dynamics on directed graphs to the phenomenology of intrinsic brain networks. △ Less

Submitted 19 July, 2020; originally announced July 2020.

Effective immunity and second waves: a dynamic causal modelling study

Authors: Karl J. Friston, Thomas Parr, Peter Zeidman, Adeel Razi, Guillaume Flandin, Jean Daunizeau, Oliver J. Hulme, Alexander J. Billig, Vladimir Litvak, Cathy J. Price, Rosalyn J. Moran, Anthony Costello, Deenan Pillay, Christian Lambert

Abstract: This technical report addresses a pressing issue in the trajectory of the coronavirus outbreak; namely, the rate at which effective immunity is lost following the first wave of the pandemic. This is a crucial epidemiological parameter that speaks to both the consequences of relaxing lockdown and the propensity for a second wave of infections. Using a dynamic causal model of reported cases and deat… ▽ More This technical report addresses a pressing issue in the trajectory of the coronavirus outbreak; namely, the rate at which effective immunity is lost following the first wave of the pandemic. This is a crucial epidemiological parameter that speaks to both the consequences of relaxing lockdown and the propensity for a second wave of infections. Using a dynamic causal model of reported cases and deaths from multiple countries, we evaluated the evidence models of progressively longer periods of immunity. The results speak to an effective population immunity of about three months that, under the model, defers any second wave for approximately six months in most countries. This may have implications for the window of opportunity for tracking and tracing, as well as for develo** vaccination programmes, and other therapeutic interventions. △ Less

Submitted 16 June, 2020; originally announced June 2020.

Comments: 20 pages, 8 figures, 3 tables (technical report)

MSC Class: q-bio.QM

Nonparametric Feature Impact and Importance

Authors: Terence Parr, James D. Wilson, Jeff Hamrick

Abstract: Practitioners use feature importance to rank and eliminate weak predictors during model development in an effort to simplify models and improve generality. Unfortunately, they also routinely conflate such feature importance measures with feature impact, the isolated effect of an explanatory variable on the response variable. This can lead to real-world consequences when importance is inappropriate… ▽ More Practitioners use feature importance to rank and eliminate weak predictors during model development in an effort to simplify models and improve generality. Unfortunately, they also routinely conflate such feature importance measures with feature impact, the isolated effect of an explanatory variable on the response variable. This can lead to real-world consequences when importance is inappropriately interpreted as impact for business or medical insight purposes. The dominant approach for computing importances is through interrogation of a fitted model, which works well for feature selection, but gives distorted measures of feature impact. The same method applied to the same data set can yield different feature importances, depending on the model, leading us to conclude that impact should be computed directly from the data. While there are nonparametric feature selection algorithms, they typically provide feature rankings, rather than measures of impact or importance. They also typically focus on single-variable associations with the response. In this paper, we give mathematical definitions of feature impact and importance, derived from partial dependence curves, that operate directly on the data. To assess quality, we show that features ranked by these definitions are competitive with existing feature selection techniques using three real data sets for predictive tasks. △ Less

Submitted 8 June, 2020; originally announced June 2020.

Sophisticated Inference

Authors: Karl Friston, Lancelot Da Costa, Danijar Hafner, Casper Hesp, Thomas Parr

Abstract: Active inference offers a first principle account of sentient behaviour, from which special and important cases can be derived, e.g., reinforcement learning, active learning, Bayes optimal inference, Bayes optimal design, etc. Active inference resolves the exploitation-exploration dilemma in relation to prior preferences, by placing information gain on the same footing as reward or value. In brief… ▽ More Active inference offers a first principle account of sentient behaviour, from which special and important cases can be derived, e.g., reinforcement learning, active learning, Bayes optimal inference, Bayes optimal design, etc. Active inference resolves the exploitation-exploration dilemma in relation to prior preferences, by placing information gain on the same footing as reward or value. In brief, active inference replaces value functions with functionals of (Bayesian) beliefs, in the form of an expected (variational) free energy. In this paper, we consider a sophisticated kind of active inference, using a recursive form of expected free energy. Sophistication describes the degree to which an agent has beliefs about beliefs. We consider agents with beliefs about the counterfactual consequences of action for states of affairs and beliefs about those latent states. In other words, we move from simply considering beliefs about 'what would happen if I did that' to 'what would I believe about what would happen if I did that'. The recursive form of the free energy functional effectively implements a deep tree search over actions and outcomes in the future. Crucially, this search is over sequences of belief states, as opposed to states per se. We illustrate the competence of this scheme, using numerical simulations of deep decision problems. △ Less

Submitted 7 June, 2020; originally announced June 2020.

Markov Blankets in the Brain

Authors: Ines Hipolito, Maxwell Ramstead, Laura Convertino, Anjali Bhat, Karl Friston, Thomas Parr

Abstract: Recent characterisations of self-organising systems depend upon the presence of a Markov blanket: a statistical boundary that mediates the interactions between what is inside of and outside of a system. We leverage this idea to provide an analysis of partitions in neuronal systems. This is applicable to brain architectures at multiple scales, enabling partitions into single neurons, brain regions,… ▽ More Recent characterisations of self-organising systems depend upon the presence of a Markov blanket: a statistical boundary that mediates the interactions between what is inside of and outside of a system. We leverage this idea to provide an analysis of partitions in neuronal systems. This is applicable to brain architectures at multiple scales, enabling partitions into single neurons, brain regions, and brain-wide networks. This treatment is based upon the canonical micro-circuitry used in empirical studies of effective connectivity, so as to speak directly to practical applications. This depends upon the dynamic coupling between functional units, whose form recapitulates that of a Markov blanket at each level. The nuance afforded by partitioning neural systems in this way highlights certain limitations of modular perspectives of brain function that only consider a single level of description. △ Less

Submitted 4 June, 2020; originally announced June 2020.

Comments: 25 pages, 5 figures, 1 table, Glossary

Tracking and tracing in the UK: a dynamic causal modelling study

Authors: Karl J. Friston, Thomas Parr, Peter Zeidman, Adeel Razi, Guillaume Flandin, Jean Daunizeau, Oliver J. Hulme, Alexander J. Billig, Vladimir Litvak, Cathy J. Price, Rosalyn J. Moran, Christian Lambert

Abstract: By equip** a previously reported dynamic causal model of COVID-19 with an isolation state, we modelled the effects of self-isolation consequent on tracking and tracing. Specifically, we included a quarantine or isolation state occupied by people who believe they might be infected but are asymptomatic, and only leave if they test negative. We recovered maximum posteriori estimates of the model pa… ▽ More By equip** a previously reported dynamic causal model of COVID-19 with an isolation state, we modelled the effects of self-isolation consequent on tracking and tracing. Specifically, we included a quarantine or isolation state occupied by people who believe they might be infected but are asymptomatic, and only leave if they test negative. We recovered maximum posteriori estimates of the model parameters using time series of new cases, daily deaths, and tests for the UK. These parameters were used to simulate the trajectory of the outbreak in the UK over an 18-month period. Several clear-cut conclusions emerged from these simulations. For example, under plausible (graded) relaxations of social distancing, a rebound of infections within weeks is unlikely. The emergence of a later second wave depends almost exclusively on the rate at which we lose immunity, inherited from the first wave. There exists no testing strategy that can attenuate mortality rates, other than by deferring or delaying a second wave. A sufficiently powerful tracking and tracing policy--implemented at the time of writing (10th May 2020)--will defer any second wave beyond a time horizon of 18 months. Crucially, this deferment is within current testing capabilities (requiring an efficacy of tracing and tracking of about 20% of asymptomatic infected cases, with less than 50,000 tests per day). These conclusions are based upon a dynamic causal model for which we provide some construct and face validation, using a comparative analysis of the United Kingdom and Germany, supplemented with recent serological studies. △ Less

Submitted 16 May, 2020; originally announced May 2020.

Comments: 31 pages, 12 figures, one table. Technical report

MSC Class: 92D30

Second waves, social distancing, and the spread of COVID-19 across America

Authors: Karl J. Friston, Thomas Parr, Peter Zeidman, Adeel Razi, Guillaume Flandin, Jean Daunizeau, Oliver J. Hulme, Alexander J. Billig, Vladimir Litvak, Cathy J. Price, Rosalyn J. Moran, Christian Lambert

Abstract: We recently described a dynamic causal model of a COVID-19 outbreak within a single region. Here, we combine several of these (epidemic) models to create a (pandemic) model of viral spread among regions. Our focus is on a second wave of new cases that may result from loss of immunity--and the exchange of people between regions--and how mortality rates can be ameliorated under different strategic r… ▽ More We recently described a dynamic causal model of a COVID-19 outbreak within a single region. Here, we combine several of these (epidemic) models to create a (pandemic) model of viral spread among regions. Our focus is on a second wave of new cases that may result from loss of immunity--and the exchange of people between regions--and how mortality rates can be ameliorated under different strategic responses. In particular, we consider hard or soft social distancing strategies predicated on national (Federal) or regional (State) estimates of the prevalence of infection in the population. The modelling is demonstrated using timeseries of new cases and deaths from the United States to estimate the parameters of a factorial (compartmental) epidemiological model of each State and, crucially, coupling between States. Using Bayesian model reduction, we identify the effective connectivity between States that best explains the initial phases of the outbreak in the United States. Using the ensuing posterior parameter estimates, we then evaluate the likely outcomes of different policies in terms of mortality, working days lost due to lockdown and demands upon critical care. The provisional results of this modelling suggest that social distancing and loss of immunity are the two key factors that underwrite a return to endemic equilibrium. △ Less

Submitted 26 April, 2020; originally announced April 2020.

Comments: Technical report: 35 pages, 14 figures, 1 table

MSC Class: q-bio.QM

Dynamic causal modelling of COVID-19

Authors: Karl J. Friston, Thomas Parr, Peter Zeidman, Adeel Razi, Guillaume Flandin, Jean Daunizeau, Oliver J. Hulme, Alexander J. Billig, Vladimir Litvak, Rosalyn J. Moran, Cathy J. Price, Christian Lambert

Abstract: This technical report describes a dynamic causal model of the spread of coronavirus through a population. The model is based upon ensemble or population dynamics that generate outcomes, like new cases and deaths over time. The purpose of this model is to quantify the uncertainty that attends predictions of relevant outcomes. By assuming suitable conditional dependencies, one can model the effects… ▽ More This technical report describes a dynamic causal model of the spread of coronavirus through a population. The model is based upon ensemble or population dynamics that generate outcomes, like new cases and deaths over time. The purpose of this model is to quantify the uncertainty that attends predictions of relevant outcomes. By assuming suitable conditional dependencies, one can model the effects of interventions (e.g., social distancing) and differences among populations (e.g., herd immunity) to predict what might happen in different circumstances. Technically, this model leverages state-of-the-art variational (Bayesian) model inversion and comparison procedures, originally developed to characterise the responses of neuronal ensembles to perturbations. Here, this modelling is applied to epidemiological populations to illustrate the kind of inferences that are supported and how the model per se can be optimised given timeseries data. Although the purpose of this paper is to describe a modelling protocol, the results illustrate some interesting perspectives on the current pandemic; for example, the nonlinear effects of herd immunity that speak to a self-organised mitigation process. △ Less

Submitted 9 April, 2020; originally announced April 2020.

Comments: Technical report: 40 pages, 13 figures and 2 tables

MSC Class: 92D30

Some interesting observations on the free energy principle

Authors: Karl Friston, Lancelot Da Costa, Thomas Parr

Abstract: Biehl et al (2020) present some interesting observations on an early formulation of the free energy principle in (Friston, 2013). We use these observations to scaffold a discussion of the technical arguments that underwrite the free energy principle. This discussion focuses on solenoidal coupling between various (subsets of) states in sparsely coupled systems that possess a Markov blanket - and th… ▽ More Biehl et al (2020) present some interesting observations on an early formulation of the free energy principle in (Friston, 2013). We use these observations to scaffold a discussion of the technical arguments that underwrite the free energy principle. This discussion focuses on solenoidal coupling between various (subsets of) states in sparsely coupled systems that possess a Markov blanket - and the distinction between exact and approximate Bayesian inference, implied by the ensuing Bayesian mechanics. △ Less

Submitted 5 February, 2020; originally announced February 2020.

Comments: A response to a technical critique [arXiv:2001.06408] of the free energy principle as presented in "Life as we know it"

MSC Class: 37Hxx

Neural dynamics under active inference: plausibility and efficiency of information processing

Authors: Lancelot Da Costa, Thomas Parr, Biswa Sengupta, Karl Friston

Abstract: Active inference is a normative framework for explaining behaviour under the free energy principle -- a theory of self-organisation originating in neuroscience. It specifies neuronal dynamics for state-estimation in terms of a descent on (variational) free energy -- a measure of the fit between an internal (generative) model and sensory observations. The free energy gradient is a prediction error… ▽ More Active inference is a normative framework for explaining behaviour under the free energy principle -- a theory of self-organisation originating in neuroscience. It specifies neuronal dynamics for state-estimation in terms of a descent on (variational) free energy -- a measure of the fit between an internal (generative) model and sensory observations. The free energy gradient is a prediction error -- plausibly encoded in the average membrane potentials of neuronal populations. Conversely, the expected probability of a state can be expressed in terms of neuronal firing rates. We show that this is consistent with current models of neuronal dynamics and establish face validity by synthesising plausible electrophysiological responses. We then show that these neuronal dynamics approximate natural gradient descent, a well-known optimisation algorithm from information geometry that follows the steepest descent of the objective in information space. We compare the information length of belief updating in both schemes, a measure of the distance traveled in information space that has a direct interpretation in terms of metabolic cost. We show that neural dynamics under active inference are metabolically efficient and suggest that neural representations in biological agents may evolve by approximating steepest descent in information space towards the point of optimal inference. △ Less

Submitted 31 January, 2021; v1 submitted 22 January, 2020; originally announced January 2020.

Comments: 12 pages, 3 figures

Journal ref: Entropy 2021

Active inference on discrete state-spaces: a synthesis

Authors: Lancelot Da Costa, Thomas Parr, Noor Sajid, Sebastijan Veselic, Victorita Neacsu, Karl Friston

Abstract: Active inference is a normative principle underwriting perception, action, planning, decision-making and learning in biological or artificial agents. From its inception, its associated process theory has grown to incorporate complex generative models, enabling simulation of a wide range of complex behaviours. Due to successive developments in active inference, it is often difficult to see how its… ▽ More Active inference is a normative principle underwriting perception, action, planning, decision-making and learning in biological or artificial agents. From its inception, its associated process theory has grown to incorporate complex generative models, enabling simulation of a wide range of complex behaviours. Due to successive developments in active inference, it is often difficult to see how its underlying principle relates to process theories and practical implementation. In this paper, we try to bridge this gap by providing a complete mathematical synthesis of active inference on discrete state-space models. This technical summary provides an overview of the theory, derives neuronal dynamics from first principles and relates this dynamics to biological processes. Furthermore, this paper provides a fundamental building block needed to understand active inference for mixed generative models; allowing continuous sensations to inform discrete representations. This paper may be used as follows: to guide research towards outstanding challenges, a practical guide on how to implement active inference to simulate experimental behaviour, or a pointer towards various in-silico neurophysiological responses that may be used to make empirical predictions. △ Less

Submitted 28 March, 2020; v1 submitted 20 January, 2020; originally announced January 2020.

Comments: 36 pages, 5 figures

Journal ref: Journal of Mathematical Psychology 2021

Active inference: demystified and compared

Authors: Noor Sajid, Philip J. Ball, Thomas Parr, Karl J. Friston

Abstract: Active inference is a first principle account of how autonomous agents operate in dynamic, non-stationary environments. This problem is also considered in reinforcement learning (RL), but limited work exists on comparing the two approaches on the same discrete-state environments. In this paper, we provide: 1) an accessible overview of the discrete-state formulation of active inference, highlightin… ▽ More Active inference is a first principle account of how autonomous agents operate in dynamic, non-stationary environments. This problem is also considered in reinforcement learning (RL), but limited work exists on comparing the two approaches on the same discrete-state environments. In this paper, we provide: 1) an accessible overview of the discrete-state formulation of active inference, highlighting natural behaviors in active inference that are generally engineered in RL; 2) an explicit discrete-state comparison between active inference and RL on an OpenAI gym baseline. We begin by providing a condensed overview of the active inference literature, in particular viewing the various natural behaviors of active inference agents through the lens of RL. We show that by operating in a pure belief-based setting, active inference agents can carry out epistemic exploration, and account for uncertainty about their environment in a Bayes-optimal fashion. Furthermore, we show that the reliance on an explicit reward signal in RL is removed in active inference, where reward can simply be treated as another observation; even in the total absence of rewards, agent behaviors are learned through preference learning. We make these properties explicit by showing two scenarios in which active inference agents can infer behaviors in reward-free environments compared to both Q-learning and Bayesian model-based RL agents; by placing zero prior preferences over rewards and by learning the prior preferences over the observations corresponding to reward. We conclude by noting that this formalism can be applied to more complex settings if appropriate generative models can be formulated. In short, we aim to demystify the behavior of active inference agents by presenting an accessible discrete state-space and time formulation, and demonstrate these behaviors in a OpenAI gym environment, alongside RL agents. △ Less

Submitted 30 October, 2020; v1 submitted 24 September, 2019; originally announced September 2019.

Journal ref: Neural Computation 2021

Technical Report: Partial Dependence through Stratification

Authors: Terence Parr, James D. Wilson

Abstract: Partial dependence curves (FPD) introduced by Friedman, are an important model interpretation tool, but are often not accessible to business analysts and scientists who typically lack the skills to choose, tune, and assess machine learning models. It is also common for the same partial dependence algorithm on the same data to give meaningfully different curves for different models, which calls int… ▽ More Partial dependence curves (FPD) introduced by Friedman, are an important model interpretation tool, but are often not accessible to business analysts and scientists who typically lack the skills to choose, tune, and assess machine learning models. It is also common for the same partial dependence algorithm on the same data to give meaningfully different curves for different models, which calls into question their precision. Expertise is required to distinguish between model artifacts and true relationships in the data. In this paper, we contribute methods for computing partial dependence curves, for both numerical (StratPD) and categorical explanatory variables (CatStratPD), that work directly from training data rather than predictions of a model. Our methods provide a direct estimate of partial dependence, and rely on approximating the partial derivative of an unknown regression function without first fitting a model and then approximating its partial derivative. We investigate settings where contemporary partial dependence methods---including FPD, ALE, and SHAP methods---give biased results. Furthermore, we demonstrate that our approach works correctly on synthetic and plausibly on real data sets. Our goal is not to argue that model-based techniques are not useful. Rather, we hope to open a new line of inquiry into nonparametric partial dependence. △ Less

Submitted 24 April, 2020; v1 submitted 15 July, 2019; originally announced July 2019.

Comments: Tweaks/clarifications, added ntrials optional hyper parameter, corrected interpretation of Integrated Gradients technique. For code, see repo https://github.com/parrt/stratx

Bayesian model reduction

Authors: Karl Friston, Thomas Parr, Peter Zeidman

Abstract: This paper reviews recent developments in statistical structure learning; namely, Bayesian model reduction. Bayesian model reduction is a method for rapidly computing the evidence and parameters of probabilistic models that differ only in their priors. In the setting of variational Bayes this has an analytical solution, which finesses the problem of scoring large model spaces in model comparison o… ▽ More This paper reviews recent developments in statistical structure learning; namely, Bayesian model reduction. Bayesian model reduction is a method for rapidly computing the evidence and parameters of probabilistic models that differ only in their priors. In the setting of variational Bayes this has an analytical solution, which finesses the problem of scoring large model spaces in model comparison or structure learning. In this technical note, we review Bayesian model reduction and provide the relevant equations for several discrete and continuous probability distributions. We provide worked examples in the context of multivariate linear regression, Gaussian mixture models and dynamical systems (dynamic causal modelling). These examples are accompanied by the Matlab scripts necessary to reproduce the results. Finally, we briefly review recent applications in the fields of neuroimaging and neuroscience. Specifically, we consider structure learning and hierarchical or empirical Bayes that can be regarded as a metaphor for neurobiological processes like abductive reasoning. △ Less

Submitted 14 October, 2019; v1 submitted 18 May, 2018; originally announced May 2018.

Comments: The manuscript has been thoroughly updated, including more detailed explanations, additional derivations and three worked examples

The Matrix Calculus You Need For Deep Learning

Authors: Terence Parr, Jeremy Howard

Abstract: This paper is an attempt to explain all the matrix calculus you need in order to understand the training of deep neural networks. We assume no math knowledge beyond what you learned in calculus 1, and provide links to help you refresh the necessary math where needed. Note that you do not need to understand this material before you start learning to train and use deep learning in practice; rather,… ▽ More This paper is an attempt to explain all the matrix calculus you need in order to understand the training of deep neural networks. We assume no math knowledge beyond what you learned in calculus 1, and provide links to help you refresh the necessary math where needed. Note that you do not need to understand this material before you start learning to train and use deep learning in practice; rather, this material is for those who are already familiar with the basics of neural networks, and wish to deepen their understanding of the underlying math. Don't worry if you get stuck at some point along the way---just go back and reread the previous section, and try writing down and working through some examples. And if you're still stuck, we're happy to answer your questions in the Theory category at forums.fast.ai. Note: There is a reference section at the end of the paper summarizing all the key matrix calculus rules and terminology discussed here. See related articles at http://explained.ai △ Less

Submitted 2 July, 2018; v1 submitted 5 February, 2018; originally announced February 2018.

Comments: PDF version of mobile/web friendly version http://explained.ai/matrix-calculus/index.html

Technical Report: Towards a Universal Code Formatter through Machine Learning

Authors: Terence Parr, Jurgin Vinju

Abstract: There are many declarative frameworks that allow us to implement code formatters relatively easily for any specific language, but constructing them is cumbersome. The first problem is that "everybody" wants to format their code differently, leading to either many formatter variants or a ridiculous number of configuration options. Second, the size of each implementation scales with a language's gra… ▽ More There are many declarative frameworks that allow us to implement code formatters relatively easily for any specific language, but constructing them is cumbersome. The first problem is that "everybody" wants to format their code differently, leading to either many formatter variants or a ridiculous number of configuration options. Second, the size of each implementation scales with a language's grammar size, leading to hundreds of rules. In this paper, we solve the formatter construction problem using a novel approach, one that automatically derives formatters for any given language without intervention from a language expert. We introduce a code formatter called CodeBuff that uses machine learning to abstract formatting rules from a representative corpus, using a carefully designed feature set. Our experiments on Java, SQL, and ANTLR grammars show that CodeBuff is efficient, has excellent accuracy, and is grammar invariant for a given language. It also generalizes to a 4th language tested during manuscript preparation. △ Less

Submitted 28 June, 2016; originally announced June 2016.