-
Towards Machines that Trust: AI Agents Learn to Trust in the Trust Game
Authors:
Ardavan S. Nobandegani,
Irina Rish,
Thomas R. Shultz
Abstract:
Widely considered a cornerstone of human morality, trust shapes many aspects of human social interactions. In this work, we present a theoretical analysis of the $\textit{trust game}$, the canonical task for studying trust in behavioral and brain sciences, along with simulation results supporting our analysis. Specifically, leveraging reinforcement learning (RL) to train our AI agents, we systemat…
▽ More
Widely considered a cornerstone of human morality, trust shapes many aspects of human social interactions. In this work, we present a theoretical analysis of the $\textit{trust game}$, the canonical task for studying trust in behavioral and brain sciences, along with simulation results supporting our analysis. Specifically, leveraging reinforcement learning (RL) to train our AI agents, we systematically investigate learning trust under various parameterizations of this task. Our theoretical analysis, corroborated by the simulations results presented, provides a mathematical basis for the emergence of trust in the trust game.
△ Less
Submitted 20 December, 2023;
originally announced December 2023.
-
A Computational Model of Children's Learning and Use of Probabilities Across Different Ages
Authors:
Zilong Wang,
Thomas R. Shultz,
Ardvan S. Nobandegani
Abstract:
Recent empirical work has shown that human children are adept at learning and reasoning with probabilities. Here, we model a recent experiment investigating the development of school-age children's non-symbolic probability reasoning ability using the Neural Probability Learner and Sampler (NPLS) system. We demonstrate that NPLS can accurately simulate children's probability judgments at different…
▽ More
Recent empirical work has shown that human children are adept at learning and reasoning with probabilities. Here, we model a recent experiment investigating the development of school-age children's non-symbolic probability reasoning ability using the Neural Probability Learner and Sampler (NPLS) system. We demonstrate that NPLS can accurately simulate children's probability judgments at different ages, tasks and difficulty levels to discriminate two probabilistic choices through accurate probability learning and sampling. An extension of NPLS using a skewed heuristic distribution can also model children's tendency to wrongly select the outcome with more favorable items but less likely to draw the favorable ones when the probabilistic choices are similar. We discuss the roles of two model parameters that can be adjusted to simulate the probability matching versus probability maximization phenomena in children, and why frequency biases children's probabilistic judgments.
△ Less
Submitted 6 May, 2023;
originally announced May 2023.
-
A Neural Model of Number Comparison with Surprisingly Robust Generalization
Authors:
Thomas R. Shultz,
Ardavan S. Nobandegani,
Zilong Wang
Abstract:
We propose a relatively simple computational neural-network model of number comparison. Training on comparisons of the integers 1-9 enable the model to efficiently and accurately simulate a wide range of phenomena, including distance and ratio effects and robust generalization to multidigit integers, negative numbers, and decimal numbers. An accompanying logical model of number comparison provides…
▽ More
We propose a relatively simple computational neural-network model of number comparison. Training on comparisons of the integers 1-9 enable the model to efficiently and accurately simulate a wide range of phenomena, including distance and ratio effects and robust generalization to multidigit integers, negative numbers, and decimal numbers. An accompanying logical model of number comparison provides further insights into the workings of number comparison and its relation to the Arabic number system. These models provide a rational basis for the psychology of number comparison and the ability of neural networks to efficiently learn a powerful system with robust generalization.
△ Less
Submitted 13 October, 2022;
originally announced October 2022.
-
Cognitive Models as Simulators: The Case of Moral Decision-Making
Authors:
Ardavan S. Nobandegani,
Thomas R. Shultz,
Irina Rish
Abstract:
To achieve desirable performance, current AI systems often require huge amounts of training data. This is especially problematic in domains where collecting data is both expensive and time-consuming, e.g., where AI systems require having numerous interactions with humans, collecting feedback from them. In this work, we substantiate the idea of $\textit{cognitive models as simulators}$, which is to…
▽ More
To achieve desirable performance, current AI systems often require huge amounts of training data. This is especially problematic in domains where collecting data is both expensive and time-consuming, e.g., where AI systems require having numerous interactions with humans, collecting feedback from them. In this work, we substantiate the idea of $\textit{cognitive models as simulators}$, which is to have AI systems interact with, and collect feedback from, cognitive models instead of humans, thereby making their training process both less costly and faster. Here, we leverage this idea in the context of moral decision-making, by having reinforcement learning (RL) agents learn about fairness through interacting with a cognitive model of the Ultimatum Game (UG), a canonical task in behavioral and brain sciences for studying fairness. Interestingly, these RL agents learn to rationally adapt their behavior depending on the emotional state of their simulated UG responder. Our work suggests that using cognitive models as simulators of humans is an effective approach for training AI systems, presenting an important way for computational cognitive science to make contributions to AI.
△ Less
Submitted 8 October, 2022;
originally announced October 2022.
-
A Computational Model of Infant Learning and Reasoning with Probabilities
Authors:
Thomas R Shultz,
Ardavan S Nobandegani
Abstract:
Recent experiments reveal that 6- to 12-month-old infants can learn probabilities and reason with them. In this work, we present a novel computational system called Neural Probability Learner and Sampler (NPLS) that learns and reasons with probabilities, providing a computationally sufficient mechanism to explain infant probabilistic learning and inference. In 24 computer simulations, NPLS simulat…
▽ More
Recent experiments reveal that 6- to 12-month-old infants can learn probabilities and reason with them. In this work, we present a novel computational system called Neural Probability Learner and Sampler (NPLS) that learns and reasons with probabilities, providing a computationally sufficient mechanism to explain infant probabilistic learning and inference. In 24 computer simulations, NPLS simulations show how probability distributions can emerge naturally from neural-network learning of event sequences, providing a novel explanation of infant probabilistic learning and reasoning. Three mathematical proofs show how and why NPLS simulates the infant results so accurately. The results are situated in relation to seven other active research lines. This work provides an effective way to integrate Bayesian and neural-network approaches to cognition.
△ Less
Submitted 30 June, 2021;
originally announced June 2021.
-
The evolution of high-fidelity social learning
Authors:
Marcel Montrey,
Thomas R. Shultz
Abstract:
A defining feature of human culture is that knowledge and technology continually improve over time. Such cumulative cultural evolution (CCE) probably depends far more heavily on how reliably information is preserved than on how efficiently it is refined. Therefore, one possible reason that CCE appears diminished or absent in other species is that it requires accurate but specialized forms of socia…
▽ More
A defining feature of human culture is that knowledge and technology continually improve over time. Such cumulative cultural evolution (CCE) probably depends far more heavily on how reliably information is preserved than on how efficiently it is refined. Therefore, one possible reason that CCE appears diminished or absent in other species is that it requires accurate but specialized forms of social learning at which humans are uniquely adept. Here, we develop a Bayesian model to contrast the evolution of high-fidelity social learning, which supports CCE, against low-fidelity social learning, which does not. We find that high-fidelity transmission evolves when (1) social and (2) individual learning are inexpensive, (3) traits are complex, (4) individual learning is abundant, (5) adaptive problems are difficult and (6) behaviour is flexible. Low-fidelity transmission differs in many respects. It not only evolves when (2) individual learning is costly and (4) infrequent but also proves more robust when (3) traits are simple and (5) adaptive problems are easy. If conditions favouring the evolution of high-fidelity transmission are stricter (3 and 5) or harder to meet (2 and 4), this could explain why social learning is common, but CCE is rare.
△ Less
Submitted 5 October, 2020;
originally announced October 2020.
-
Outgroup Homogeneity Bias Causes Ingroup Favoritism
Authors:
Marcel Montrey,
Thomas R. Shultz
Abstract:
Ingroup favoritism, the tendency to favor ingroup over outgroup, is often explained as a product of intergroup conflict, or correlations between group tags and behavior. Such accounts assume that group membership is meaningful, whereas human data show that ingroup favoritism occurs even when it confers no advantage and groups are transparently arbitrary. Another possibility is that ingroup favorit…
▽ More
Ingroup favoritism, the tendency to favor ingroup over outgroup, is often explained as a product of intergroup conflict, or correlations between group tags and behavior. Such accounts assume that group membership is meaningful, whereas human data show that ingroup favoritism occurs even when it confers no advantage and groups are transparently arbitrary. Another possibility is that ingroup favoritism arises due to perceptual biases like outgroup homogeneity, the tendency for humans to have greater difficulty distinguishing outgroup members than ingroup ones. We present a prisoner's dilemma model, where individuals use Bayesian inference to learn how likely others are to cooperate, and then act rationally to maximize expected utility. We show that, when such individuals exhibit outgroup homogeneity bias, ingroup favoritism between arbitrary groups arises through direct reciprocity. However, this outcome may be mitigated by: (1) raising the benefits of cooperation, (2) increasing population diversity, and (3) imposing a more restrictive social structure.
△ Less
Submitted 22 August, 2019;
originally announced August 2019.
-
Neural-network simulations of memory consolidation and reconsolidation
Authors:
Peter Helfer,
Thomas R. Shultz
Abstract:
In the mammalian brain newly acquired memories depend on the hippocampus for maintenance and recall, but over time these functions are taken over by the neocortex through a process called systems consolidation. However, reactivation of a consolidated memory can induce a brief period of temporary hippocampus-dependence, followed by return to hippocampus-independence. Here we present a computational…
▽ More
In the mammalian brain newly acquired memories depend on the hippocampus for maintenance and recall, but over time these functions are taken over by the neocortex through a process called systems consolidation. However, reactivation of a consolidated memory can induce a brief period of temporary hippocampus-dependence, followed by return to hippocampus-independence. Here we present a computational model that uses simulation of recently described mechanisms of synaptic plasticity to account for findings from the systems consolidation/reconsolidation literature and to make predictions for future research.
△ Less
Submitted 18 January, 2019; v1 submitted 8 January, 2019;
originally announced January 2019.
-
Comparing fitness and drift explanations of Neanderthal replacement
Authors:
Daniel R. Shultz,
Marcel Montrey,
Thomas R. Shultz
Abstract:
There is a general consensus among archaeologists that replacement of Neanderthals by anatomically modern humans in Europe occurred around 40K to 35K YBP. However, the causal mechanism for this replacement continues to be debated. Searching for specific fitness advantages in the archaeological record has proven difficult, as these may be obscured, absent, or subject to interpretation. Proposed mod…
▽ More
There is a general consensus among archaeologists that replacement of Neanderthals by anatomically modern humans in Europe occurred around 40K to 35K YBP. However, the causal mechanism for this replacement continues to be debated. Searching for specific fitness advantages in the archaeological record has proven difficult, as these may be obscured, absent, or subject to interpretation. Proposed models have therefore featured either fitness advantages in favor of anatomically modern humans, or invoked neutral drift under various preconditions. To bridge this gap, we rigorously compare the system-level properties of fitness- and drift-based explanations of Neanderthal replacement. Our stochastic simulations and analytical predictions show that, although both fitness and drift can produce fixation, they present important differences in 1) required initial conditions, 2) reliability, 3) time to replacement, and 4) path to replacement (population histories). These results present useful opportunities for comparison with archaeological and genetic data. We find far greater agreement between the available empirical evidence and the system-level properties of replacement by differential fitness, rather than by neutral drift.
△ Less
Submitted 29 October, 2018;
originally announced October 2018.
-
Bringing Order to the Cognitive Fallacy Zoo
Authors:
Ardavan S. Nobandegani,
William Campoli,
Thomas R. Shultz
Abstract:
In the eyes of a rationalist like Descartes or Spinoza, human reasoning is flawless, marching toward uncovering ultimate truth. A few centuries later, however, culminating in the work of Kahneman and Tversky, human reasoning was portrayed as anything but flawless, filled with numerous misjudgments, biases, and cognitive fallacies. With further investigations, new cognitive fallacies continually em…
▽ More
In the eyes of a rationalist like Descartes or Spinoza, human reasoning is flawless, marching toward uncovering ultimate truth. A few centuries later, however, culminating in the work of Kahneman and Tversky, human reasoning was portrayed as anything but flawless, filled with numerous misjudgments, biases, and cognitive fallacies. With further investigations, new cognitive fallacies continually emerged, leading to a state of affairs which can fairly be characterized as the cognitive fallacy zoo! In this largely methodological work, we formally present a principled way to bring order to this zoo. We introduce the idea of establishing implication relationships (IRs) between cognitive fallacies, formally characterizing how one fallacy implies another. IR is analogous to, and partly inspired by, the fundamental concept of reduction in computational complexity theory. We present several examples of IRs involving experimentally well-documented cognitive fallacies: base-rate neglect, availability bias, conjunction fallacy, decoy effect, framing effect, and Allais paradox. We conclude by discussing how our work: (i) allows for identifying those pivotal cognitive fallacies whose investigation would be the most rewarding research agenda, and importantly (ii) permits a systematized, guided research program on cognitive fallacies, motivating influential theoretical as well as experimental avenues of future research.
△ Less
Submitted 15 October, 2018;
originally announced October 2018.
-
Over-representation of Extreme Events in Decision-Making: A Rational Metacognitive Account
Authors:
Ardavan S. Nobandegani,
Kevin da Silva Castanheira,
A. Ross Otto,
Thomas R. Shultz
Abstract:
The Availability bias, manifested in the over-representation of extreme eventualities in decision-making, is a well-known cognitive bias, and is generally taken as evidence of human irrationality. In this work, we present the first rational, metacognitive account of the Availability bias, formally articulated at Marr's algorithmic level of analysis. Concretely, we present a normative, metacognitiv…
▽ More
The Availability bias, manifested in the over-representation of extreme eventualities in decision-making, is a well-known cognitive bias, and is generally taken as evidence of human irrationality. In this work, we present the first rational, metacognitive account of the Availability bias, formally articulated at Marr's algorithmic level of analysis. Concretely, we present a normative, metacognitive model of how a cognitive system should over-represent extreme eventualities, depending on the amount of time available at its disposal for decision-making. Our model also accounts for two well-known framing effects in human decision-making under risk---the fourfold pattern of risk preferences in outcome probability (Tversky & Kahneman, 1992) and in outcome magnitude (Markovitz, 1952)---thereby providing the first metacognitively-rational basis for those effects. Empirical evidence, furthermore, confirms an important prediction of our model. Surprisingly, our model is unimaginably robust with respect to its focal parameter. We discuss the implications of our work for studies on human decision-making, and conclude by presenting a counterintuitive prediction of our model, which, if confirmed, would have intriguing implications for human decision-making under risk. To our knowledge, our model is the first metacognitive, resource-rational process model of cognitive biases in decision-making.
△ Less
Submitted 29 January, 2018;
originally announced January 2018.
-
Coupled feedback loops maintain synaptic long-term potentiation: A computational model of PKMzeta synthesis and AMPA receptor trafficking
Authors:
Peter Helfer,
Thomas R. Shultz
Abstract:
In long-term potentiation (LTP), one of the most studied types of neural plasticity, synaptic strength is persistently increased in response to stimulation. Although a number of different proteins have been implicated in the sub-cellular molecular processes underlying induction and maintenance of LTP, the precise mechanisms remain unknown. A particular challenge is to demonstrate that a proposed m…
▽ More
In long-term potentiation (LTP), one of the most studied types of neural plasticity, synaptic strength is persistently increased in response to stimulation. Although a number of different proteins have been implicated in the sub-cellular molecular processes underlying induction and maintenance of LTP, the precise mechanisms remain unknown. A particular challenge is to demonstrate that a proposed molecular mechanism can provide the level of stability needed to maintain memories for months or longer, in spite of the fact that many of the participating molecules have much shorter life spans. Here we present a computational model that combines simulations of several biochemical reactions that have been suggested in the LTP literature and show that the resulting system does exhibit the required stability. At the core of the model are two interlinked feedback loops of molecular reactions, one involving the atypical protein kinase PKMĪ¶ and its messenger RNA, the other involving PKMĪ¶ and GluA2-containing AMPA receptors. We demonstrate that robust bistability - stable equilibria both in the synapse's potentiated and unpotentiated states - can arise from a set of simple molecular reactions. The model is able to account for a wide range of empirical results, including induction and maintenance of late-phase LTP, cellular memory reconsolidation and the effects of different pharmaceutical interventions.
△ Less
Submitted 30 April, 2018; v1 submitted 2 December, 2017;
originally announced December 2017.
-
A Computational Model of Systems Memory Consolidation and Reconsolidation
Authors:
Peter Helfer,
Thomas R. Shultz
Abstract:
In the mammalian brain, newly acquired memories depend on the hippocampus for maintenance and recall, but over time the neocortex takes over these functions, rendering memories hippocampus-independent. The process responsible for this transformation is called systems memory consolidation. However, reactivation of a well-consolidated memory can trigger a temporary return to a hippocampus-dependent…
▽ More
In the mammalian brain, newly acquired memories depend on the hippocampus for maintenance and recall, but over time the neocortex takes over these functions, rendering memories hippocampus-independent. The process responsible for this transformation is called systems memory consolidation. However, reactivation of a well-consolidated memory can trigger a temporary return to a hippocampus-dependent state, a phenomenon known as systems memory reconsolidation. The neural mechanisms underlying systems memory consolidation and reconsolidation are not well understood. Here, we propose a neural model based on well-documented mechanisms of synaptic plasticity and stability and describe a computational implementation that demonstrates the model's ability to account for a range of findings from the systems consolidation and reconsolidation literature. We derive several predictions from the computational model and suggest experiments that may test its validity.
△ Less
Submitted 27 September, 2019; v1 submitted 3 March, 2017;
originally announced March 2017.
-
Converting Cascade-Correlation Neural Nets into Probabilistic Generative Models
Authors:
Ardavan Salehi Nobandegani,
Thomas R. Shultz
Abstract:
Humans are not only adept in recognizing what class an input instance belongs to (i.e., classification task), but perhaps more remarkably, they can imagine (i.e., generate) plausible instances of a desired class with ease, when prompted. Inspired by this, we propose a framework which allows transforming Cascade-Correlation Neural Networks (CCNNs) into probabilistic generative models, thereby enabl…
▽ More
Humans are not only adept in recognizing what class an input instance belongs to (i.e., classification task), but perhaps more remarkably, they can imagine (i.e., generate) plausible instances of a desired class with ease, when prompted. Inspired by this, we propose a framework which allows transforming Cascade-Correlation Neural Networks (CCNNs) into probabilistic generative models, thereby enabling CCNNs to generate samples from a category of interest. CCNNs are a well-known class of deterministic, discriminative NNs, which autonomously construct their topology, and have been successful in giving accounts for a variety of psychological phenomena. Our proposed framework is based on a Markov Chain Monte Carlo (MCMC) method, called the Metropolis-adjusted Langevin algorithm, which capitalizes on the gradient information of the target distribution to direct its explorations towards regions of high probability, thereby achieving good mixing properties. Through extensive simulations, we demonstrate the efficacy of our proposed framework.
△ Less
Submitted 18 January, 2017;
originally announced January 2017.
-
Cooperative Intergroup Mating Can Overcome Ethnocentrism in Diverse Populations
Authors:
Caitlin J. Mouri,
Thomas R. Shultz
Abstract:
Ethnocentrism is a behavioral strategy seen on every scale of social interaction. Game-theory models demonstrate that evolution selects ethnocentrism because it boosts cooperation, which increases reproductive fitness. However, some believe that interethnic unions have the potential to foster universal cooperation and overcome in-group biases in humans. Here, we use agent-based computer simulation…
▽ More
Ethnocentrism is a behavioral strategy seen on every scale of social interaction. Game-theory models demonstrate that evolution selects ethnocentrism because it boosts cooperation, which increases reproductive fitness. However, some believe that interethnic unions have the potential to foster universal cooperation and overcome in-group biases in humans. Here, we use agent-based computer simulations to test this hypothesis. Cooperative intergroup mating does lend an advantage to a universal cooperation strategy when the cost/benefit ratio of cooperation is low and local population diversity is high.
△ Less
Submitted 30 April, 2015;
originally announced April 2015.
-
Neural Implementation of Probabilistic Models of Cognition
Authors:
Milad Kharratzadeh,
Thomas R. Shultz
Abstract:
Bayesian models of cognition hypothesize that human brains make sense of data by representing probability distributions and applying Bayes' rule to find the best explanation for available data. Understanding the neural mechanisms underlying probabilistic models remains important because Bayesian models provide a computational framework, rather than specifying mechanistic processes. Here, we propos…
▽ More
Bayesian models of cognition hypothesize that human brains make sense of data by representing probability distributions and applying Bayes' rule to find the best explanation for available data. Understanding the neural mechanisms underlying probabilistic models remains important because Bayesian models provide a computational framework, rather than specifying mechanistic processes. Here, we propose a deterministic neural-network model which estimates and represents probability distributions from observable events --- a phenomenon related to the concept of probability matching. Our model learns to represent probabilities without receiving any representation of them from the external world, but rather by experiencing the occurrence patterns of individual events. Our neural implementation of probability matching is paired with a neural module applying Bayes' rule, forming a comprehensive neural scheme to simulate human Bayesian learning and inference. Our model also provides novel explanations of base-rate neglect, a notable deviation from Bayes.
△ Less
Submitted 14 April, 2016; v1 submitted 13 January, 2015;
originally announced January 2015.
-
Evolving useful delusions: Subjectively rational selfishness leads to objectively irrational cooperation
Authors:
Artem Kaznatcheev,
Marcel Montrey,
Thomas R. Shultz
Abstract:
We introduce a framework within evolutionary game theory for studying the distinction between objective and subjective rationality and apply it to the evolution of cooperation on 3-regular random graphs. In our simulations, agents evolve misrepresentations of objective reality that help them cooperate and maintain higher social welfare in the Prisoner's dilemma. These agents act rationally on thei…
▽ More
We introduce a framework within evolutionary game theory for studying the distinction between objective and subjective rationality and apply it to the evolution of cooperation on 3-regular random graphs. In our simulations, agents evolve misrepresentations of objective reality that help them cooperate and maintain higher social welfare in the Prisoner's dilemma. These agents act rationally on their subjective representations of the world, but irrationally from the perspective of an external observer. We model misrepresentations as subjective perceptions of payoffs and quasi-magical thinking as an inferential bias, finding that the former is more conducive to cooperation. This highlights the importance of internal representations, not just observed behavior, in evolutionary thought. Our results provide support for the interface theory of perception and suggest that the individual's interface can serve not only the individual's aims, but also society as a whole, offering insight into social phenomena such as religion.
△ Less
Submitted 30 April, 2014;
originally announced May 2014.
-
Managing Uncertainty in Rule Based Cognitive Models
Authors:
Thomas R. Shultz
Abstract:
An experiment replicated and extended recent findings on psychologically realistic ways of modeling propagation of uncertainty in rule based reasoning. Within a single production rule, the antecedent evidence can be summarized by taking the maximum of disjunctively connected antecedents and the minimum of conjunctively connected antecedents. The maximum certainty factor attached to each of the r…
▽ More
An experiment replicated and extended recent findings on psychologically realistic ways of modeling propagation of uncertainty in rule based reasoning. Within a single production rule, the antecedent evidence can be summarized by taking the maximum of disjunctively connected antecedents and the minimum of conjunctively connected antecedents. The maximum certainty factor attached to each of the rule's conclusions can be sealed down by multiplication with this summarized antecedent certainty. Heckerman's modified certainty factor technique can be used to combine certainties for common conclusions across production rules.
△ Less
Submitted 27 March, 2013;
originally announced April 2013.
