-
Large Language Models Show Human-like Social Desirability Biases in Survey Responses
Authors:
Aadesh Salecha,
Molly E. Ireland,
Shashanka Subrahmanya,
João Sedoc,
Lyle H. Ungar,
Johannes C. Eichstaedt
Abstract:
As Large Language Models (LLMs) become widely used to model and simulate human behavior, understanding their biases becomes critical. We developed an experimental framework using Big Five personality surveys and uncovered a previously undetected social desirability bias in a wide range of LLMs. By systematically varying the number of questions LLMs were exposed to, we demonstrate their ability to…
▽ More
As Large Language Models (LLMs) become widely used to model and simulate human behavior, understanding their biases becomes critical. We developed an experimental framework using Big Five personality surveys and uncovered a previously undetected social desirability bias in a wide range of LLMs. By systematically varying the number of questions LLMs were exposed to, we demonstrate their ability to infer when they are being evaluated. When personality evaluation is inferred, LLMs skew their scores towards the desirable ends of trait dimensions (i.e., increased extraversion, decreased neuroticism, etc). This bias exists in all tested models, including GPT-4/3.5, Claude 3, Llama 3, and PaLM-2. Bias levels appear to increase in more recent models, with GPT-4's survey responses changing by 1.20 (human) standard deviations and Llama 3's by 0.98 standard deviations-very large effects. This bias is robust to randomization of question order and paraphrasing. Reverse-coding all the questions decreases bias levels but does not eliminate them, suggesting that this effect cannot be attributed to acquiescence bias. Our findings reveal an emergent social desirability bias and suggest constraints on profiling LLMs with psychometric tests and on using LLMs as proxies for human participants.
△ Less
Submitted 9 May, 2024;
originally announced May 2024.
-
Time Complexity Analysis of an Evolutionary Algorithm for approximating Nash Equilibriums
Authors:
Aadesh Salecha
Abstract:
The framework outlined in [arXiv:2010.13024] provides an approximation algorithm for computing Nash equilibria of normal form games. Since NASH is a well-known PPAD-complete problem, this framework has potential applications to other $PPAD$ problems. The correctness of this framework has been empirically validated on 4 well-studied 2x2 games: Prisoner's Dilemma, Stag Hunt, Battle, and Chicken. In…
▽ More
The framework outlined in [arXiv:2010.13024] provides an approximation algorithm for computing Nash equilibria of normal form games. Since NASH is a well-known PPAD-complete problem, this framework has potential applications to other $PPAD$ problems. The correctness of this framework has been empirically validated on 4 well-studied 2x2 games: Prisoner's Dilemma, Stag Hunt, Battle, and Chicken. In this paper, we provide the asymptotic time-complexities for these methods and in particular, verify that for 2x2 games the worst-case complexity is linear in the number of actions an agent can choose from.
△ Less
Submitted 26 October, 2021;
originally announced October 2021.
-
Detecting Fake News Spreaders in Social Networks using Inductive Representation Learning
Authors:
Bhavtosh Rath,
Aadesh Salecha,
Jaideep Srivastava
Abstract:
An important aspect of preventing fake news dissemination is to proactively detect the likelihood of its spreading. Research in the domain of fake news spreader detection has not been explored much from a network analysis perspective. In this paper, we propose a graph neural network based approach to identify nodes that are likely to become spreaders of false information. Using the community healt…
▽ More
An important aspect of preventing fake news dissemination is to proactively detect the likelihood of its spreading. Research in the domain of fake news spreader detection has not been explored much from a network analysis perspective. In this paper, we propose a graph neural network based approach to identify nodes that are likely to become spreaders of false information. Using the community health assessment model and interpersonal trust we propose an inductive representation learning framework to predict nodes of densely-connected community structures that are most likely to spread fake news, thus making the entire community vulnerable to the infection. Using topology and interaction based trust properties of nodes in real-world Twitter networks, we are able to predict false information spreaders with an accuracy of over 90%.
△ Less
Submitted 21 November, 2020;
originally announced November 2020.
-
Emergence and Stability of Self-Evolved Cooperative Strategies using Stochastic Machines
Authors:
** Hong Kuan,
Aadesh Salecha
Abstract:
To investigate the origin of cooperative behaviors, we developed an evolutionary model of sequential strategies and tested our model with computer simulations. The sequential strategies represented by stochastic machines were evaluated through games of Iterated Prisoner's Dilemma (IPD) with other agents in the population, allowing co-evolution to occur. We expanded upon past works by proposing a n…
▽ More
To investigate the origin of cooperative behaviors, we developed an evolutionary model of sequential strategies and tested our model with computer simulations. The sequential strategies represented by stochastic machines were evaluated through games of Iterated Prisoner's Dilemma (IPD) with other agents in the population, allowing co-evolution to occur. We expanded upon past works by proposing a novel mechanism to mutate stochastic Moore machines that enables a richer class of machines to be evolved. These machines were then subjected to various selection mechanisms and the resulting evolved strategies were analyzed. We found that cooperation can indeed emerge spontaneously in evolving populations playing iterated PD, specifically in the form of trigger strategies. In addition, we found that the resulting populations converged to evolutionarily stable states and were resilient towards mutation. In order to test the generalizability of our proposed mutation mechanism and simulation approach, we also evolved the machines to play other games such as Chicken, Stag Hunt, and Battle, and obtained strategies that perform as well as mixed strategies in Nash Equilibrium.
△ Less
Submitted 24 October, 2020;
originally announced October 2020.
