-
Using LLMs to Model the Beliefs and Preferences of Targeted Populations
Authors:
Keiichi Namikoshi,
Alex Filipowicz,
David A. Shamma,
Rumen Iliev,
Candice L. Hogan,
Nikos Arechiga
Abstract:
We consider the problem of aligning a large language model (LLM) to model the preferences of a human population. Modeling the beliefs, preferences, and behaviors of a specific population can be useful for a variety of different applications, such as conducting simulated focus groups for new products, conducting virtual surveys, and testing behavioral interventions, especially for interventions tha…
▽ More
We consider the problem of aligning a large language model (LLM) to model the preferences of a human population. Modeling the beliefs, preferences, and behaviors of a specific population can be useful for a variety of different applications, such as conducting simulated focus groups for new products, conducting virtual surveys, and testing behavioral interventions, especially for interventions that are expensive, impractical, or unethical. Existing work has had mixed success using LLMs to accurately model human behavior in different contexts. We benchmark and evaluate two well-known fine-tuning approaches and evaluate the resulting populations on their ability to match the preferences of real human respondents on a survey of preferences for battery electric vehicles (BEVs). We evaluate our models against their ability to match population-wide statistics as well as their ability to match individual responses, and we investigate the role of temperature in controlling the trade-offs between these two. Additionally, we propose and evaluate a novel loss term to improve model performance on responses that require a numeric response.
△ Less
Submitted 29 March, 2024;
originally announced March 2024.
-
Visual Elements and Cognitive Biases Influence Interpretations of Trends in Scatter Plots
Authors:
Alexandre Filipowicz,
Scott Carter,
Nayeli Bravo,
Rumen Iliev,
Shabnam Hakimi,
David Ayman Shamma,
Kent Lyons,
Candice Hogan,
Charlene Wu
Abstract:
Visualizations are common methods to convey information but also increasingly used to spread misinformation. It is therefore important to understand the factors people use to interpret visualizations. In this paper, we focus on factors that influence interpretations of scatter plots, investigating the extent to which common visual aspects of scatter plots (outliers and trend lines) and cognitive b…
▽ More
Visualizations are common methods to convey information but also increasingly used to spread misinformation. It is therefore important to understand the factors people use to interpret visualizations. In this paper, we focus on factors that influence interpretations of scatter plots, investigating the extent to which common visual aspects of scatter plots (outliers and trend lines) and cognitive biases (people's beliefs) influence perception of correlation trends. We highlight three main findings: outliers skew trend perception but exert less influence than other points; trend lines make trends seem stronger but also mitigate the influence of some outliers; and people's beliefs have a small influence on perceptions of weak, but not strong correlations. From these results we derive guidelines for adjusting visual elements to mitigate the influence of factors that distort interpretations of scatter plots. We explore how these guidelines may generalize to other visualization types and make recommendations for future studies.
△ Less
Submitted 23 October, 2023;
originally announced October 2023.
-
Understanding and Shifting Preferences for Battery Electric Vehicles
Authors:
Nikos Arechiga,
Francine Chen,
Rumen Iliev,
Emily Sumner,
Scott Carter,
Alex Filipowicz,
Nayeli Bravo,
Monica Van,
Kate Glazko,
Kalani Murakami,
Laurent Denoue,
Candice Hogan,
Katharine Sieck,
Charlene Wu,
Kent Lyons
Abstract:
Identifying personalized interventions for an individual is an important task. Recent work has shown that interventions that do not consider the demographic background of individual consumers can, in fact, produce the reverse effect, strengthening opposition to electric vehicles. In this work, we focus on methods for personalizing interventions based on an individual's demographics to shift the pr…
▽ More
Identifying personalized interventions for an individual is an important task. Recent work has shown that interventions that do not consider the demographic background of individual consumers can, in fact, produce the reverse effect, strengthening opposition to electric vehicles. In this work, we focus on methods for personalizing interventions based on an individual's demographics to shift the preferences of consumers to be more positive towards Battery Electric Vehicles (BEVs). One of the constraints in building models to suggest interventions for shifting preferences is that each intervention can influence the effectiveness of later interventions. This, in turn, requires many subjects to evaluate effectiveness of each possible intervention. To address this, we propose to identify personalized factors influencing BEV adoption, such as barriers and motivators. We present a method for predicting these factors and show that the performance is better than always predicting the most frequent factors. We then present a Reinforcement Learning (RL) model that learns the most effective interventions, and compare the number of subjects required for each approach.
△ Less
Submitted 8 February, 2022;
originally announced February 2022.
-
Accelerating Understanding of Scientific Experiments with End to End Symbolic Regression
Authors:
Nikos Arechiga,
Francine Chen,
Yan-Ying Chen,
Yanxia Zhang,
Rumen Iliev,
Heishiro Toyoda,
Kent Lyons
Abstract:
We consider the problem of learning free-form symbolic expressions from raw data, such as that produced by an experiment in any scientific domain. Accurate and interpretable models of scientific phenomena are the cornerstone of scientific research. Simple yet interpretable models, such as linear or logistic regression and decision trees often lack predictive accuracy. Alternatively, accurate black…
▽ More
We consider the problem of learning free-form symbolic expressions from raw data, such as that produced by an experiment in any scientific domain. Accurate and interpretable models of scientific phenomena are the cornerstone of scientific research. Simple yet interpretable models, such as linear or logistic regression and decision trees often lack predictive accuracy. Alternatively, accurate blackbox models such as deep neural networks provide high predictive accuracy, but do not readily admit human understanding in a way that would enrich the scientific theory of the phenomenon. Many great breakthroughs in science revolve around the development of parsimonious equational models with high predictive accuracy, such as Newton's laws, universal gravitation, and Maxwell's equations. Previous work on automating the search of equational models from data combine domain-specific heuristics as well as computationally expensive techniques, such as genetic programming and Monte-Carlo search. We develop a deep neural network (MACSYMA) to address the symbolic regression problem as an end-to-end supervised learning problem. MACSYMA can generate symbolic expressions that describe a dataset. The computational complexity of the task is reduced to the feedforward computation of a neural network. We train our neural network on a synthetic dataset consisting of data tables of varying length and varying levels of noise, for which the neural network must learn to produce the correct symbolic expression token by token. Finally, we validate our technique by running on a public dataset from behavioral science.
△ Less
Submitted 7 December, 2021;
originally announced December 2021.
-
Machine learning reveals how personalized climate communication can both succeed and backfire
Authors:
Totte Harinen,
Alexandre Filipowicz,
Shabnam Hakimi,
Rumen Iliev,
Matthew Klenk,
Emily Sumner
Abstract:
Different advertising messages work for different people. Machine learning can be an effective way to personalise climate communications. In this paper we use machine learning to reanalyse findings from a recent study, showing that online advertisements increased some people's belief in climate change while resulting in decreased belief in others. In particular, we show that the effect of the adve…
▽ More
Different advertising messages work for different people. Machine learning can be an effective way to personalise climate communications. In this paper we use machine learning to reanalyse findings from a recent study, showing that online advertisements increased some people's belief in climate change while resulting in decreased belief in others. In particular, we show that the effect of the advertisements could change depending on people's age and ethnicity.
△ Less
Submitted 10 September, 2021;
originally announced September 2021.
