-
Is Temperature the Creativity Parameter of Large Language Models?
Authors:
Max Peeperkorn,
Tom Kouwenhoven,
Dan Brown,
Anna Jordanous
Abstract:
Large language models (LLMs) are applied to all sorts of creative tasks, and their outputs vary from beautiful, to peculiar, to pastiche, into plain plagiarism. The temperature parameter of an LLM regulates the amount of randomness, leading to more diverse outputs; therefore, it is often claimed to be the creativity parameter. Here, we investigate this claim using a narrative generation task with…
▽ More
Large language models (LLMs) are applied to all sorts of creative tasks, and their outputs vary from beautiful, to peculiar, to pastiche, into plain plagiarism. The temperature parameter of an LLM regulates the amount of randomness, leading to more diverse outputs; therefore, it is often claimed to be the creativity parameter. Here, we investigate this claim using a narrative generation task with a predetermined fixed context, model and prompt. Specifically, we present an empirical analysis of the LLM output for different temperature values using four necessary conditions for creativity in narrative generation: novelty, typicality, cohesion, and coherence. We find that temperature is weakly correlated with novelty, and unsurprisingly, moderately correlated with incoherence, but there is no relationship with either cohesion or typicality. However, the influence of temperature on creativity is far more nuanced and weak than suggested by the "creativity parameter" claim; overall results suggest that the LLM generates slightly more novel outputs as temperatures get higher. Finally, we discuss ideas to allow more controlled LLM creativity, rather than relying on chance via changing the temperature parameter.
△ Less
Submitted 1 May, 2024;
originally announced May 2024.
-
Modelling Creativity: Identifying Key Components through a Corpus-Based Approach
Authors:
Anna Jordanous,
Bill Keller
Abstract:
Creativity is a complex, multi-faceted concept encompassing a variety of related aspects, abilities, properties and behaviours. If we wish to study creativity scientifically, then a tractable and well-articulated model of creativity is required. Such a model would be of great value to researchers investigating the nature of creativity and in particular, those concerned with the evaluation of creat…
▽ More
Creativity is a complex, multi-faceted concept encompassing a variety of related aspects, abilities, properties and behaviours. If we wish to study creativity scientifically, then a tractable and well-articulated model of creativity is required. Such a model would be of great value to researchers investigating the nature of creativity and in particular, those concerned with the evaluation of creative practice. This paper describes a unique approach to develo** a suitable model of how creative behaviour emerges that is based on the words people use to describe the concept. Using techniques from the field of statistical natural language processing, we identify a collection of fourteen key components of creativity through an analysis of a corpus of academic papers on the topic. Words are identified which appear significantly often in connection with discussions of the concept. Using a measure of lexical similarity to help cluster these words, a number of distinct themes emerge, which collectively contribute to a comprehensive and multi-perspective model of creativity. The components provide an ontology of creativity: a set of building blocks which can be used to model creative practice in a variety of domains. The components have been employed in two case studies to evaluate the creativity of computational systems and have proven useful in articulating achievements of this work and directions for further research.
△ Less
Submitted 12 September, 2016;
originally announced September 2016.
-
Computer-Assisted Processing of Intertextuality in Ancient Languages
Authors:
Mark Hedges,
Anna Jordanous,
K. Faith Lawrence,
Charlotte Roueché,
Charlotte Tupman
Abstract:
The production of digital critical editions of texts using TEI is now a widely-adopted procedure within digital humanities. The work described in this paper extends this approach to the publication of gnomologia (anthologies of wise sayings), which formed a widespread literary genre in many cultures of the medieval Mediterranean. These texts are challenging because they were rarely copied straight…
▽ More
The production of digital critical editions of texts using TEI is now a widely-adopted procedure within digital humanities. The work described in this paper extends this approach to the publication of gnomologia (anthologies of wise sayings), which formed a widespread literary genre in many cultures of the medieval Mediterranean. These texts are challenging because they were rarely copied straightforwardly; rather, sayings were selected, reorganised, modified or re-attributed between manuscripts, resulting in a highly interconnected corpus for which a standard approach to digital publication is insufficient. Focusing on Greek and Arabic collections, we address this challenge using semantic web techniques to create an ecosystem of texts, relationships and annotations, and consider a new model - organic, collaborative, interconnected, and open-ended - of what constitutes an edition. This semantic web-based approach allows scholars to add their own materials and annotations to the network of information and to explore the conceptual networks that arise from these interconnected sayings.
△ Less
Submitted 1 January, 2017; v1 submitted 31 January, 2016;
originally announced February 2016.
-
Implementing feedback in creative systems: A workshop approach
Authors:
Joseph Corneli,
Anna Jordanous
Abstract:
One particular challenge in AI is the computational modelling and simulation of creativity. Feedback and learning from experience are key aspects of the creative process. Here we investigate how we could implement feedback in creative systems using a social model. From the field of creative writing we borrow the concept of a Writers Workshop as a model for learning through feedback. The Writers Wo…
▽ More
One particular challenge in AI is the computational modelling and simulation of creativity. Feedback and learning from experience are key aspects of the creative process. Here we investigate how we could implement feedback in creative systems using a social model. From the field of creative writing we borrow the concept of a Writers Workshop as a model for learning through feedback. The Writers Workshop encourages examination, discussion and debates of a piece of creative work using a prescribed format of activities. We propose a computational model of the Writers Workshop as a roadmap for incorporation of feedback in artificial creativity systems. We argue that the Writers Workshop setting describes the anatomy of the creative process. We support our claim with a case study that describes how to implement the Writers Workshop model in a computational creativity system. We present this work using patterns other people can follow to implement similar designs in their own systems. We conclude by discussing the broader relevance of this model to other aspects of AI.
△ Less
Submitted 26 May, 2015;
originally announced May 2015.
-
Modelling serendipity in a computational context
Authors:
Joseph Corneli,
Anna Jordanous,
Christian Guckelsberger,
Alison Pease,
Simon Colton
Abstract:
The term serendipity describes a creative process that develops, in context, with the active participation of a creative agent, but not entirely within that agent's control. While a system cannot be made to perform serendipitously on demand, we argue that its $\mathit{serendipity\ potential}$ can be increased by means of a suitable system architecture and other design choices. We distil a unified…
▽ More
The term serendipity describes a creative process that develops, in context, with the active participation of a creative agent, but not entirely within that agent's control. While a system cannot be made to perform serendipitously on demand, we argue that its $\mathit{serendipity\ potential}$ can be increased by means of a suitable system architecture and other design choices. We distil a unified description of serendipitous occurrences from historical theorisations of serendipity and creativity. This takes the form of a framework with six phases: $\mathit{perception}$, $\mathit{attention}$, $\mathit{interest}$, $\mathit{explanation}$, $\mathit{bridge}$, and $\mathit{valuation}$. We then use this framework to organise a survey of literature in cognitive science, philosophy, and computing, which yields practical definitions of the six phases, along with heuristics for implementation. We use the resulting model to evaluate the serendipity potential of four existing systems developed by others, and two systems previously developed by two of the authors. Most existing research that considers serendipity in a computing context deals with serendipity as a service; here we relate theories of serendipity to the development of autonomous systems and computational creativity practice. We argue that serendipity is not teleologically blind, and outline representative directions for future applications of our model. We conclude that it is feasible to equip computational systems with the potential for serendipity, and that this could be beneficial in varied computational creativity/AI applications, particularly those designed to operate responsively in real-world contexts.
△ Less
Submitted 19 April, 2020; v1 submitted 3 November, 2014;
originally announced November 2014.
