-
Design Considerations for Automatic Musical Soundscapes of Visual Art for People with Blindness or Low Vision
Authors:
Stephen James Krol,
Maria Teresa Llano,
Matthew Butler,
Cagatay Goncu
Abstract:
Music has been identified as a promising medium to enhance the accessibility and experience of visual art for people who are blind or have low vision (BLV). However, composing music and designing soundscapes for visual art is a time-consuming, resource intensive process - limiting its scalability for large exhibitions. In this paper, we investigate the use of automated soundscapes to increase the…
▽ More
Music has been identified as a promising medium to enhance the accessibility and experience of visual art for people who are blind or have low vision (BLV). However, composing music and designing soundscapes for visual art is a time-consuming, resource intensive process - limiting its scalability for large exhibitions. In this paper, we investigate the use of automated soundscapes to increase the accessibility of visual art. We built a prototype system and ran a qualitative study to evaluate the aesthetic experience provided by the automated soundscapes with 10 BLV participants. From the study, we identified a set of design considerations that reveal requirements from BLV people for the development of automated soundscape systems, setting new directions in which creative systems could enrich the aesthetic experience conveyed by these.
△ Less
Submitted 23 May, 2024;
originally announced May 2024.
-
No Longer Trending on Artstation: Prompt Analysis of Generative AI Art
Authors:
Jon McCormack,
Maria Teresa Llano,
Stephen James Krol,
Nina Rajcic
Abstract:
Image generation using generative AI is rapidly becoming a major new source of visual media, with billions of AI generated images created using diffusion models such as Stable Diffusion and Midjourney over the last few years. In this paper we collect and analyse over 3 million prompts and the images they generate. Using natural language processing, topic analysis and visualisation methods we aim t…
▽ More
Image generation using generative AI is rapidly becoming a major new source of visual media, with billions of AI generated images created using diffusion models such as Stable Diffusion and Midjourney over the last few years. In this paper we collect and analyse over 3 million prompts and the images they generate. Using natural language processing, topic analysis and visualisation methods we aim to understand collectively how people are using text prompts, the impact of these systems on artists, and more broadly on the visual cultures they promote. Our study shows that prompting focuses largely on surface aesthetics, reinforcing cultural norms, popular conventional representations and imagery. We also find that many users focus on popular topics (such as making colouring books, fantasy art, or Christmas cards), suggesting that the dominant use for the systems analysed is recreational rather than artistic.
△ Less
Submitted 24 January, 2024;
originally announced January 2024.
-
Creative Discovery using QD Search
Authors:
Jon McCormack,
Camilo Cruz Gambardella,
Stephen James Krol
Abstract:
In creative design, where aesthetics play a crucial role in determining the quality of outcomes, there are often multiple worthwhile possibilities, rather than a single ``best'' design. This challenge is compounded in the use of computational generative systems, where the sheer number of potential outcomes can be overwhelming. This paper introduces a method that combines evolutionary optimisation…
▽ More
In creative design, where aesthetics play a crucial role in determining the quality of outcomes, there are often multiple worthwhile possibilities, rather than a single ``best'' design. This challenge is compounded in the use of computational generative systems, where the sheer number of potential outcomes can be overwhelming. This paper introduces a method that combines evolutionary optimisation with AI-based image classification to perform quality-diversity search, allowing for the creative exploration of complex design spaces. The process begins by randomly sampling the genotype space, followed by map** the generated phenotypes to a reduced representation of the solution space, as well as evaluating them based on their visual characteristics. This results in an elite group of diverse outcomes that span the solution space. The elite is then progressively updated via sampling and simple mutation. We tested our method on a generative system that produces abstract drawings. The results demonstrate that the system can effectively evolve populations of phenotypes with high aesthetic value and greater visual diversity compared to traditional optimisation-focused evolutionary approaches.
△ Less
Submitted 8 May, 2023;
originally announced May 2023.
-
Is Writing Prompts Really Making Art?
Authors:
Jon McCormack,
Camilo Cruz Gambardella,
Nina Rajcic,
Stephen James Krol,
Maria Teresa Llano,
Meng Yang
Abstract:
In recent years Generative Machine Learning systems have advanced significantly. A current wave of generative systems use text prompts to create complex imagery, video, even 3D datasets. The creators of these systems claim a revolution in bringing creativity and art to anyone who can type a prompt. In this position paper, we question the basis for these claims, dividing our analysis into three are…
▽ More
In recent years Generative Machine Learning systems have advanced significantly. A current wave of generative systems use text prompts to create complex imagery, video, even 3D datasets. The creators of these systems claim a revolution in bringing creativity and art to anyone who can type a prompt. In this position paper, we question the basis for these claims, dividing our analysis into three areas: the limitations of linguistic descriptions, implications of the dataset, and lastly, matters of materiality and embodiment. We conclude with an analysis of the creative possibilities enabled by prompt-based systems, asking if they can be considered a new artistic medium.
△ Less
Submitted 1 February, 2023; v1 submitted 25 January, 2023;
originally announced January 2023.
-
Towards the Generation of Musical Explanations with GPT-3
Authors:
Stephen James Krol,
Maria Teresa Llano,
Jon McCormack
Abstract:
Open AI's language model, GPT-3, has shown great potential for many NLP tasks, with applications in many different domains. In this work we carry out a first study on GPT-3's capability to communicate musical decisions through textual explanations when prompted with a textual representation of a piece of music. Enabling a dialogue in human-AI music partnerships is an important step towards more en…
▽ More
Open AI's language model, GPT-3, has shown great potential for many NLP tasks, with applications in many different domains. In this work we carry out a first study on GPT-3's capability to communicate musical decisions through textual explanations when prompted with a textual representation of a piece of music. Enabling a dialogue in human-AI music partnerships is an important step towards more engaging and creative human-AI interactions. Our results show that GPT-3 lacks the necessary intelligence to really understand musical decisions. A major barrier to reach a better performance is the lack of data that includes explanations of the creative process carried out by artists for musical pieces. We believe such a resource would aid the understanding and collaboration with AI music systems.
△ Less
Submitted 11 May, 2022;
originally announced June 2022.
