-
Speaking Your Language: Spatial Relationships in Interpretable Emergent Communication
Authors:
Olaf Lipinski,
Adam J. Sobey,
Federico Cerutti,
Timothy J. Norman
Abstract:
Effective communication requires the ability to refer to specific parts of an observation in relation to others. While emergent communication literature shows success in develo** various language properties, no research has shown the emergence of such positional references. This paper demonstrates how agents can communicate about spatial relationships within their observations. The results indic…
▽ More
Effective communication requires the ability to refer to specific parts of an observation in relation to others. While emergent communication literature shows success in develo** various language properties, no research has shown the emergence of such positional references. This paper demonstrates how agents can communicate about spatial relationships within their observations. The results indicate that agents can develop a language capable of expressing the relationships between parts of their observation, achieving over 90% accuracy when trained in a referential game which requires such communication. Using a collocation measure, we demonstrate how the agents create such references. This analysis suggests that agents use a mixture of non-compositional and compositional messages to convey spatial relationships. We also show that the emergent language is interpretable by humans. The translation accuracy is tested by communicating with the receiver agent, where the receiver achieves over 78% accuracy using parts of this lexicon, confirming that the interpretation of the emergent language was successful.
△ Less
Submitted 11 June, 2024;
originally announced June 2024.
-
Agent based modelling for continuously varying supply chains
Authors:
Wan Wang,
Haiyan Wang,
Adam J. Sobey
Abstract:
Problem definition: Supply chains are constantly evolving networks. Reinforcement learning is increasingly proposed as a solution to provide optimal control of these networks. Academic/practical: However, learning in continuously varying environments remains a challenge in the reinforcement learning literature.Methodology: This paper therefore seeks to address whether agents can control varying su…
▽ More
Problem definition: Supply chains are constantly evolving networks. Reinforcement learning is increasingly proposed as a solution to provide optimal control of these networks. Academic/practical: However, learning in continuously varying environments remains a challenge in the reinforcement learning literature.Methodology: This paper therefore seeks to address whether agents can control varying supply chain problems, transferring learning between environments that require different strategies and avoiding catastrophic forgetting of tasks that have not been seen in a while. To evaluate this approach, two state-of-the-art Reinforcement Learning (RL) algorithms are compared: an actor-critic learner, Proximal Policy Optimisation(PPO), and a Recurrent Proximal Policy Optimisation (RPPO), PPO with a Long Short-Term Memory(LSTM) layer, which is showing popularity in online learning environments. Results: First these methods are compared on six sets of environments with varying degrees of stochasticity. The results show that more lean strategies adopted in Batch environments are different from those adopted in Stochastic environments with varying products. The methods are also compared on various continuous supply chain scenarios, where the PPO agents are shown to be able to adapt through continuous learning when the tasks are similar but show more volatile performance when changing between the extreme tasks. However, the RPPO, with an ability to remember histories, is able to overcome this to some extent and takes on a more realistic strategy. Managerial implications: Our results provide a new perspective on the continuously varying supply chain, the cooperation and coordination of agents are crucial for improving the overall performance in uncertain and semi-continuous non-stationary supply chain environments without the need to retrain the environment as the demand changes.
△ Less
Submitted 24 December, 2023;
originally announced December 2023.
-
It's About Time: Temporal References in Emergent Communication
Authors:
Olaf Lipinski,
Adam J. Sobey,
Federico Cerutti,
Timothy J. Norman
Abstract:
Emergent communication studies the development of language between autonomous agents, aiming to improve understanding of natural language evolution and increase communication efficiency. While temporal aspects of language have been considered in computational linguistics, there has been no research on temporal references in emergent communication. This paper addresses this gap, by exploring how ag…
▽ More
Emergent communication studies the development of language between autonomous agents, aiming to improve understanding of natural language evolution and increase communication efficiency. While temporal aspects of language have been considered in computational linguistics, there has been no research on temporal references in emergent communication. This paper addresses this gap, by exploring how agents communicate about temporal relationships. We analyse three potential influences for the emergence of temporal references: environmental, external, and architectural changes. Our experiments demonstrate that altering the loss function is insufficient for temporal references to emerge; rather, architectural changes are necessary. However, a minimal change in agent architecture, using a different batching method, allows the emergence of temporal references. This modified design is compared with the standard architecture in a temporal referential games environment, which emphasises temporal relationships. The analysis indicates that over 95\% of the agents with the modified batching method develop temporal references, without changes to their loss function. We consider temporal referencing necessary for future improvements to the agents' communication efficiency, yielding a closer to optimal coding as compared to purely compositional languages. Our readily transferable architectural insights provide the basis for their incorporation into other emergent communication settings.
△ Less
Submitted 3 May, 2024; v1 submitted 10 October, 2023;
originally announced October 2023.
-
The Effect of Epigenetic Blocking on Dynamic Multi-Objective Optimisation Problems
Authors:
Sizhe Yuen,
Thomas H. G. Ezard,
Adam J. Sobey
Abstract:
Hundreds of Evolutionary Computation approaches have been reported. From an evolutionary perspective they focus on two fundamental mechanisms: cultural inheritance in Swarm Intelligence and genetic inheritance in Evolutionary Algorithms. Contemporary evolutionary biology looks beyond genetic inheritance, proposing a so-called ``Extended Evolutionary Synthesis''. Many concepts from the Extended Evo…
▽ More
Hundreds of Evolutionary Computation approaches have been reported. From an evolutionary perspective they focus on two fundamental mechanisms: cultural inheritance in Swarm Intelligence and genetic inheritance in Evolutionary Algorithms. Contemporary evolutionary biology looks beyond genetic inheritance, proposing a so-called ``Extended Evolutionary Synthesis''. Many concepts from the Extended Evolutionary Synthesis have been left out of Evolutionary Computation as interest has moved toward specific implementations of the same general mechanisms. One such concept is epigenetic inheritance, which is increasingly considered central to evolutionary thinking. Epigenetic mechanisms allow quick non- or partially-genetic adaptations to environmental changes. Dynamic multi-objective optimisation problems represent similar circumstances to the natural world where fitness can be determined by multiple objectives (traits), and the environment is constantly changing.
This paper asks if the advantages that epigenetic inheritance provide in the natural world are replicated in dynamic multi-objective optimisation problems. Specifically, an epigenetic blocking mechanism is applied to a state-of-the-art multi-objective genetic algorithm, MOEA/D-DE, and its performance is compared on three sets of dynamic test functions, FDA, JY, and UDF. The mechanism shows improved performance on 12 of the 16 test problems, providing initial evidence that more algorithms should explore the wealth of epigenetic mechanisms seen in the natural world.
△ Less
Submitted 25 November, 2022;
originally announced November 2022.
-
From Intelligent Agents to Trustworthy Human-Centred Multiagent Systems
Authors:
Mohammad Divband Soorati,
Enrico H. Gerding,
Enrico Marchioni,
Pavel Naumov,
Timothy J. Norman,
Sarvapali D. Ramchurn,
Bahar Rastegari,
Adam Sobey,
Sebastian Stein,
Danesh Tarpore,
Vahid Yazdanpanah,
Jie Zhang
Abstract:
The Agents, Interaction and Complexity research group at the University of Southampton has a long track record of research in multiagent systems (MAS). We have made substantial scientific contributions across learning in MAS, game-theoretic techniques for coordinating agent systems, and formal methods for representation and reasoning. We highlight key results achieved by the group and elaborate on…
▽ More
The Agents, Interaction and Complexity research group at the University of Southampton has a long track record of research in multiagent systems (MAS). We have made substantial scientific contributions across learning in MAS, game-theoretic techniques for coordinating agent systems, and formal methods for representation and reasoning. We highlight key results achieved by the group and elaborate on recent work and open research challenges in develo** trustworthy autonomous systems and deploying human-centred AI systems that aim to support societal good.
△ Less
Submitted 5 October, 2022;
originally announced October 2022.
-
Epigenetic opportunities for Evolutionary Computation
Authors:
Sizhe Yuen,
Thomas H. G. Ezard,
Adam J. Sobey
Abstract:
Evolutionary Computation is a group of biologically inspired algorithms used to solve complex optimisation problems. It can be split into Evolutionary Algorithms, which take inspiration from genetic inheritance, and Swarm Intelligence algorithms, that take inspiration from cultural inheritance. However, recent developments have focused on computational or mathematical adaptions, leaving their biol…
▽ More
Evolutionary Computation is a group of biologically inspired algorithms used to solve complex optimisation problems. It can be split into Evolutionary Algorithms, which take inspiration from genetic inheritance, and Swarm Intelligence algorithms, that take inspiration from cultural inheritance. However, recent developments have focused on computational or mathematical adaptions, leaving their biological roots behind. This has left much of the modern evolutionary literature relatively unexplored.
To understand which evolutionary mechanisms have been considered, and which have been overlooked, this paper breaks down successful bio-inspired algorithms under a contemporary biological framework based on the Extended Evolutionary Synthesis, an extension of the classical, genetics focussed, Modern Synthesis. The analysis shows that Darwinism and the Modern Synthesis have been incorporated into Evolutionary Computation but that the Extended Evolutionary Synthesis has been broadly ignored beyond:cultural inheritance, incorporated in the sub-set of Swarm Intelligence algorithms, evolvability, through CMA-ES, and multilevel selection, through Multi-Level Selection Genetic Algorithm.
The framework shows a missing gap in epigenetic inheritance for Evolutionary Computation, despite being a key building block in modern interpretations of how evolution occurs. Epigenetic inheritance can explain fast adaptation, without changes in an individual's genotype, by allowing biological organisms to self-adapt quickly to environmental cues, which, increases the speed of convergence while maintaining stability in changing environments. This leaves a diverse range of biologically inspired mechanisms as low hanging fruit that should be explored further within Evolutionary Computation.
△ Less
Submitted 10 August, 2021;
originally announced August 2021.
-
Automation for Interpretable Machine Learning Through a Comparison of Loss Functions to Regularisers
Authors:
A. I. Parkes,
J. Camilleri,
D. A. Hudson,
A. J. Sobey
Abstract:
To increase the ubiquity of machine learning it needs to be automated. Automation is cost-effective as it allows experts to spend less time tuning the approach, which leads to shorter development times. However, while this automation produces highly accurate architectures, they can be uninterpretable, acting as `black-boxes' which produce low conventional errors but fail to model the underlying in…
▽ More
To increase the ubiquity of machine learning it needs to be automated. Automation is cost-effective as it allows experts to spend less time tuning the approach, which leads to shorter development times. However, while this automation produces highly accurate architectures, they can be uninterpretable, acting as `black-boxes' which produce low conventional errors but fail to model the underlying input-output relationships -- the ground truth. This paper explores the use of the Fit to Median Error measure in machine learning regression automation, using evolutionary computation in order to improve the approximation of the ground truth. When used alongside conventional error measures it improves interpretability by regularising learnt input-output relationships to the conditional median. It is compared to traditional regularisers to illustrate that the use of the Fit to Median Error produces regression neural networks which model more consistent input-output relationships. The problem considered is ship power prediction using a fuel-saving air lubrication system, which is highly stochastic in nature. The networks optimised for their Fit to Median Error are shown to approximate the ground truth more consistently, without sacrificing conventional Minkowski-r error values.
△ Less
Submitted 7 June, 2021;
originally announced June 2021.
-
Lifetime policy reuse and the importance of task capacity
Authors:
David M. Bossens,
Adam J. Sobey
Abstract:
A long-standing challenge in artificial intelligence is lifelong reinforcement learning, where learners are given many tasks in sequence and must transfer knowledge between tasks while avoiding catastrophic forgetting. Policy reuse and other multi-policy reinforcement learning techniques can learn multiple tasks but may generate many policies. This paper presents two novel contributions, namely 1)…
▽ More
A long-standing challenge in artificial intelligence is lifelong reinforcement learning, where learners are given many tasks in sequence and must transfer knowledge between tasks while avoiding catastrophic forgetting. Policy reuse and other multi-policy reinforcement learning techniques can learn multiple tasks but may generate many policies. This paper presents two novel contributions, namely 1) Lifetime Policy Reuse, a model-agnostic policy reuse algorithm that avoids generating many policies by optimising a fixed number of near-optimal policies through a combination of policy optimisation and adaptive policy selection; and 2) the task capacity, a measure for the maximal number of tasks that a policy can accurately solve. Comparing two state-of-the-art base-learners, the results demonstrate the importance of Lifetime Policy Reuse and task capacity based pre-selection on an 18-task partially observable Pacman domain and a Cartpole domain of up to 125 tasks.
△ Less
Submitted 20 October, 2023; v1 submitted 3 June, 2021;
originally announced June 2021.
-
Towards Error Measures which Influence a Learners Inductive Bias to the Ground Truth
Authors:
A. I. Parkes,
A. J. Sobey,
D. A. Hudson
Abstract:
Artificial intelligence is applied in a range of sectors, and is relied upon for decisions requiring a high level of trust. For regression methods, trust is increased if they approximate the true input-output relationships and perform accurately outside the bounds of the training data. But often performance off-test-set is poor, especially when data is sparse. This is because the conditional avera…
▽ More
Artificial intelligence is applied in a range of sectors, and is relied upon for decisions requiring a high level of trust. For regression methods, trust is increased if they approximate the true input-output relationships and perform accurately outside the bounds of the training data. But often performance off-test-set is poor, especially when data is sparse. This is because the conditional average, which in many scenarios is a good approximation of the `ground truth', is only modelled with conventional Minkowski-r error measures when the data set adheres to restrictive assumptions, with many real data sets violating these. To combat this there are several methods that use prior knowledge to approximate the `ground truth'. However, prior knowledge is not always available, and this paper investigates how error measures affect the ability for a regression method to model the `ground truth' in these scenarios. Current error measures are shown to create an unhelpful bias and a new error measure is derived which does not exhibit this behaviour. This is tested on 36 representative data sets with different characteristics, showing that it is more consistent in determining the `ground truth' and in giving improved predictions in regions beyond the range of the training data.
△ Less
Submitted 4 May, 2021;
originally announced May 2021.
-
cMLSGA: A Co-Evolutionary Multi-Level Selection Genetic Algorithm for Multi-Objective Optimization
Authors:
P. A. Grudniewski,
A. J. Sobey
Abstract:
In practical optimisation the dominant characteristics of the problem are often not known prior. Therefore, there is a need to develop general solvers as it is not always possible to tailor a specialised approach to each application. The hybrid form of Multi-Level Selection Genetic Algorithm (MLSGA) already shows good performance on range of problems due to its diversity-first approach, which is r…
▽ More
In practical optimisation the dominant characteristics of the problem are often not known prior. Therefore, there is a need to develop general solvers as it is not always possible to tailor a specialised approach to each application. The hybrid form of Multi-Level Selection Genetic Algorithm (MLSGA) already shows good performance on range of problems due to its diversity-first approach, which is rare among Evolutionary Algorithms. To increase the generality of its performance this paper proposes a distinct set of co-evolutionary mechanisms, which defines co-evolution as competition between collectives rather than individuals. This distinctive approach to co-evolutionary provides less regular communication between sub-populations and different fitness definitions between individuals and collectives. This encourages the collectives to act more independently creating a unique sub-regional search, leading to the development of co-evolutionary MLSGA (cMLSGA). To test this methodology nine genetic algorithms are selected to generate several variants of cMLSGA, which incorporates these approaches at the individual level. The new mechanisms are tested on over 100 different functions and benchmarked against the 9 state-of-the-art competitors in order to find the best general solver. The results show that the diversity of co-evolutionary approaches is more important than their individual performances. This allows the selection of two competing algorithms that improve the generality of cMLSGA, without large loss of performance on any specific problem type. When compared to the state-of-the-art, the proposed methodology is the most universal and robust, leading to an algorithm more likely to solve complex problems with limited knowledge about the search space.
△ Less
Submitted 22 April, 2021;
originally announced April 2021.
