-
Learning to Request Guidance in Emergent Communication
Authors:
Benjamin Kolb,
Leon Lang,
Henning Bartsch,
Arwin Gansekoele,
Raymond Koopmanschap,
Leonardo Romor,
David Speck,
Mathijs Mul,
Elia Bruni
Abstract:
Previous research into agent communication has shown that a pre-trained guide can speed up the learning process of an imitation learning agent. The guide achieves this by providing the agent with discrete messages in an emerged language about how to solve the task. We extend this one-directional communication by a one-bit communication channel from the learner back to the guide: It is able to ask…
▽ More
Previous research into agent communication has shown that a pre-trained guide can speed up the learning process of an imitation learning agent. The guide achieves this by providing the agent with discrete messages in an emerged language about how to solve the task. We extend this one-directional communication by a one-bit communication channel from the learner back to the guide: It is able to ask the guide for help, and we limit the guidance by penalizing the learner for these requests. During training, the agent learns to control this gate based on its current observation. We find that the amount of requested guidance decreases over time and guidance is requested in situations of high uncertainty. We investigate the agent's performance in cases of open and closed gates and discuss potential motives for the observed gating behavior.
△ Less
Submitted 11 December, 2019;
originally announced December 2019.
-
Compositionality decomposed: how do neural networks generalise?
Authors:
Dieuwke Hupkes,
Verna Dankers,
Mathijs Mul,
Elia Bruni
Abstract:
Despite a multitude of empirical studies, little consensus exists on whether neural networks are able to generalise compositionally, a controversy that, in part, stems from a lack of agreement about what it means for a neural model to be compositional. As a response to this controversy, we present a set of tests that provide a bridge between, on the one hand, the vast amount of linguistic and phil…
▽ More
Despite a multitude of empirical studies, little consensus exists on whether neural networks are able to generalise compositionally, a controversy that, in part, stems from a lack of agreement about what it means for a neural model to be compositional. As a response to this controversy, we present a set of tests that provide a bridge between, on the one hand, the vast amount of linguistic and philosophical theory about compositionality of language and, on the other, the successful neural models of language. We collect different interpretations of compositionality and translate them into five theoretically grounded tests for models that are formulated on a task-independent level. In particular, we provide tests to investigate (i) if models systematically recombine known parts and rules (ii) if models can extend their predictions beyond the length they have seen in the training data (iii) if models' composition operations are local or global (iv) if models' predictions are robust to synonym substitutions and (v) if models favour rules or exceptions during training. To demonstrate the usefulness of this evaluation paradigm, we instantiate these five tests on a highly compositional data set which we dub PCFG SET and apply the resulting tests to three popular sequence-to-sequence models: a recurrent, a convolution-based and a transformer model. We provide an in-depth analysis of the results, which uncover the strengths and weaknesses of these three architectures and point to potential areas of improvement.
△ Less
Submitted 23 February, 2020; v1 submitted 22 August, 2019;
originally announced August 2019.
-
Mastering emergent language: learning to guide in simulated navigation
Authors:
Mathijs Mul,
Diane Bouchacourt,
Elia Bruni
Abstract:
To cooperate with humans effectively, virtual agents need to be able to understand and execute language instructions. A typical setup to achieve this is with a scripted teacher which guides a virtual agent using language instructions. However, such setup has clear limitations in scalability and, more importantly, it is not interactive. Here, we introduce an autonomous agent that uses discrete comm…
▽ More
To cooperate with humans effectively, virtual agents need to be able to understand and execute language instructions. A typical setup to achieve this is with a scripted teacher which guides a virtual agent using language instructions. However, such setup has clear limitations in scalability and, more importantly, it is not interactive. Here, we introduce an autonomous agent that uses discrete communication to interactively guide other agents to navigate and act on a simulated environment. The developed communication protocol is trainable, emergent and requires no additional supervision. The emergent language speeds up learning of new agents, it generalizes across incrementally more difficult tasks and, contrary to most other emergent languages, it is highly interpretable. We demonstrate how the emitted messages correlate with particular actions and observations, and how new agents become less dependent on this guidance as training progresses. By exploiting the correlations identified in our analysis, we manage to successfully address the agents in their own language.
△ Less
Submitted 14 August, 2019;
originally announced August 2019.
-
Siamese recurrent networks learn first-order logic reasoning and exhibit zero-shot compositional generalization
Authors:
Mathijs Mul,
Willem Zuidema
Abstract:
Can neural nets learn logic? We approach this classic question with current methods, and demonstrate that recurrent neural networks can learn to recognize first order logical entailment relations between expressions. We define an artificial language in first-order predicate logic, generate a large dataset of sample 'sentences', and use an automatic theorem prover to infer the relation between rand…
▽ More
Can neural nets learn logic? We approach this classic question with current methods, and demonstrate that recurrent neural networks can learn to recognize first order logical entailment relations between expressions. We define an artificial language in first-order predicate logic, generate a large dataset of sample 'sentences', and use an automatic theorem prover to infer the relation between random pairs of such sentences. We describe a Siamese neural architecture trained to predict the logical relation, and experiment with recurrent and recursive networks. Siamese Recurrent Networks are surprisingly successful at the entailment recognition task, reaching near perfect performance on novel sentences (consisting of known words), and even outperforming recursive networks. We report a series of experiments to test the ability of the models to perform compositional generalization. In particular, we study how they deal with sentences of unseen length, and sentences containing unseen words. We show that set-ups using LSTMs and GRUs obtain high scores on these tests, demonstrating a form of compositionality.
△ Less
Submitted 1 June, 2019;
originally announced June 2019.
-
Increasing altruistic and cooperative behaviour with simple moral nudges
Authors:
Valerio Capraro,
Glorianna Jagfeld,
Rana Klein,
Mathijs Mul,
Iris van de Pol
Abstract:
The conflict between pro-self and pro-social behaviour is at the core of many key problems of our time, as, for example, the reduction of air pollution and the redistribution of scarce resources. For the well-being of our societies, it is thus crucial to find mechanisms to promote pro-social choices over egoistic ones. Particularly important, because cheap and easy to implement, are those mechanis…
▽ More
The conflict between pro-self and pro-social behaviour is at the core of many key problems of our time, as, for example, the reduction of air pollution and the redistribution of scarce resources. For the well-being of our societies, it is thus crucial to find mechanisms to promote pro-social choices over egoistic ones. Particularly important, because cheap and easy to implement, are those mechanisms that can change people's behaviour without forbidding any options or significantly changing their economic incentives, the so-called "nudges". Previous research has found that moral nudges (e.g., making norms salient) can promote pro-social behaviour. However, little is known about whether their effect persists over time and spills across context. This question is key in light of research showing that pro-social actions are often followed by selfish actions, thus suggesting that some moral manipulations may backfire. Here we present a class of simple moral nudges that have a great positive impact on pro-sociality. In Studies 1-4 (total N = 1,400), we use economic games to demonstrate that asking subjects to self-report "what they think is the morally right thing to do" does not only increase pro-sociality in the choice immediately after, but also in subsequent choices, and even when the social context changes. In Study 5, we explore whether moral nudges promote charity donations to humanitarian organisations in a large (N=1,800) crowdfunding campaign. We find that, in this context, moral nudges increase donations by about 44 percent.
△ Less
Submitted 12 July, 2019; v1 submitted 15 November, 2017;
originally announced November 2017.
