Conditional Generation and Snapshot Learning in Neural Dialogue Systems
Authors:
Tsung-Hsien Wen,
Milica Gasic,
Nikola Mrksic,
Lina M. Rojas-Barahona,
Pei-Hao Su,
Stefan Ultes,
David Vandyke,
Steve Young
Abstract:
Recently a variety of LSTM-based conditional language models (LM) have been applied across a range of language generation tasks. In this work we study various model architectures and different ways to represent and aggregate the source information in an end-to-end neural dialogue system framework. A method called snapshot learning is also proposed to facilitate learning from supervised sequential…
▽ More
Recently a variety of LSTM-based conditional language models (LM) have been applied across a range of language generation tasks. In this work we study various model architectures and different ways to represent and aggregate the source information in an end-to-end neural dialogue system framework. A method called snapshot learning is also proposed to facilitate learning from supervised sequential signals by applying a companion cross-entropy objective function to the conditioning vector. The experimental and analytical results demonstrate firstly that competition occurs between the conditioning vector and the LM, and the differing architectures provide different trade-offs between the two. Secondly, the discriminative power and transparency of the conditioning vector is key to providing both model interpretability and better performance. Thirdly, snapshot learning leads to consistent performance improvements independent of which architecture is used.
△ Less
Submitted 10 June, 2016;
originally announced June 2016.
A Network-based End-to-End Trainable Task-oriented Dialogue System
Authors:
Tsung-Hsien Wen,
David Vandyke,
Nikola Mrksic,
Milica Gasic,
Lina M. Rojas-Barahona,
Pei-Hao Su,
Stefan Ultes,
Steve Young
Abstract:
Teaching machines to accomplish tasks by conversing naturally with humans is challenging. Currently, develo** task-oriented dialogue systems requires creating multiple components and typically this involves either a large amount of handcrafting, or acquiring costly labelled datasets to solve a statistical learning problem for each component. In this work we introduce a neural network-based text-…
▽ More
Teaching machines to accomplish tasks by conversing naturally with humans is challenging. Currently, develo** task-oriented dialogue systems requires creating multiple components and typically this involves either a large amount of handcrafting, or acquiring costly labelled datasets to solve a statistical learning problem for each component. In this work we introduce a neural network-based text-in, text-out end-to-end trainable goal-oriented dialogue system along with a new way of collecting dialogue data based on a novel pipe-lined Wizard-of-Oz framework. This approach allows us to develop dialogue systems easily and without making too many assumptions about the task at hand. The results show that the model can converse with human subjects naturally whilst hel** them to accomplish tasks in a restaurant search domain.
△ Less
Submitted 24 April, 2017; v1 submitted 15 April, 2016;
originally announced April 2016.
