-
Under the Hood of Tabular Data Generation Models: the Strong Impact of Hyperparameter Tuning
Authors:
G. Charbel N. Kindji,
Lina Maria Rojas-Barahona,
Elisa Fromont,
Tanguy Urvoy
Abstract:
We investigate the impact of dataset-specific hyperparameter, feature encoding, and architecture tuning on five recent model families for tabular data generation through an extensive benchmark on 16 datasets. This study addresses the practical need for a unified evaluation of models that fully considers hyperparameter optimization. Additionally, we propose a reduced search space for each model tha…
▽ More
We investigate the impact of dataset-specific hyperparameter, feature encoding, and architecture tuning on five recent model families for tabular data generation through an extensive benchmark on 16 datasets. This study addresses the practical need for a unified evaluation of models that fully considers hyperparameter optimization. Additionally, we propose a reduced search space for each model that allows for quick optimization, achieving nearly equivalent performance at a significantly lower cost.Our benchmark demonstrates that, for most models, large-scale dataset-specific tuning substantially improves performance compared to the original configurations. Furthermore, we confirm that diffusion-based models generally outperform other models on tabular data. However, this advantage is not significant when the entire tuning and training process is restricted to the same GPU budget for all models.
△ Less
Submitted 18 June, 2024;
originally announced June 2024.
-
Conditional Generation and Snapshot Learning in Neural Dialogue Systems
Authors:
Tsung-Hsien Wen,
Milica Gasic,
Nikola Mrksic,
Lina M. Rojas-Barahona,
Pei-Hao Su,
Stefan Ultes,
David Vandyke,
Steve Young
Abstract:
Recently a variety of LSTM-based conditional language models (LM) have been applied across a range of language generation tasks. In this work we study various model architectures and different ways to represent and aggregate the source information in an end-to-end neural dialogue system framework. A method called snapshot learning is also proposed to facilitate learning from supervised sequential…
▽ More
Recently a variety of LSTM-based conditional language models (LM) have been applied across a range of language generation tasks. In this work we study various model architectures and different ways to represent and aggregate the source information in an end-to-end neural dialogue system framework. A method called snapshot learning is also proposed to facilitate learning from supervised sequential signals by applying a companion cross-entropy objective function to the conditioning vector. The experimental and analytical results demonstrate firstly that competition occurs between the conditioning vector and the LM, and the differing architectures provide different trade-offs between the two. Secondly, the discriminative power and transparency of the conditioning vector is key to providing both model interpretability and better performance. Thirdly, snapshot learning leads to consistent performance improvements independent of which architecture is used.
△ Less
Submitted 10 June, 2016;
originally announced June 2016.
-
A Network-based End-to-End Trainable Task-oriented Dialogue System
Authors:
Tsung-Hsien Wen,
David Vandyke,
Nikola Mrksic,
Milica Gasic,
Lina M. Rojas-Barahona,
Pei-Hao Su,
Stefan Ultes,
Steve Young
Abstract:
Teaching machines to accomplish tasks by conversing naturally with humans is challenging. Currently, develo** task-oriented dialogue systems requires creating multiple components and typically this involves either a large amount of handcrafting, or acquiring costly labelled datasets to solve a statistical learning problem for each component. In this work we introduce a neural network-based text-…
▽ More
Teaching machines to accomplish tasks by conversing naturally with humans is challenging. Currently, develo** task-oriented dialogue systems requires creating multiple components and typically this involves either a large amount of handcrafting, or acquiring costly labelled datasets to solve a statistical learning problem for each component. In this work we introduce a neural network-based text-in, text-out end-to-end trainable goal-oriented dialogue system along with a new way of collecting dialogue data based on a novel pipe-lined Wizard-of-Oz framework. This approach allows us to develop dialogue systems easily and without making too many assumptions about the task at hand. The results show that the model can converse with human subjects naturally whilst hel** them to accomplish tasks in a restaurant search domain.
△ Less
Submitted 24 April, 2017; v1 submitted 15 April, 2016;
originally announced April 2016.
