-
PIZZA: A new benchmark for complex end-to-end task-oriented parsing
Authors:
Konstantine Arkoudas,
Nicolas Guenon des Mesnards,
Melanie Rubino,
Sandesh Swamy,
Saarthak Khanna,
Weiqi Sun,
Khan Haidar
Abstract:
Much recent work in task-oriented parsing has focused on finding a middle ground between flat slots and intents, which are inexpressive but easy to annotate, and powerful representations such as the lambda calculus, which are expressive but costly to annotate. This paper continues the exploration of task-oriented parsing by introducing a new dataset for parsing pizza and drink orders, whose semant…
▽ More
Much recent work in task-oriented parsing has focused on finding a middle ground between flat slots and intents, which are inexpressive but easy to annotate, and powerful representations such as the lambda calculus, which are expressive but costly to annotate. This paper continues the exploration of task-oriented parsing by introducing a new dataset for parsing pizza and drink orders, whose semantics cannot be captured by flat slots and intents. We perform an extensive evaluation of deep-learning techniques for task-oriented parsing on this dataset, including different flavors of seq2seq systems and RNNGs. The dataset comes in two main versions, one in a recently introduced utterance-level hierarchical notation that we call TOP, and one whose targets are executable representations (EXR). We demonstrate empirically that training the parser to directly generate EXR notation not only solves the problem of entity resolution in one fell swoop and overcomes a number of expressive limitations of TOP notation, but also results in significantly greater parsing accuracy.
△ Less
Submitted 30 November, 2022;
originally announced December 2022.
-
Cross-TOP: Zero-Shot Cross-Schema Task-Oriented Parsing
Authors:
Melanie Rubino,
Nicolas Guenon des Mesnards,
Uday Shah,
Nanjiang Jiang,
Weiqi Sun,
Konstantine Arkoudas
Abstract:
Deep learning methods have enabled task-oriented semantic parsing of increasingly complex utterances. However, a single model is still typically trained and deployed for each task separately, requiring labeled training data for each, which makes it challenging to support new tasks, even within a single business vertical (e.g., food-ordering or travel booking). In this paper we describe Cross-TOP (…
▽ More
Deep learning methods have enabled task-oriented semantic parsing of increasingly complex utterances. However, a single model is still typically trained and deployed for each task separately, requiring labeled training data for each, which makes it challenging to support new tasks, even within a single business vertical (e.g., food-ordering or travel booking). In this paper we describe Cross-TOP (Cross-Schema Task-Oriented Parsing), a zero-shot method for complex semantic parsing in a given vertical. By leveraging the fact that user requests from the same vertical share lexical and semantic similarities, a single cross-schema parser is trained to service an arbitrary number of tasks, seen or unseen, within a vertical. We show that Cross-TOP can achieve high accuracy on a previously unseen task without requiring any additional training data, thereby providing a scalable way to bootstrap semantic parsers for new tasks. As part of this work we release the FoodOrdering dataset, a task-oriented parsing dataset in the food-ordering vertical, with utterances and annotations derived from five schemas, each from a different restaurant menu.
△ Less
Submitted 10 June, 2022;
originally announced June 2022.
-
Training Naturalized Semantic Parsers with Very Little Data
Authors:
Subendhu Rongali,
Konstantine Arkoudas,
Melanie Rubino,
Wael Hamza
Abstract:
Semantic parsing is an important NLP problem, particularly for voice assistants such as Alexa and Google Assistant. State-of-the-art (SOTA) semantic parsers are seq2seq architectures based on large language models that have been pretrained on vast amounts of text. To better leverage that pretraining, recent work has explored a reformulation of semantic parsing whereby the output sequences are them…
▽ More
Semantic parsing is an important NLP problem, particularly for voice assistants such as Alexa and Google Assistant. State-of-the-art (SOTA) semantic parsers are seq2seq architectures based on large language models that have been pretrained on vast amounts of text. To better leverage that pretraining, recent work has explored a reformulation of semantic parsing whereby the output sequences are themselves natural language sentences, but in a controlled fragment of natural language. This approach delivers strong results, particularly for few-shot semantic parsing, which is of key importance in practice and the focus of our paper. We push this line of work forward by introducing an automated methodology that delivers very significant additional improvements by utilizing modest amounts of unannotated data, which is typically easy to obtain. Our method is based on a novel synthesis of four techniques: joint training with auxiliary unsupervised tasks; constrained decoding; self-training; and paraphrasing. We show that this method delivers new SOTA few-shot performance on the Overnight dataset, particularly in very low-resource settings, and very compelling few-shot results on a new semantic parsing dataset.
△ Less
Submitted 4 May, 2022; v1 submitted 29 April, 2022;
originally announced April 2022.
-
Unfreeze with Care: Space-Efficient Fine-Tuning of Semantic Parsing Models
Authors:
Weiqi Sun,
Haidar Khan,
Nicolas Guenon des Mesnards,
Melanie Rubino,
Konstantine Arkoudas
Abstract:
Semantic parsing is a key NLP task that maps natural language to structured meaning representations. As in many other NLP tasks, SOTA performance in semantic parsing is now attained by fine-tuning a large pretrained language model (PLM). While effective, this approach is inefficient in the presence of multiple downstream tasks, as a new set of values for all parameters of the PLM needs to be store…
▽ More
Semantic parsing is a key NLP task that maps natural language to structured meaning representations. As in many other NLP tasks, SOTA performance in semantic parsing is now attained by fine-tuning a large pretrained language model (PLM). While effective, this approach is inefficient in the presence of multiple downstream tasks, as a new set of values for all parameters of the PLM needs to be stored for each task separately. Recent work has explored methods for adapting PLMs to downstream tasks while kee** most (or all) of their parameters frozen. We examine two such promising techniques, prefix tuning and bias-term tuning, specifically on semantic parsing. We compare them against each other on two different semantic parsing datasets, and we also compare them against full and partial fine-tuning, both in few-shot and conventional data settings. While prefix tuning is shown to do poorly for semantic parsing tasks off the shelf, we modify it by adding special token embeddings, which results in very strong performance without compromising parameter savings.
△ Less
Submitted 4 March, 2022;
originally announced March 2022.
-
Detectability of large-scale counter-rotating stellar disks in galaxies with integral-field spectroscopy
Authors:
M. Rubino,
A. Pizzella,
L. Morelli,
L. Coccato,
E. Portaluri,
V. P. Debattista,
E. M. Corsini,
E. Dalla Bontà
Abstract:
In recent years integral-field spectroscopic surveys have revealed that the presence of kinematically decoupled stellar components is not a rare phenomenon in nearby galaxies. However, complete statistics are still lacking because they depend on the detection limit of these objects. We investigate the kinematic signatures of two large-scale counter-rotating stellar disks in mock integral-field spe…
▽ More
In recent years integral-field spectroscopic surveys have revealed that the presence of kinematically decoupled stellar components is not a rare phenomenon in nearby galaxies. However, complete statistics are still lacking because they depend on the detection limit of these objects. We investigate the kinematic signatures of two large-scale counter-rotating stellar disks in mock integral-field spectroscopic data to address their detection limits as a function of the galaxy properties and instrumental setup. We built a set of mock data of two large-scale counter-rotating stellar disks as if they were observed with the Multi-Unit Spectroscopic Explorer (MUSE). We accounted for different photometric, kinematic, and stellar population properties of the two counter-rotating components as a function of galaxy inclination. We extracted the stellar kinematics in the wavelength region of the calcium triplet absorption lines by adopting a Gauss-Hermite (GH) parameterization of the line-of-sight velocity distribution (LOSVD). We confirm that the strongest signature of the presence of two counter-rotating stellar disks is the symmetric double peak in the velocity dispersion map, already known as the $2σ$ feature. The size, shape, and slope of the 2$σ$ peak strongly depend on the velocity separation and relative light contribution of the two counter-rotating stellar disks. When the $2σ$ peak is difficult to detect due to the low signal-to-noise ratio of the data, the large-scale structure in the $h_3$ map can be used as a diagnostic for strong and weak counter-rotation. The counter-rotating kinematic signatures become fainter at lower viewing angles as an effect of the smaller projected velocity separation between the two counter-rotating components. We confirm that the observed frequency of $2σ$ galaxies represents only a lower limit of the stellar counter-rotation phenomenon.
△ Less
Submitted 25 July, 2021; v1 submitted 5 July, 2021;
originally announced July 2021.
-
Dynamical structure of small bulges reveals their early formation in ΛCDM paradigm
Authors:
Luca Costantin,
Jairo Méndez-Abreu,
Enrico M. Corsini,
Lorenzo Morelli,
Adriana de Lorenzo-Cáceres,
Ilaria Pagotto,
Virginia Cuomo,
J. Alfonso L. Aguerri,
Michela Rubino
Abstract:
The Λ cold dark matter (ΛCDM) paradigm of galaxy formation predicts that dense spheroidal stellar structures invariably grow at early cosmic time. These primordial spheroids evolve toward a virialized dynamical status as they finally become today's elliptical galaxies and large bulges at the center of disk galaxies. However, observations reveal that small bulges in spiral galaxies are common in th…
▽ More
The Λ cold dark matter (ΛCDM) paradigm of galaxy formation predicts that dense spheroidal stellar structures invariably grow at early cosmic time. These primordial spheroids evolve toward a virialized dynamical status as they finally become today's elliptical galaxies and large bulges at the center of disk galaxies. However, observations reveal that small bulges in spiral galaxies are common in the nearby universe. The prevailing belief that all small bulges form at later times from internal processes occurring in the disk represents a challenge for the ΛCDM scenario. Notably, the coevolution of bulges and central supermassive black holes (SMBHs) at early phases of galaxy evolution is also at stake. However, observations have so far not provided conclusive evidence against their possible early origin. Here, we report new observations of small bulges showing that they follow the mass-velocity dispersion relation expected for virialized systems. Contrary to previous claims, small bulges bridge the gap between massive ellipticals and globular clusters. This dynamical picture supports a scenario where systems over seven orders of magnitude in stellar mass form at early cosmic time. These results alleviate the tension between ΛCDM simulations and observations at galactic scales. We hypothesize that these small bulges are actually the low-mass descendants of compact objects observed at high redshift, also known as red nuggets, which are consistently produced in cosmological ΛCDM simulations. Therefore, this also suggests that the established coevolution of SMBHs and large bulges naturally extends to spheroids in the low-mass regime.
△ Less
Submitted 14 January, 2020;
originally announced January 2020.
