-
An alternating peak-optimization method for optimal trajectory generation of quadrotor drones
Authors:
Wytze A. B. de Vries,
Ming Li,
Qirui Song,
Zhiyong Sun
Abstract:
In this paper, we propose an alternating optimization method to address a time-optimal trajectory generation problem. Different from the existing solutions, our approach introduces a new formulation that minimizes the overall trajectory running time while maintaining the polynomial smoothness constraints and incorporating hard limits on motion derivatives to ensure feasibility. To address this pro…
▽ More
In this paper, we propose an alternating optimization method to address a time-optimal trajectory generation problem. Different from the existing solutions, our approach introduces a new formulation that minimizes the overall trajectory running time while maintaining the polynomial smoothness constraints and incorporating hard limits on motion derivatives to ensure feasibility. To address this problem, an alternating peak-optimization method is developed, which splits the optimization process into two sub-optimizations: the first sub-optimization optimizes polynomial coefficients for smoothness, and the second sub-optimization adjusts the time allocated to each trajectory segment. These are alternated until a feasible minimum-time solution is found. We offer a comprehensive set of simulations and experiments to showcase the superior performance of our approach in comparison to existing methods.
A collection of demonstration videos with real drone flying experiments can be accessed at https://www.youtube.com/playlist?list=PLQGtPFK17zUYkwFT-fr0a8E49R8Uq712l .
△ Less
Submitted 5 December, 2023;
originally announced December 2023.
-
DUMB: A Benchmark for Smart Evaluation of Dutch Models
Authors:
Wietse de Vries,
Martijn Wieling,
Malvina Nissim
Abstract:
We introduce the Dutch Model Benchmark: DUMB. The benchmark includes a diverse set of datasets for low-, medium- and high-resource tasks. The total set of nine tasks includes four tasks that were previously not available in Dutch. Instead of relying on a mean score across tasks, we propose Relative Error Reduction (RER), which compares the DUMB performance of language models to a strong baseline w…
▽ More
We introduce the Dutch Model Benchmark: DUMB. The benchmark includes a diverse set of datasets for low-, medium- and high-resource tasks. The total set of nine tasks includes four tasks that were previously not available in Dutch. Instead of relying on a mean score across tasks, we propose Relative Error Reduction (RER), which compares the DUMB performance of language models to a strong baseline which can be referred to in the future even when assessing different sets of language models. Through a comparison of 14 pre-trained language models (mono- and multi-lingual, of varying sizes), we assess the internal consistency of the benchmark tasks, as well as the factors that likely enable high performance. Our results indicate that current Dutch monolingual models under-perform and suggest training larger Dutch models with other architectures and pre-training objectives. At present, the highest performance is achieved by DeBERTaV3 (large), XLM-R (large) and mDeBERTaV3 (base). In addition to highlighting best strategies for training larger Dutch models, DUMB will foster further research on Dutch. A public leaderboard is available at https://dumbench.nl.
△ Less
Submitted 13 October, 2023; v1 submitted 22 May, 2023;
originally announced May 2023.
-
Adapting Monolingual Models: Data can be Scarce when Language Similarity is High
Authors:
Wietse de Vries,
Martijn Bartelds,
Malvina Nissim,
Martijn Wieling
Abstract:
For many (minority) languages, the resources needed to train large models are not available. We investigate the performance of zero-shot transfer learning with as little data as possible, and the influence of language similarity in this process. We retrain the lexical layers of four BERT-based models using data from two low-resource target language varieties, while the Transformer layers are indep…
▽ More
For many (minority) languages, the resources needed to train large models are not available. We investigate the performance of zero-shot transfer learning with as little data as possible, and the influence of language similarity in this process. We retrain the lexical layers of four BERT-based models using data from two low-resource target language varieties, while the Transformer layers are independently fine-tuned on a POS-tagging task in the model's source language. By combining the new lexical layers and fine-tuned Transformer layers, we achieve high task performance for both target languages. With high language similarity, 10MB of data appears sufficient to achieve substantial monolingual transfer performance. Monolingual BERT-based models generally achieve higher downstream task performance after retraining the lexical layer than multilingual BERT, even when the target language is included in the multilingual model.
△ Less
Submitted 22 May, 2021; v1 submitted 6 May, 2021;
originally announced May 2021.
-
As Good as New. How to Successfully Recycle English GPT-2 to Make Models for Other Languages
Authors:
Wietse de Vries,
Malvina Nissim
Abstract:
Large generative language models have been very successful for English, but other languages lag behind, in part due to data and computational limitations. We propose a method that may overcome these problems by adapting existing pre-trained models to new languages. Specifically, we describe the adaptation of English GPT-2 to Italian and Dutch by retraining lexical embeddings without tuning the Tra…
▽ More
Large generative language models have been very successful for English, but other languages lag behind, in part due to data and computational limitations. We propose a method that may overcome these problems by adapting existing pre-trained models to new languages. Specifically, we describe the adaptation of English GPT-2 to Italian and Dutch by retraining lexical embeddings without tuning the Transformer layers. As a result, we obtain lexical embeddings for Italian and Dutch that are aligned with the original English lexical embeddings. Additionally, we scale up complexity by transforming relearned lexical embeddings of GPT-2 small to the GPT-2 medium embedding space. This method minimises the amount of training and prevents losing information during adaptation that was learned by GPT-2. English GPT-2 models with relearned lexical embeddings can generate realistic sentences in Italian and Dutch. Though on average these sentences are still identifiable as artificial by humans, they are assessed on par with sentences generated by a GPT-2 model fully trained from scratch.
△ Less
Submitted 9 June, 2021; v1 submitted 10 December, 2020;
originally announced December 2020.
-
Neural Representations for Modeling Variation in Speech
Authors:
Martijn Bartelds,
Wietse de Vries,
Faraz Sanal,
Caitlin Richter,
Mark Liberman,
Martijn Wieling
Abstract:
Variation in speech is often quantified by comparing phonetic transcriptions of the same utterance. However, manually transcribing speech is time-consuming and error prone. As an alternative, therefore, we investigate the extraction of acoustic embeddings from several self-supervised neural models. We use these representations to compute word-based pronunciation differences between non-native and…
▽ More
Variation in speech is often quantified by comparing phonetic transcriptions of the same utterance. However, manually transcribing speech is time-consuming and error prone. As an alternative, therefore, we investigate the extraction of acoustic embeddings from several self-supervised neural models. We use these representations to compute word-based pronunciation differences between non-native and native speakers of English, and between Norwegian dialect speakers. For comparison with several earlier studies, we evaluate how well these differences match human perception by comparing them with available human judgements of similarity. We show that speech representations extracted from a specific type of neural model (i.e. Transformers) lead to a better match with human perception than two earlier approaches on the basis of phonetic transcriptions and MFCC-based acoustic features. We furthermore find that features from the neural models can generally best be extracted from one of the middle hidden layers than from the final layer. We also demonstrate that neural speech representations not only capture segmental differences, but also intonational and durational differences that cannot adequately be represented by a set of discrete symbols used in phonetic transcriptions.
△ Less
Submitted 26 January, 2022; v1 submitted 25 November, 2020;
originally announced November 2020.
-
Tangled: A Cooperative Anycast Testbed
Authors:
Leandro M. Bertholdo,
Joao M. Ceron,
Wouter B. de Vries,
Ricardo de O. Schmitt,
Lisandro Zambenedetti Granville,
Roland van Rijswijk-Deij,
Aiko Pras
Abstract:
Anycast routing is an area of studies that has been attracting interest of several researchers in recent years. Most anycast studies conducted in the past relied on coarse measurement data, mainly due to the lack of infrastructure where it is possible to test and collect data at same time. In this paper we present Tangled, an anycast test environment where researchers can run experiments and bette…
▽ More
Anycast routing is an area of studies that has been attracting interest of several researchers in recent years. Most anycast studies conducted in the past relied on coarse measurement data, mainly due to the lack of infrastructure where it is possible to test and collect data at same time. In this paper we present Tangled, an anycast test environment where researchers can run experiments and better understand the impacts of their proposals on a global infrastructure connected to the Internet.
△ Less
Submitted 28 August, 2020;
originally announced August 2020.
-
What's so special about BERT's layers? A closer look at the NLP pipeline in monolingual and multilingual models
Authors:
Wietse de Vries,
Andreas van Cranenburgh,
Malvina Nissim
Abstract:
Peeking into the inner workings of BERT has shown that its layers resemble the classical NLP pipeline, with progressively more complex tasks being concentrated in later layers. To investigate to what extent these results also hold for a language other than English, we probe a Dutch BERT-based model and the multilingual BERT model for Dutch NLP tasks. In addition, through a deeper analysis of part-…
▽ More
Peeking into the inner workings of BERT has shown that its layers resemble the classical NLP pipeline, with progressively more complex tasks being concentrated in later layers. To investigate to what extent these results also hold for a language other than English, we probe a Dutch BERT-based model and the multilingual BERT model for Dutch NLP tasks. In addition, through a deeper analysis of part-of-speech tagging, we show that also within a given task, information is spread over different parts of the network and the pipeline might not be as neat as it seems. Each layer has different specialisations, so that it may be more useful to combine information from different layers, instead of selecting a single one based on the best overall performance.
△ Less
Submitted 12 October, 2020; v1 submitted 14 April, 2020;
originally announced April 2020.
-
BERTje: A Dutch BERT Model
Authors:
Wietse de Vries,
Andreas van Cranenburgh,
Arianna Bisazza,
Tommaso Caselli,
Gertjan van Noord,
Malvina Nissim
Abstract:
The transformer-based pre-trained language model BERT has helped to improve state-of-the-art performance on many natural language processing (NLP) tasks. Using the same architecture and parameters, we developed and evaluated a monolingual Dutch BERT model called BERTje. Compared to the multilingual BERT model, which includes Dutch but is only based on Wikipedia text, BERTje is based on a large and…
▽ More
The transformer-based pre-trained language model BERT has helped to improve state-of-the-art performance on many natural language processing (NLP) tasks. Using the same architecture and parameters, we developed and evaluated a monolingual Dutch BERT model called BERTje. Compared to the multilingual BERT model, which includes Dutch but is only based on Wikipedia text, BERTje is based on a large and diverse dataset of 2.4 billion tokens. BERTje consistently outperforms the equally-sized multilingual BERT model on downstream NLP tasks (part-of-speech tagging, named-entity recognition, semantic role labeling, and sentiment analysis). Our pre-trained Dutch BERT model is made available at https://github.com/wietsedv/bertje.
△ Less
Submitted 19 December, 2019;
originally announced December 2019.
