-
Improving the quality of neural TTS using long-form content and multi-speaker multi-style modeling
Authors:
Tuomo Raitio,
Javier Latorre,
Andrea Davis,
Tuuli Morrill,
Ladan Golipour
Abstract:
Neural text-to-speech (TTS) can provide quality close to natural speech if an adequate amount of high-quality speech material is available for training. However, acquiring speech data for TTS training is costly and time-consuming, especially if the goal is to generate different speaking styles. In this work, we show that we can transfer speaking style across speakers and improve the quality of syn…
▽ More
Neural text-to-speech (TTS) can provide quality close to natural speech if an adequate amount of high-quality speech material is available for training. However, acquiring speech data for TTS training is costly and time-consuming, especially if the goal is to generate different speaking styles. In this work, we show that we can transfer speaking style across speakers and improve the quality of synthetic speech by training a multi-speaker multi-style (MSMS) model with long-form recordings, in addition to regular TTS recordings. In particular, we show that 1) multi-speaker modeling improves the overall TTS quality, 2) the proposed MSMS approach outperforms pre-training and fine-tuning approach when utilizing additional multi-speaker data, and 3) long-form speaking style is highly rated regardless of the target text domain.
△ Less
Submitted 28 June, 2023; v1 submitted 20 December, 2022;
originally announced December 2022.
-
Combining speakers of multiple languages to improve quality of neural voices
Authors:
Javier Latorre,
Charlotte Bailleul,
Tuuli Morrill,
Alistair Conkie,
Yannis Stylianou
Abstract:
In this work, we explore multiple architectures and training procedures for develo** a multi-speaker and multi-lingual neural TTS system with the goals of a) improving the quality when the available data in the target language is limited and b) enabling cross-lingual synthesis. We report results from a large experiment using 30 speakers in 8 different languages across 15 different locales. The s…
▽ More
In this work, we explore multiple architectures and training procedures for develo** a multi-speaker and multi-lingual neural TTS system with the goals of a) improving the quality when the available data in the target language is limited and b) enabling cross-lingual synthesis. We report results from a large experiment using 30 speakers in 8 different languages across 15 different locales. The system is trained on the same amount of data per speaker. Compared to a single-speaker model, when the suggested system is fine tuned to a speaker, it produces significantly better quality in most of the cases while it only uses less than $40\%$ of the speaker's data used to build the single-speaker model. In cross-lingual synthesis, on average, the generated quality is within $80\%$ of native single-speaker models, in terms of Mean Opinion Score.
△ Less
Submitted 17 August, 2021;
originally announced August 2021.
-
Effect of data reduction on sequence-to-sequence neural TTS
Authors:
Javier Latorre,
Jakub Lachowicz,
Jaime Lorenzo-Trueba,
Thomas Merritt,
Thomas Drugman,
Srikanth Ronanki,
Klimkov Viacheslav
Abstract:
Recent speech synthesis systems based on sampling from autoregressive neural networks models can generate speech almost undistinguishable from human recordings. However, these models require large amounts of data. This paper shows that the lack of data from one speaker can be compensated with data from other speakers. The naturalness of Tacotron2-like models trained on a blend of 5k utterances fro…
▽ More
Recent speech synthesis systems based on sampling from autoregressive neural networks models can generate speech almost undistinguishable from human recordings. However, these models require large amounts of data. This paper shows that the lack of data from one speaker can be compensated with data from other speakers. The naturalness of Tacotron2-like models trained on a blend of 5k utterances from 7 speakers is better than that of speaker dependent models trained on 15k utterances, but in terms of stability multi-speaker models are always more stable. We also demonstrate that models mixing only 1250 utterances from a target speaker with 5k utterances from another 6 speakers can produce significantly better quality than state-of-the-art DNN-guided unit selection systems trained on more than 10 times the data from the target speaker.
△ Less
Submitted 23 November, 2018; v1 submitted 15 November, 2018;
originally announced November 2018.
-
Towards achieving robust universal neural vocoding
Authors:
Jaime Lorenzo-Trueba,
Thomas Drugman,
Javier Latorre,
Thomas Merritt,
Bartosz Putrycz,
Roberto Barra-Chicote,
Alexis Moinet,
Vatsal Aggarwal
Abstract:
This paper explores the potential universality of neural vocoders. We train a WaveRNN-based vocoder on 74 speakers coming from 17 languages. This vocoder is shown to be capable of generating speech of consistently good quality (98% relative mean MUSHRA when compared to natural speech) regardless of whether the input spectrogram comes from a speaker or style seen during training or from an out-of-d…
▽ More
This paper explores the potential universality of neural vocoders. We train a WaveRNN-based vocoder on 74 speakers coming from 17 languages. This vocoder is shown to be capable of generating speech of consistently good quality (98% relative mean MUSHRA when compared to natural speech) regardless of whether the input spectrogram comes from a speaker or style seen during training or from an out-of-domain scenario when the recording conditions are studio-quality. When the recordings show significant changes in quality, or when moving towards non-speech vocalizations or singing, the vocoder still significantly outperforms speaker-dependent vocoders, but operates at a lower average relative MUSHRA of 75%. These results are shown to be consistent across languages, regardless of them being seen during training (e.g. English or Japanese) or unseen (e.g. Wolof, Swahili, Ahmaric).
△ Less
Submitted 4 July, 2019; v1 submitted 15 November, 2018;
originally announced November 2018.
