Speaking rate attention-based duration prediction for speed control TTS
Authors:
Jesuraj Bandekar,
Sathvik Udupa,
Abhayjeet Singh,
Anjali Jayakumar,
Deekshitha G,
Sandhya Badiger,
Saurabh Kumar,
Pooja VH,
Prasanta Kumar Ghosh
Abstract:
With the advent of high-quality speech synthesis, there is a lot of interest in controlling various prosodic attributes of speech. Speaking rate is an essential attribute towards modelling the expressivity of speech. In this work, we propose a novel approach to control the speaking rate for non-autoregressive TTS. We achieve this by conditioning the speaking rate inside the duration predictor, all…
▽ More
With the advent of high-quality speech synthesis, there is a lot of interest in controlling various prosodic attributes of speech. Speaking rate is an essential attribute towards modelling the expressivity of speech. In this work, we propose a novel approach to control the speaking rate for non-autoregressive TTS. We achieve this by conditioning the speaking rate inside the duration predictor, allowing implicit speaking rate control. We show the benefits of this approach by synthesising audio at various speaking rate factors and measuring the quality of speaking rate-controlled synthesised speech. Further, we study the effect of the speaking rate distribution of the training data towards effective rate control. Finally, we fine-tune a baseline pretrained TTS model to obtain speaking rate control TTS. We provide various analyses to showcase the benefits of using this proposed approach, along with objective as well as subjective metrics. We find that the proposed methods have higher subjective scores and lower speaker rate errors across many speaking rate factors over the baseline.
△ Less
Submitted 13 October, 2023;
originally announced October 2023.
Model Adaptation for ASR in low-resource Indian Languages
Authors:
Abhayjeet Singh,
Arjun Singh Mehta,
Ashish Khuraishi K S,
Deekshitha G,
Gauri Date,
Jai Nanavati,
Jesuraja Bandekar,
Karnalius Basumatary,
Karthika P,
Sandhya Badiger,
Sathvik Udupa,
Saurabh Kumar,
Savitha,
Prasanta Kumar Ghosh,
Prashanthi V,
Priyanka Pai,
Raoul Nanavati,
Rohan Saxena,
Sai Praneeth Reddy Mora,
Srinivasa Raghavan
Abstract:
Automatic speech recognition (ASR) performance has improved drastically in recent years, mainly enabled by self-supervised learning (SSL) based acoustic models such as wav2vec2 and large-scale multi-lingual training like Whisper. A huge challenge still exists for low-resource languages where the availability of both audio and text is limited. This is further complicated by the presence of multiple…
▽ More
Automatic speech recognition (ASR) performance has improved drastically in recent years, mainly enabled by self-supervised learning (SSL) based acoustic models such as wav2vec2 and large-scale multi-lingual training like Whisper. A huge challenge still exists for low-resource languages where the availability of both audio and text is limited. This is further complicated by the presence of multiple dialects like in Indian languages. However, many Indian languages can be grouped into the same families and share the same script and grammatical structure. This is where a lot of adaptation and fine-tuning techniques can be applied to overcome the low-resource nature of the data by utilising well-resourced similar languages.
In such scenarios, it is important to understand the extent to which each modality, like acoustics and text, is important in building a reliable ASR. It could be the case that an abundance of acoustic data in a language reduces the need for large text-only corpora. Or, due to the availability of various pretrained acoustic models, the vice-versa could also be true. In this proposed special session, we encourage the community to explore these ideas with the data in two low-resource Indian languages of Bengali and Bhojpuri. These approaches are not limited to Indian languages, the solutions are potentially applicable to various languages spoken around the world.
△ Less
Submitted 16 July, 2023;
originally announced July 2023.
