Less is More: Accurate Speech Recognition & Translation without Web-Scale Data
Authors:
Krishna C. Puvvada,
Piotr Żelasko,
He Huang,
Oleksii Hrinchuk,
Nithin Rao Koluguri,
Kunal Dhawan,
Somshubra Majumdar,
Elena Rastorgueva,
Zhehuai Chen,
Vitaly Lavrukhin,
Jagadeesh Balam,
Boris Ginsburg
Abstract:
Recent advances in speech recognition and translation rely on hundreds of thousands of hours of Internet speech data. We argue that state-of-the art accuracy can be reached without relying on web-scale data. Canary - multilingual ASR and speech translation model, outperforms current state-of-the-art models - Whisper, OWSM, and Seamless-M4T on English, French, Spanish, and German languages, while b…
▽ More
Recent advances in speech recognition and translation rely on hundreds of thousands of hours of Internet speech data. We argue that state-of-the art accuracy can be reached without relying on web-scale data. Canary - multilingual ASR and speech translation model, outperforms current state-of-the-art models - Whisper, OWSM, and Seamless-M4T on English, French, Spanish, and German languages, while being trained on an order of magnitude less data than these models. Three key factors enables such data-efficient model: (1) a FastConformer-based attention encoder-decoder architecture (2) training on synthetic data generated with machine translation and (3) advanced training techniques: data-balancing, dynamic data blending, dynamic bucketing and noise-robust fine-tuning. The model, weights, and training code will be open-sourced.
△ Less
Submitted 28 June, 2024;
originally announced June 2024.
SpeechBrain: A General-Purpose Speech Toolkit
Authors:
Mirco Ravanelli,
Titouan Parcollet,
Peter Plantinga,
Aku Rouhe,
Samuele Cornell,
Loren Lugosch,
Cem Subakan,
Nauman Dawalatabad,
Abdelwahab Heba,
Jianyuan Zhong,
Ju-Chieh Chou,
Sung-Lin Yeh,
Szu-Wei Fu,
Chien-Feng Liao,
Elena Rastorgueva,
François Grondin,
William Aris,
Hwidong Na,
Yan Gao,
Renato De Mori,
Yoshua Bengio
Abstract:
SpeechBrain is an open-source and all-in-one speech toolkit. It is designed to facilitate the research and development of neural speech processing technologies by being simple, flexible, user-friendly, and well-documented. This paper describes the core architecture designed to support several tasks of common interest, allowing users to naturally conceive, compare and share novel speech processing…
▽ More
SpeechBrain is an open-source and all-in-one speech toolkit. It is designed to facilitate the research and development of neural speech processing technologies by being simple, flexible, user-friendly, and well-documented. This paper describes the core architecture designed to support several tasks of common interest, allowing users to naturally conceive, compare and share novel speech processing pipelines. SpeechBrain achieves competitive or state-of-the-art performance in a wide range of speech benchmarks. It also provides training recipes, pretrained models, and inference scripts for popular speech datasets, as well as tutorials which allow anyone with basic Python proficiency to familiarize themselves with speech technologies.
△ Less
Submitted 8 June, 2021;
originally announced June 2021.
