-
SynthASR: Unlocking Synthetic Data for Speech Recognition
Authors:
Amin Fazel,
Wei Yang,
Yulan Liu,
Roberto Barra-Chicote,
Yixiong Meng,
Roland Maas,
Jasha Droppo
Abstract:
End-to-end (E2E) automatic speech recognition (ASR) models have recently demonstrated superior performance over the traditional hybrid ASR models. Training an E2E ASR model requires a large amount of data which is not only expensive but may also raise dependency on production data. At the same time, synthetic speech generated by the state-of-the-art text-to-speech (TTS) engines has advanced to nea…
▽ More
End-to-end (E2E) automatic speech recognition (ASR) models have recently demonstrated superior performance over the traditional hybrid ASR models. Training an E2E ASR model requires a large amount of data which is not only expensive but may also raise dependency on production data. At the same time, synthetic speech generated by the state-of-the-art text-to-speech (TTS) engines has advanced to near-human naturalness. In this work, we propose to utilize synthetic speech for ASR training (SynthASR) in applications where data is sparse or hard to get for ASR model training. In addition, we apply continual learning with a novel multi-stage training strategy to address catastrophic forgetting, achieved by a mix of weighted multi-style training, data augmentation, encoder freezing, and parameter regularization. In our experiments conducted on in-house datasets for a new application of recognizing medication names, training ASR RNN-T models with synthetic audio via the proposed multi-stage training improved the recognition performance on new application by more than 65% relative, without degradation on existing general applications. Our observations show that SynthASR holds great promise in training the state-of-the-art large-scale E2E ASR models for new applications while reducing the costs and dependency on production data.
△ Less
Submitted 14 June, 2021;
originally announced June 2021.
-
A Wearable Social Interaction Aid for Children with Autism
Authors:
Nick Haber,
Catalin Voss,
Jena Daniels,
Peter Washington,
Azar Fazel,
Aaron Kline,
Titas De,
Terry Winograd,
Carl Feinstein,
Dennis P. Wall
Abstract:
With most recent estimates giving an incidence rate of 1 in 68 children in the United States, the autism spectrum disorder (ASD) is a growing public health crisis. Many of these children struggle to make eye contact, recognize facial expressions, and engage in social interactions. Today the standard for treatment of the core autism-related deficits focuses on a form of behavior training known as A…
▽ More
With most recent estimates giving an incidence rate of 1 in 68 children in the United States, the autism spectrum disorder (ASD) is a growing public health crisis. Many of these children struggle to make eye contact, recognize facial expressions, and engage in social interactions. Today the standard for treatment of the core autism-related deficits focuses on a form of behavior training known as Applied Behavioral Analysis. To address perceived deficits in expression recognition, ABA approaches routinely involve the use of prompts such as flash cards for repetitive emotion recognition training via memorization. These techniques must be administered by trained practitioners and often at clinical centers that are far outnumbered by and out of reach from the many children and families in need of attention. Waitlists for access are up to 18 months long, and this wait may lead to children regressing down a path of isolation that worsens their long-term prognosis. There is an urgent need to innovate new methods of care delivery that can appropriately empower caregivers of children at risk or with a diagnosis of autism, and that capitalize on mobile tools and wearable devices for use outside of clinical settings.
△ Less
Submitted 19 April, 2020;
originally announced April 2020.
-
Benchmarking TinyML Systems: Challenges and Direction
Authors:
Colby R. Banbury,
Vijay Janapa Reddi,
Max Lam,
William Fu,
Amin Fazel,
Jeremy Holleman,
Xinyuan Huang,
Robert Hurtado,
David Kanter,
Anton Lokhmotov,
David Patterson,
Danilo Pau,
Jae-sun Seo,
Jeff Sieracki,
Urmish Thakker,
Marian Verhelst,
Poonam Yadav
Abstract:
Recent advancements in ultra-low-power machine learning (TinyML) hardware promises to unlock an entirely new class of smart applications. However, continued progress is limited by the lack of a widely accepted benchmark for these systems. Benchmarking allows us to measure and thereby systematically compare, evaluate, and improve the performance of systems and is therefore fundamental to a field re…
▽ More
Recent advancements in ultra-low-power machine learning (TinyML) hardware promises to unlock an entirely new class of smart applications. However, continued progress is limited by the lack of a widely accepted benchmark for these systems. Benchmarking allows us to measure and thereby systematically compare, evaluate, and improve the performance of systems and is therefore fundamental to a field reaching maturity. In this position paper, we present the current landscape of TinyML and discuss the challenges and direction towards develo** a fair and useful hardware benchmark for TinyML workloads. Furthermore, we present our four benchmarks and discuss our selection methodology. Our viewpoints reflect the collective thoughts of the TinyMLPerf working group that is comprised of over 30 organizations.
△ Less
Submitted 29 January, 2021; v1 submitted 10 March, 2020;
originally announced March 2020.
-
Superpower Glass: Delivering Unobtrusive Real-time Social Cues in Wearable Systems
Authors:
Catalin Voss,
Peter Washington,
Nick Haber,
Aaron Kline,
Jena Daniels,
Azar Fazel,
Titas De,
Beth McCarthy,
Carl Feinstein,
Terry Winograd,
Dennis Wall
Abstract:
We have developed a system for automatic facial expression recognition, which runs on Google Glass and delivers real-time social cues to the wearer. We evaluate the system as a behavioral aid for children with Autism Spectrum Disorder (ASD), who can greatly benefit from real-time non-invasive emotional cues and are more sensitive to sensory input than neurotypically develo** children. In additio…
▽ More
We have developed a system for automatic facial expression recognition, which runs on Google Glass and delivers real-time social cues to the wearer. We evaluate the system as a behavioral aid for children with Autism Spectrum Disorder (ASD), who can greatly benefit from real-time non-invasive emotional cues and are more sensitive to sensory input than neurotypically develo** children. In addition, we present a mobile application that enables users of the wearable aid to review their videos along with auto-curated emotional information on the video playback bar. This integrates our learning aid into the context of behavioral therapy. Expanding on our previous work describing in-lab trials, this paper presents our system and application-level design decisions in depth as well as the interface learnings gathered during the use of the system by multiple children with ASD in an at-home iterative trial.
△ Less
Submitted 16 February, 2020;
originally announced February 2020.
-
Designing a Holistic At-Home Learning Aid for Autism
Authors:
Catalin Voss,
Nick Haber,
Peter Washington,
Aaron Kline,
Beth McCarthy,
Jena Daniels,
Azar Fazel,
Titas De,
Carl Feinstein,
Terry Winograd,
Dennis Wall
Abstract:
In recent years, much focus has been put on employing technology to make novel behavioural aids for those with autism. Most of these are digital adaptations of tools used in standard behavioural therapy to enforce normative skills. These digital counterparts are often used outside of both the larger therapeutic context and the real world, in which the learned skills might apply. To address this, w…
▽ More
In recent years, much focus has been put on employing technology to make novel behavioural aids for those with autism. Most of these are digital adaptations of tools used in standard behavioural therapy to enforce normative skills. These digital counterparts are often used outside of both the larger therapeutic context and the real world, in which the learned skills might apply. To address this, we are designing a system of automatic expression recognition on wearable devices that integrates directly into the families daily social interactions, to give children and their caregivers the tools and information they need to design their own holistic therapy. In order to develop a tool that will be truly useful to families, we proactively include children with autism and their families as co-designers in the development process. By providing an app and interface with interchangeable social feedback options, we aim to produce a framework for therapy that folds into their daily lives, tailored to their specific needs.
△ Less
Submitted 11 February, 2020;
originally announced February 2020.
-
Convolutional Neural Networks for Facial Expression Recognition
Authors:
Shima Alizadeh,
Azar Fazel
Abstract:
We have developed convolutional neural networks (CNN) for a facial expression recognition task. The goal is to classify each facial image into one of the seven facial emotion categories considered in this study. We trained CNN models with different depth using gray-scale images. We developed our models in Torch and exploited Graphics Processing Unit (GPU) computation in order to expedite the train…
▽ More
We have developed convolutional neural networks (CNN) for a facial expression recognition task. The goal is to classify each facial image into one of the seven facial emotion categories considered in this study. We trained CNN models with different depth using gray-scale images. We developed our models in Torch and exploited Graphics Processing Unit (GPU) computation in order to expedite the training process. In addition to the networks performing based on raw pixel data, we employed a hybrid feature strategy by which we trained a novel CNN model with the combination of raw pixel data and Histogram of Oriented Gradients (HOG) features. To reduce the overfitting of the models, we utilized different techniques including dropout and batch normalization in addition to L2 regularization. We applied cross validation to determine the optimal hyper-parameters and evaluated the performance of the developed models by looking at their training histories. We also present the visualization of different layers of a network to show what features of a face can be learned by CNN models.
△ Less
Submitted 22 April, 2017;
originally announced April 2017.
