-
DDKtor: Automatic Diadochokinetic Speech Analysis
Authors:
Yael Segal,
Kasia Hitczenko,
Matthew Goldrick,
Adam Buchwald,
Angela Roberts,
Joseph Keshet
Abstract:
Diadochokinetic speech tasks (DDK), in which participants repeatedly produce syllables, are commonly used as part of the assessment of speech motor impairments. These studies rely on manual analyses that are time-intensive, subjective, and provide only a coarse-grained picture of speech. This paper presents two deep neural network models that automatically segment consonants and vowels from unanno…
▽ More
Diadochokinetic speech tasks (DDK), in which participants repeatedly produce syllables, are commonly used as part of the assessment of speech motor impairments. These studies rely on manual analyses that are time-intensive, subjective, and provide only a coarse-grained picture of speech. This paper presents two deep neural network models that automatically segment consonants and vowels from unannotated, untranscribed speech. Both models work on the raw waveform and use convolutional layers for feature extraction. The first model is based on an LSTM classifier followed by fully connected layers, while the second model adds more convolutional layers followed by fully connected layers. These segmentations predicted by the models are used to obtain measures of speech rate and sound duration. Results on a young healthy individuals dataset show that our LSTM model outperforms the current state-of-the-art systems and performs comparably to trained human annotators. Moreover, the LSTM model also presents comparable results to trained human annotators when evaluated on unseen older individuals with Parkinson's Disease dataset.
△ Less
Submitted 29 June, 2022;
originally announced June 2022.
-
Neurocompositional computing: From the Central Paradox of Cognition to a new generation of AI systems
Authors:
Paul Smolensky,
R. Thomas McCoy,
Roland Fernandez,
Matthew Goldrick,
Jianfeng Gao
Abstract:
What explains the dramatic progress from 20th-century to 21st-century AI, and how can the remaining limitations of current AI be overcome? The widely accepted narrative attributes this progress to massive increases in the quantity of computational and data resources available to support statistical learning in deep artificial neural networks. We show that an additional crucial factor is the develo…
▽ More
What explains the dramatic progress from 20th-century to 21st-century AI, and how can the remaining limitations of current AI be overcome? The widely accepted narrative attributes this progress to massive increases in the quantity of computational and data resources available to support statistical learning in deep artificial neural networks. We show that an additional crucial factor is the development of a new type of computation. Neurocompositional computing adopts two principles that must be simultaneously respected to enable human-level cognition: the principles of Compositionality and Continuity. These have seemed irreconcilable until the recent mathematical discovery that compositionality can be realized not only through discrete methods of symbolic computing, but also through novel forms of continuous neural computing. The revolutionary recent progress in AI has resulted from the use of limited forms of neurocompositional computing. New, deeper forms of neurocompositional computing create AI systems that are more robust, accurate, and comprehensible.
△ Less
Submitted 2 May, 2022;
originally announced May 2022.
-
Dr.VOT : Measuring Positive and Negative Voice Onset Time in the Wild
Authors:
Yosi Shrem,
Matthew Goldrick,
Joseph Keshet
Abstract:
Voice Onset Time (VOT), a key measurement of speech for basic research and applied medical studies, is the time between the onset of a stop burst and the onset of voicing. When the voicing onset precedes burst onset the VOT is negative; if voicing onset follows the burst, it is positive. In this work, we present a deep-learning model for accurate and reliable measurement of VOT in naturalistic spe…
▽ More
Voice Onset Time (VOT), a key measurement of speech for basic research and applied medical studies, is the time between the onset of a stop burst and the onset of voicing. When the voicing onset precedes burst onset the VOT is negative; if voicing onset follows the burst, it is positive. In this work, we present a deep-learning model for accurate and reliable measurement of VOT in naturalistic speech. The proposed system addresses two critical issues: it can measure positive and negative VOT equally well, and it is trained to be robust to variation across annotations. Our approach is based on the structured prediction framework, where the feature functions are defined to be RNNs. These learn to capture segmental variation in the signal. Results suggest that our method substantially improves over the current state-of-the-art. In contrast to previous work, our Deep and Robust VOT annotator, Dr.VOT, can successfully estimate negative VOTs while maintaining state-of-the-art performance on positive VOTs. This high level of performance generalizes to new corpora without further retraining. Index Terms: structured prediction, multi-task learning, adversarial training, recurrent neural networks, sequence segmentation.
△ Less
Submitted 27 October, 2019;
originally announced October 2019.
-
Securing Vehicle to Vehicle Communications using Blockchain through Visible Light and Acoustic Side-Channels
Authors:
Sean Rowan,
Michael Clear,
Mario Gerla,
Meriel Huggard,
Ciarán Mc Goldrick
Abstract:
Autonomous and self-driving vehicles are appearing on the public highways. These vehicles commonly use wireless communication techniques for both vehicle-to-vehicle and vehicle-to-infrastructure communications. Manufacturers, regulators and the public are understandably concerned about large-scale systems failure or malicious attack via these wireless vehicular networks. This paper explores the us…
▽ More
Autonomous and self-driving vehicles are appearing on the public highways. These vehicles commonly use wireless communication techniques for both vehicle-to-vehicle and vehicle-to-infrastructure communications. Manufacturers, regulators and the public are understandably concerned about large-scale systems failure or malicious attack via these wireless vehicular networks. This paper explores the use of sensing and signalling devices that are commonly integrated into modern vehicles for side-channel communication purposes. Visible light (using a CMOS camera) and acoustic (ultrasonic audio) side-channel encoding techniques are proposed, developed and evaluated in this context. The side-channels are examined both theoretically and experimentally and an upper bound on the line code modulation rate that is achievable with these side channel schemes in the vehicular networking context is established. A novel inter-vehicle session key establishment protocol, leveraging both side-channels and a blockchain public key infrastructure, is then presented. In light of the limited channel capacity and the interoperability/security requirements for vehicular communications, techniques for constraining the throughput requirement, providing device independence and validating the location of the intended recipient vehicle, are presented. These reduce the necessary device handshake throughput to 176 bits for creating symmetric encryption and message authentication keys and in verifying a vehicle's certificate with a recognised certification authority.
△ Less
Submitted 8 April, 2017;
originally announced April 2017.
-
Automatic measurement of vowel duration via structured prediction
Authors:
Yossi Adi,
Joseph Keshet,
Emily Cibelli,
Erin Gustafson,
Cynthia Clopper,
Matthew Goldrick
Abstract:
A key barrier to making phonetic studies scalable and replicable is the need to rely on subjective, manual annotation. To help meet this challenge, a machine learning algorithm was developed for automatic measurement of a widely used phonetic measure: vowel duration. Manually-annotated data were used to train a model that takes as input an arbitrary length segment of the acoustic signal containing…
▽ More
A key barrier to making phonetic studies scalable and replicable is the need to rely on subjective, manual annotation. To help meet this challenge, a machine learning algorithm was developed for automatic measurement of a widely used phonetic measure: vowel duration. Manually-annotated data were used to train a model that takes as input an arbitrary length segment of the acoustic signal containing a single vowel that is preceded and followed by consonants and outputs the duration of the vowel. The model is based on the structured prediction framework. The input signal and a hypothesized set of a vowel's onset and offset are mapped to an abstract vector space by a set of acoustic feature functions. The learning algorithm is trained in this space to minimize the difference in expectations between predicted and manually-measured vowel durations. The trained model can then automatically estimate vowel durations without phonetic or orthographic transcription. Results comparing the model to three sets of manually annotated data suggest it out-performed the current gold standard for duration measurement, an HMM-based forced aligner (which requires orthographic or phonetic transcription as an input).
△ Less
Submitted 26 October, 2016;
originally announced October 2016.
-
Sequence Segmentation Using Joint RNN and Structured Prediction Models
Authors:
Yossi Adi,
Joseph Keshet,
Emily Cibelli,
Matthew Goldrick
Abstract:
We describe and analyze a simple and effective algorithm for sequence segmentation applied to speech processing tasks. We propose a neural architecture that is composed of two modules trained jointly: a recurrent neural network (RNN) module and a structured prediction model. The RNN outputs are considered as feature functions to the structured model. The overall model is trained with a structured…
▽ More
We describe and analyze a simple and effective algorithm for sequence segmentation applied to speech processing tasks. We propose a neural architecture that is composed of two modules trained jointly: a recurrent neural network (RNN) module and a structured prediction model. The RNN outputs are considered as feature functions to the structured model. The overall model is trained with a structured loss function which can be designed to the given segmentation task. We demonstrate the effectiveness of our method by applying it to two simple tasks commonly used in phonetic studies: word segmentation and voice onset time segmentation. Results sug- gest the proposed model is superior to previous methods, ob- taining state-of-the-art results on the tested datasets.
△ Less
Submitted 25 October, 2016;
originally announced October 2016.
