-
Data Quality as Predictor of Voice Anti-Spoofing Generalization
Authors:
Bhusan Chettri,
Rosa González Hautamäki,
Md Sahidullah,
Tomi Kinnunen
Abstract:
Voice anti-spoofing aims at classifying a given utterance either as a bonafide human sample, or a spoofing attack (e.g. synthetic or replayed sample). Many anti-spoofing methods have been proposed but most of them fail to generalize across domains (corpora) -- and we do not know \emph{why}. We outline a novel interpretative framework for gauging the impact of data quality upon anti-spoofing perfor…
▽ More
Voice anti-spoofing aims at classifying a given utterance either as a bonafide human sample, or a spoofing attack (e.g. synthetic or replayed sample). Many anti-spoofing methods have been proposed but most of them fail to generalize across domains (corpora) -- and we do not know \emph{why}. We outline a novel interpretative framework for gauging the impact of data quality upon anti-spoofing performance. Our within- and between-domain experiments pool data from seven public corpora and three anti-spoofing methods based on Gaussian mixture and convolutive neural network models. We assess the impacts of long-term spectral information, speaker population (through x-vector speaker embeddings), signal-to-noise ratio, and selected voice quality features.
△ Less
Submitted 21 June, 2021; v1 submitted 26 March, 2021;
originally announced March 2021.
-
Dataset artefacts in anti-spoofing systems: a case study on the ASVspoof 2017 benchmark
Authors:
Bhusan Chettri,
Emmanouil Benetos,
Bob L. T. Sturm
Abstract:
The Automatic Speaker Verification Spoofing and Countermeasures Challenges motivate research in protecting speech biometric systems against a variety of different access attacks. The 2017 edition focused on replay spoofing attacks, and involved participants building and training systems on a provided dataset (ASVspoof 2017). More than 60 research papers have so far been published with this dataset…
▽ More
The Automatic Speaker Verification Spoofing and Countermeasures Challenges motivate research in protecting speech biometric systems against a variety of different access attacks. The 2017 edition focused on replay spoofing attacks, and involved participants building and training systems on a provided dataset (ASVspoof 2017). More than 60 research papers have so far been published with this dataset, but none have sought to answer why countermeasures appear successful in detecting spoofing attacks. This article shows how artefacts inherent to the dataset may be contributing to the apparent success of published systems. We first inspect the ASVspoof 2017 dataset and summarize various artefacts present in the dataset. Second, we demonstrate how countermeasure models can exploit these artefacts to appear successful in this dataset. Third, for reliable and robust performance estimates on this dataset we propose discarding nonspeech segments and silence before and after the speech utterance during training and inference. We create speech start and endpoint annotations in the dataset and demonstrate how using them helps countermeasure models become less vulnerable from being manipulated using artefacts found in the dataset. Finally, we provide several new benchmark results for both frame-level and utterance-level models that can serve as new baselines on this dataset.
△ Less
Submitted 15 October, 2020;
originally announced October 2020.
-
Ensemble Models for Spoofing Detection in Automatic Speaker Verification
Authors:
Bhusan Chettri,
Daniel Stoller,
Veronica Morfi,
Marco A. Martínez Ramírez,
Emmanouil Benetos,
Bob L. Sturm
Abstract:
Detecting spoofing attempts of automatic speaker verification (ASV) systems is challenging, especially when using only one modeling approach. For robustness, we use both deep neural networks and traditional machine learning models and combine them as ensemble models through logistic regression. They are trained to detect logical access (LA) and physical access (PA) attacks on the dataset released…
▽ More
Detecting spoofing attempts of automatic speaker verification (ASV) systems is challenging, especially when using only one modeling approach. For robustness, we use both deep neural networks and traditional machine learning models and combine them as ensemble models through logistic regression. They are trained to detect logical access (LA) and physical access (PA) attacks on the dataset released as part of the ASV Spoofing and Countermeasures Challenge 2019. We propose dataset partitions that ensure different attack types are present during training and validation to improve system robustness. Our ensemble model outperforms all our single models and the baselines from the challenge for both attack types. We investigate why some models on the PA dataset strongly outperform others and find that spoofed recordings in the dataset tend to have longer silences at the end than genuine ones. By removing them, the PA task becomes much more challenging, with the tandem detection cost function (t-DCF) of our best single model rising from 0.1672 to 0.5018 and equal error rate (EER) increasing from 5.98% to 19.8% on the development set.
△ Less
Submitted 4 July, 2019; v1 submitted 9 April, 2019;
originally announced April 2019.
-
A Study On Convolutional Neural Network Based End-To-End Replay Anti-Spoofing
Authors:
Bhusan Chettri,
Saumitra Mishra,
Bob L. Sturm,
Emmanouil Benetos
Abstract:
The second Automatic Speaker Verification Spoofing and Countermeasures challenge (ASVspoof 2017) focused on "replay attack" detection. The best deep-learning systems to compete in ASVspoof 2017 used Convolutional Neural Networks (CNNs) as a feature extractor. In this paper, we study their performance in an end-to-end setting. We find that these architectures show poor generalization in the evaluat…
▽ More
The second Automatic Speaker Verification Spoofing and Countermeasures challenge (ASVspoof 2017) focused on "replay attack" detection. The best deep-learning systems to compete in ASVspoof 2017 used Convolutional Neural Networks (CNNs) as a feature extractor. In this paper, we study their performance in an end-to-end setting. We find that these architectures show poor generalization in the evaluation dataset, but find a compact architecture that shows good generalization on the development data. We demonstrate that for this dataset it is not easy to obtain a similar level of generalization on both the development and evaluation data. This leads to a variety of open questions about what the differences are in the data; why these are more evident in an end-to-end setting; and how these issues can be overcome by increasing the training data.
△ Less
Submitted 22 May, 2018;
originally announced May 2018.
