Training-Free Deepfake Voice Recognition by Leveraging Large-Scale Pre-Trained Models

Abstract

Generalization is a main issue for current audio deepfake detectors, which struggle to provide reliable results on out-of-distribution data. Given the speed at which more and more accurate synthesis methods are developed, it is very important to design techniques that work well also on data they were not trained for. In this paper we study the potential of large-scale pre-trained models for audio deepfake detection, with special focus on generalization ability. To this end, the detection problem is reformulated in a speaker verification framework and fake audios are exposed by the mismatch between the voice sample under test and the voice of the claimed identity. With this paradigm, no fake speech sample is necessary in training, cutting off any link with the generation method at the root, and ensuring full generalization ability. Features are extracted by general-purpose large pre-trained models, with no need for training or fine-tuning on specific fake detection or speaker verification datasets. At detection time only a limited set of voice fragments of the identity under test is required. Experiments on several datasets widespread in the community show that detectors based on pre-trained models achieve excellent performance and show strong generalization ability, rivaling supervised methods on in-distribution data and largely overcoming them on out-of-distribution data.

I Introduction

As deepfake editing tools become more and more accessible, manipulated content begins spreading widely on social networks. In this context, voice deepfakes are playing a key role in the surge of misinformation because a spoofed recording can be just as effective at fooling people as a video deepfake is, but it is much easier to generate. In a recent well-known episode, malicious actors used AI to impersonate a German chief executive’s voice on the phone, obtaining a fraudulent money transfer of 220,000 dollars. Even more worrisome, voice deepfake can be used to spread disinformation in political campaigns, to defame people, arouse racial hatred, and a wide range of other illicit purposes. This justifies the fast growing attention in recent years for voice deepfakes and their detection. Actually, the interest on anti-spoofing methods is old with a large scientific literature that was mostly focused on Text to Speech (TTS) synthesis and Voice Conversion (VC) with classical methods.

Some papers rely on the extraction of acoustic features in the Fourier domain. In fact, it has been recognized that synthesized speech exhibits unusual correlations between magnitude and phase spectra at the high frequencies [2] and also that they exhibit an anomalous energy distribution among vocal formants [15]. These model-based approaches provide not only a good performance but also high interpretability [29]. Detectors based on handcrafted features, however, depend too much on the artifacts they are looking for. To avoid this problem, in [32] a deep neural network classifier is developed that works directly on the raw speech signal. The first layer of the network is constrained to act as a filter bank, with filters of fixed shape and learned cut-in and cut-off frequencies. Other solutions have been proposed to avoid over-fitting to specific artifacts, such as using features defined in different domains, e.g., the temporal and spectral intervals [31, 22, 8], exploiting the limited ability of synthetic generators to perfectly reproduce the vocal tract [6] or replicate certain phonemes, or using features related to the human articulation system [18].

The main problem with such approaches is that they cannot cope with the large variety of synthetic generators that are proposed day after day. In fact, most of them use supervised learning and are not able to generalize to unseen attacks [24]. A possible solution is to follow a one-class approach where training is carried out only on the real class (bona fide speech) and data that depart from the learned model are classified as anomalies [36]. As fakes are not used during training, generalization would be automatically ensured. It goes without saying that accurately modeling the actual voice class, with all its myriad variations, is simply out of reach. However, following this line of thinking, it is possible to reframe the problem of detection as a speaker verification task. That is, instead of investigating whether or not a specific segment of speech was spoken by a human and not by a machine, a simpler question can be asked: was it uttered by this specific human being? Does this voice really belong to the person it is said to belong to? This is obviously a different task, speaker verification, but in practice it implies voice deepfake detection. In fact, if the claimed speaker is not recognized, there is good reason to suspect that the audio was manipulated and, at the very least, proceed with further and more thorough analyses. On the other hand, recognizing the identity of the speaker is essential in order to give credence to the message conveyed by any speech. Who would give credit to messages coming from unknown people? Not by chance, the first method designed to recognize deepfake videos with this approach aimed at “protecting world leaders” [1] that is, very influential and well-known persons, which led to the so called person-of-interest (PoI)-based deepfake detectors [14, 7]. Note that these methods are different from [9, 19], where a large dataset of real/synthetic audios is needed for training. It is also interesting to highlight an exchange of roles. Here speaker verification is used as a means to detect forgeries, while usually, in biometric access systems, forgery detection is used as a preliminary screening before proceeding to speaker recognition [30]. Of course, our approach also relies on a few hypotheses. We assume that we know the identity of the speaker for the audio under analysis and, more importantly, that we have a reference set of pristine audio tracks for that identity. This scenario is similar to some previous works [27, 12] where, however, the model must be explicitly trained for identity recognition.

Figure 1: Making decisions based on large pre-trained models. Audio signals are fed to the model to extract the corresponding embeddings. The decision statistic is computed in the latent space, as the maximum similarity score between the audio under test and the audios of the reference set.

In this paper we move an important step forward, leveraging the power of large pre-trained models to develop a solution that is fully training-free. A few recent papers follow a similar path [5, 26] but, while relying on pre-trained features, they still need to train a classifier on a suitable dataset of real and synthetic data. In our method, instead, any dependency on fake samples is removed, ensuring the highest generalization.

II Proposed Framework

We consider a scenario in which i) the identity of the speaker is known and ii) a reference set, $\mathcal{R}$, of real audios of this same identity is available. Both hypotheses are certainly reasonable and easily met. It is difficult to imagine situations in which anonymous audio would be relevant to the point of having to verify its authenticity. Furthermore, in the age of social networks, many audio samples are usually available for any individual, and in any case, a person who is the subject of a possible deepfake may be more than willing to generate new samples to prove or disprove the attack. For example, a person accused of uttering sexist or racist phrases off-air can prove the audio snippet is a deepfake by releasing a large collection of new phrases for reference. The comparison between the audio under test and those coming from the reference set is carried out in the embedding space. The verification/detection procedure is outlined in Fig.1. Both the audio under test (orange) and the reference samples (blue) are provided as input to a pre-trained model to obtain the corresponding features in the embedding space, $x$ and $\left\{r_{i},i=1,\ldots,|\mathcal{R}|\right\}$, respectively. Then, similarity scores $s_{i}=sim(x,r_{i})$ are computed, using cosine-based similarity. Finally, the maximum similarity score $s=\max\left\{s_{i}\right\}_{i\in\mathcal{R}}$ is taken as the decision statistic and compared to an appropriate threshold to label the audio as real (the claimed identity is confirmed) or fake.

The maximum similarity criterion takes into account the intrinsic variability of speech samples generated by a single speaker, which can lead to pretty widespread features in the embedding space. To be validated as real, the test audio only needs to be very similar to one of the certified reference samples, while the requirement to be similar to all of them could be too stringent and ultimately misleading, an intuition also confirmed by the experiments carried out in [14, 12]. For this reason, it is very important to have an adequate number of different speech samples in the reference, so as to cover the entire region of the embedding space in which features of the same speaker can occur. On the other hand, as the number of reference samples grows, one of them might happen by chance to be too close to the deepfake under test, and the latter could be classified as real. The dependence of performance on the reference set size and on the decision threshold will be explored experimentally in subsection 3.3.

In this work, we choose not to design or re-train a model to extract dedicated features for the problem at hand, but rather to use generic features extracted from large pre-trained audio models. In Fig.1 this choice is represented by the ice symbol on the feature extractor, which means it is taken off-the-shelf, with frozen weights, and no training is required at any stage of the process. This choice is motivated by the impressive performance exhibited by large pre-trained models in many downstream applications, including some in image forensics [25, 13], and by the clear advantage of having a training-free solution, thus eliminating any dependence on training sets and their inevitable polarizations.

Model	Type	# Params	Speech (h)	Task
Wav2Vec2-xlsr	CNN + Transf.	2B	436K	SR
AudioCLIP	CNN + Transf.	134M	5K	LRL
LaionCLAP	Transformer	158M	9K	LRL
BEATs	Transformer	90M	5K	LRL

TABLE I: Pre-trained models used in our analysis with information on the type of architecture, the number of parameters, the hours of speech in training and the training task: Speech Recognition (SR) and Latent Representation Learning (LRL).

	ASVSpoof2019			ASVSpoof2021			InTheWild			Average			$\sigma$
Method	EER	t-DCF	AUC	EER	t-DCF	AUC	EER	t-DCF	AUC	EER	t-DCF	AUC	AUC
RawNet2 [32]	18.21	0.595	88.49	22.33	0.700	82.79	49.53	0.999	50.68	30.02	0.765	73.99	16.64
RawGATstMul [31]	1.07	0.031	99.90	12.52	0.211	92.07	52.53	0.994	47.36	22.04	0.412	79.78	23.14
AASIST [22]	0.83	0.027	99.92	11.20	0.231	96.16	43.01	0.901	60.12	18.35	0.386	85.40	17.94
TitaNet+LR [5]	16.04	0.499	83.96	21.87	0.593	78.13	46.74	0.945	53.26	28.22	0.679	71.78	13.31
SAMO [19]	0.88	0.026	99.71	12.09	0.212	94.74	46.94	0.952	55.17	19.97	0.397	83.21	19.92
WavLM [10]	15.12	0.493	92.58	0.00	0.000	99.99	23.52	0.597	84.30	12.88	0.363	92.30	6.40
ECAPA-TDNN [17]	20.04	0.682	86.52	4.74	0.159	99.09	27.37	0.848	76.60	17.39	0.562	87.40	9.20
TitaNet [23]	14.17	0.456	93.61	0.02	0.000	99.99	24.10	0.645	83.77	12.76	0.367	92.46	6.67
POI-Audio [27]	11.50	0.358	95.34	0.37	0.006	99.94	14.73	0.379	92.06	8.87	0.248	95.78	3.23
Wav2Vec2-xlsr-2B	8.65	0.244	96.68	5.15	0.101	97.98	19.19	0.550	86.95	11.00	0.298	93.87	4.92
AudioCLIP	32.09	0.845	75.85	2.98	0.076	99.52	38.52	0.998	64.93	24.53	0.640	80.10	14.43
LaionCLAP	16.61	0.488	91.14	3.74	0.109	99.46	11.76	0.348	95.16	10.77	0.315	95.25	3.39
BEATs	2.73	0.079	99.61	0.86	0.027	99.97	1.62	0.035	99.73	1.74	0.047	99.77	0.14

TABLE II: Comparison with SOTA in terms of EER, t-DCF, AUC. The first section of the table reports scores for supervised models trained on ASVSpoof2019. The second section is dedicated to speaker verification methods. The bottom section gives the performance of the proposed approach with different large pre-trained models. The last four columns report the average values for all metrics and, only for the AUC, also the standard deviation ($\sigma$). For each column, the best result is in bold.

Here we consider and compare several popular pre-trained models. They are described below, while a summary of their characteristics is reported in Table I. Wav2Vec2-xlsr [3] is a large-scale pre-trained model for speech representation learning. It is based on wav2vec 2.0 [4] and trained on nearly half a million hours of publicly available speech audio, an order of magnitude more data than previous models. Its main peculiarity is the focus on cross-lingual training, spanning 128 different languages from different world regions, which turns out to be important for some applications. AudioCLIP [21] extends the CLIP model [28], originally proposed for text-image representation alignment, by adding ESResNeXt [20] as audio head. Through this addition, the model is trained to learn aligned text-image, text-audio, and audio-image representation. The trained model allows the base architecture to operate also with the audio modality and enables cross-querying using images, text, and audio. LaionCLAP [34] is a large model trained on the LAION-Audio-630 dataset containing 633526 audio-text pairs collected from different sources. It implements a CLIP-like training strategy with image data replaced by audio data: audio and text data are fed to the network and features of correlated data are brought together while those of unrelated data are spaced apart. Audios of different lengths can be dealt with seamlessly. Finally, BEATs [11] proposes an innovative iterative pre-training framework for audio representation learning. The proposed architecture includes both a tokenizer and an audio self-supervised learning model which are trained jointly through a series of iterations, with each model contributing to the other’s learning process. In the end, semantically rich tokens are extracted along with insightful representations of the audio.

III Experimental results

For our experiments we consider the datasets that were proposed for the 2019 and 2021 editions of the Automatic Verification Spoofing and Countermeasure challenge, namely ASVSpoof2019 [33] and ASVSpoof2021 [35]. The former contains 7,355 real and 63,882 spoofed audios generated using 13 different speech synthesis methods and voice conversion models, while the latter contains 22,617 real tracks and 589,212 fake ones. We use only a subset of this dataset for which the speaker identity is known, for a total of 159,696 audio files. To simulate a real use case, the DF audio tracks are compressed and decompressed using general purpose audio compression tools.

We also use the InTheWild dataset [24], comprising 11,816 fake and 19,963 pristine audios and aiming at representing a real-world scenario. The dataset includes 54 identities, with real audios selected to closely match the corresponding fakes (e.g. similar speaker emotion, background noise, duration etc.). Since it is collected in the wild, this dataset provides no information on the techniques used for generation.

To measure performance we consider the metrics usually adopted in the specific literature: the equal error rate (EER) which is the point on the receiver operating characteristic curve (ROC) where false acceptance rate and false rejection rate are equal; the tandem detection cost function (t-DCF) which is a weighted sum of missing detection rate, false alarm rate, and prior class probabilities; and the area under the ROC curve (AUC).

III-A Comparison with the state-of-the-art

We compare our method with various state-of-the-art approaches from the current literature. We consider the following supervised methods: RawNet2 [32], RawGATstMul [31], AASIST [22], Titanet + LR [5], SAMO [19], all trained on the fake/real ASVSpoof2019 dataset. Then, we consider unsupervised approaches that work under the same hypothesis as ours, that is, they only assume to know a reference set of pristine audios (from 3 to 711, in this experiment, depending on the speaker, with an average value of 240). This set includes some speaker verification methods: WavLM [10], ECAPA-TDNN [17] and TitaNet [23], and also a speech deepfake detection method, POI-Audio [27] pre-trained on real audios of more than 5,000 identities.

All results are reported in Table II. As expected, supervised approaches, trained on the training partition of ASVSpoof2019, perform very well in the perfectly aligned scenario, that is, on ASVSpoof2019, but the performance worsen significantly as soon as new datasets are considered, to collapse altogether on the more challenging InTheWild dataset, with coin-tossing results except for AASIST, which shows an AUC just above $60\%$. Methods based on the prior knowledge of the individual under test (second section) are much more stable, with top performance on ASVSpoof2021 dataset and competitive results on the others, including InTheWild. The average AUC is always around $90\%$. Very similar results are observed also for methods of the third section, based on large-scale pre-trained models. This confirms the potential of the identity-based approach for deepfake detection, especially in terms of generalization, even in the absence of any specific training for the speaker verification or identity recognition tasks. To further underline this point, in the last column of the table we also report the standard deviation of the AUC, by which a clear gap between identity-based and supervised methods appears, with the only notable exception of AudioCLIP. The case of BEATs (last row) deserves some more words. Average results speak by themselves: they are the best under all metrics, with $1.74\%$ EER (second best $8.87\%$), $0.047$ t-DCF (second best $0.248$), and $99.77$ AUC (second best $95.78$). In fact, they are not only very good, but also extremely stable across all datasets, as also obvious by the very small standard deviation of the AUC shown in the last column. The quality of its representation, based on a semantic-rich acoustic tokenizer, appears to be much superior to that of other pre-trained models, opening the way to excellent results in this application.

III-B Representation quality of large models

The experimental results discussed before confirm the prevailing importance of a good learned representation, based on a large suitably pre-trained model, over specific adjustments of the detection method. To better investigate this point, and explain the impact of representation quality on performance, in Figure 2 we show a scatter plot of the 2D projections of 50 embedded vectors for each class. Such projections are obtained by means of the t-SNE dimensionality reduction technique [16]. Vectors are extracted from real and synthetic voices of four identities from the InTheWild dataset. For Wav2Vec2 and especially for AudioCLIP it is difficult to recognize clusters associated with specific speakers, nor is it possible to separate real samples from fake ones. Although these are only 2D projections, they indicate a poor representation quality, which explains some disappointing results. On the contrary, with LaionCLAP definite clusters begin to emerge, which then become extremely compact and well-separated with BEATs. They allow to easily distinguish each speaker from all the others as well as real audio from fake audio, explaining the excellent performance observed in Tab.II.

(a) Wav2Vec2-xlsr

(b) AudioCLIP

(c) LaionCLAP

(d) BEATs

Figure 2: T-SNE representation of real and fake feature embeddings of four identities taken from the InTheWild dataset.

It seems clear that BEATs ensures a largely improved quality of representation so we dedicate a few more words to describe it and possibly learn useful lessons. BEATs moves from the observation that successful image and video large models are trained to predict semantic-rich discrete labels, such as “horse”, “bicycle”, etc. On the contrary, state-of-the-art audio large models are trained to minimize a reconstruction loss, like the mean square error, that focuses on redundant and perceptually useless details rather than high-level semantic aspects. In other words, a measure of distortion that is only weakly correlated with the actual distortion perceived by the end user. This happens because it is difficult to associate semantic labels to audio fragments. The key step taken in BEATs is to train a semantic tokenizer jointly with the large audio model in an iterative modality. A random acoustic tokenizer is used to train the model, which then distills semantic knowledge useful for training a better tokenizer, and so on in a virtuous circle. Experimental results, including those presented here, demonstrate that this solution is very effective and that BEATs is able to provide audio representations with high semantic value.

III-C Impact of the reference set size

Finally, we study the impact of the reference set size on performance for the large pre-trained models described above and for the PoI-Audio method. In Figure 3, with reference to the InTheWild dataset, we report the AUC as a function of the number of speech segments available in the reference set for each identity under test. In all cases, the AUC improves steadily with increasing number of audios, but the ranking of methods remains the same over the whole range. With BEATs, an excellent performance ($>0.9$) is achieved already with five audios, while LaionCLAP, the second best method, reaches a comparable level only with 100 samples. Only AudioCLIP confirms to struggle, no matter how many reference audios are available, but this is easily explained by the poor representation quality observed in Fig.2.

Figure 3: AUC for the InTheWild dataset as a function of the number of audios in the reference set.

In Figure 4, only for the most interesting case of BEATs, we show the histograms of the maximum similarity scores for real and fake test samples (leave-one-out method) for various sizes of the reference set. Starting from $|\mathcal{R}|=5$ the real and fake histograms are well separated. As $|\mathcal{R}|$ grows, The former becomes more and more concentrated at large values, as finding a very similar reference becomes progressively easier. The latter, on the contrary, remains quite stable, which dispels the fear of accidentally finding very similar samples even when the test audio is fake, just because there are so many of them. Of course, the optimal decision threshold changes as the reference set size increases, but only marginally so, as shown in Figure 5, which allows to set the threshold in advance to a reasonable value, say, 0.85, that works well in all cases (again, except the case of a single reference).

IV Conclusion

Synthetic speech detection is becoming more and more relevant nowadays. Conventional detectors, based on supervised learning, provide an excellent performance on data they were trained for, but suffer dramatic performance losses in out-of-distribution real-world scenarios, where the model used to generate fake audios is not known. To ensure good generalization, we propose to use a one-class strategy, based on the only hypotheses to know the speaker identity and have a few reference audios at test time. In this context, we analyze a training-free solution based on the use of large scale models pre-trained on audio-related tasks. Results are extremely promising, competitive with supervised methods in aligned settings, and much superior to them in real-world scenarios. All this without any training on synthetic or field-specific data. Among the other interesting results, our experimental analysis demonstrates the excellent performance of BEATs, a large audio model pre-trained to exploit semantic information, which suggests equally good results for other semantic tasks.

Figure 4: Histograms of maximum similarity scores for BEATs on InTheWild dataset.

Figure 5: Accuracy of BEATs on InTheWild dataset as a function of threshold.