-
Mosaic Memory: Fuzzy Duplication in Copyright Traps for Large Language Models
Authors:
Igor Shilov,
Matthieu Meeus,
Yves-Alexandre de Montjoye
Abstract:
The immense datasets used to develop Large Language Models (LLMs) often include copyright-protected content, typically without the content creator's consent. Copyright traps have been proposed to be injected into the original content, improving content detectability in newly released LLMs. Traps, however, rely on the exact duplication of a unique text sequence, leaving them vulnerable to commonly…
▽ More
The immense datasets used to develop Large Language Models (LLMs) often include copyright-protected content, typically without the content creator's consent. Copyright traps have been proposed to be injected into the original content, improving content detectability in newly released LLMs. Traps, however, rely on the exact duplication of a unique text sequence, leaving them vulnerable to commonly deployed data deduplication techniques. We here propose the generation of fuzzy copyright traps, featuring slight modifications across duplication. When injected in the fine-tuning data of a 1.3B LLM, we show fuzzy trap sequences to be memorized nearly as well as exact duplicates. Specifically, the Membership Inference Attack (MIA) ROC AUC only drops from 0.90 to 0.87 when 4 tokens are replaced across the fuzzy duplicates. We also find that selecting replacement positions to minimize the exact overlap between fuzzy duplicates leads to similar memorization, while making fuzzy duplicates highly unlikely to be removed by any deduplication process. Lastly, we argue that the fact that LLMs memorize across fuzzy duplicates challenges the study of LLM memorization relying on naturally occurring duplicates. Indeed, we find that the commonly used training dataset, The Pile, contains significant amounts of fuzzy duplicates. This introduces a previously unexplored confounding factor in post-hoc studies of LLM memorization, and questions the effectiveness of (exact) data deduplication as a privacy protection technique.
△ Less
Submitted 24 May, 2024;
originally announced May 2024.
-
Copyright Traps for Large Language Models
Authors:
Matthieu Meeus,
Igor Shilov,
Manuel Faysse,
Yves-Alexandre de Montjoye
Abstract:
Questions of fair use of copyright-protected content to train Large Language Models (LLMs) are being actively debated. Document-level inference has been proposed as a new task: inferring from black-box access to the trained model whether a piece of content has been seen during training. SOTA methods however rely on naturally occurring memorization of (part of) the content. While very effective aga…
▽ More
Questions of fair use of copyright-protected content to train Large Language Models (LLMs) are being actively debated. Document-level inference has been proposed as a new task: inferring from black-box access to the trained model whether a piece of content has been seen during training. SOTA methods however rely on naturally occurring memorization of (part of) the content. While very effective against models that memorize significantly, we hypothesize--and later confirm--that they will not work against models that do not naturally memorize, e.g. medium-size 1B models. We here propose to use copyright traps, the inclusion of fictitious entries in original content, to detect the use of copyrighted materials in LLMs with a focus on models where memorization does not naturally occur. We carefully design a randomized controlled experimental setup, inserting traps into original content (books) and train a 1.3B LLM from scratch. We first validate that the use of content in our target model would be undetectable using existing methods. We then show, contrary to intuition, that even medium-length trap sentences repeated a significant number of times (100) are not detectable using existing methods. However, we show that longer sequences repeated a large number of times can be reliably detected (AUC=0.75) and used as copyright traps. Beyond copyright applications, our findings contribute to the study of LLM memorization: the randomized controlled setup enables us to draw causal relationships between memorization and certain sequence properties such as repetition in model training data and perplexity.
△ Less
Submitted 4 June, 2024; v1 submitted 14 February, 2024;
originally announced February 2024.
-
Defending against Reconstruction Attacks with Rényi Differential Privacy
Authors:
Pierre Stock,
Igor Shilov,
Ilya Mironov,
Alexandre Sablayrolles
Abstract:
Reconstruction attacks allow an adversary to regenerate data samples of the training set using access to only a trained model. It has been recently shown that simple heuristics can reconstruct data samples from language models, making this threat scenario an important aspect of model release. Differential privacy is a known solution to such attacks, but is often used with a relatively large privac…
▽ More
Reconstruction attacks allow an adversary to regenerate data samples of the training set using access to only a trained model. It has been recently shown that simple heuristics can reconstruct data samples from language models, making this threat scenario an important aspect of model release. Differential privacy is a known solution to such attacks, but is often used with a relatively large privacy budget (epsilon > 8) which does not translate to meaningful guarantees. In this paper we show that, for a same mechanism, we can derive privacy guarantees for reconstruction attacks that are better than the traditional ones from the literature. In particular, we show that larger privacy budgets do not protect against membership inference, but can still protect extraction of rare secrets. We show experimentally that our guarantees hold against various language models, including GPT-2 finetuned on Wikitext-103.
△ Less
Submitted 15 February, 2022;
originally announced February 2022.
-
Opacus: User-Friendly Differential Privacy Library in PyTorch
Authors:
Ashkan Yousefpour,
Igor Shilov,
Alexandre Sablayrolles,
Davide Testuggine,
Karthik Prasad,
Mani Malek,
John Nguyen,
Sayan Ghosh,
Akash Bharadwaj,
Jessica Zhao,
Graham Cormode,
Ilya Mironov
Abstract:
We introduce Opacus, a free, open-source PyTorch library for training deep learning models with differential privacy (hosted at opacus.ai). Opacus is designed for simplicity, flexibility, and speed. It provides a simple and user-friendly API, and enables machine learning practitioners to make a training pipeline private by adding as little as two lines to their code. It supports a wide variety of…
▽ More
We introduce Opacus, a free, open-source PyTorch library for training deep learning models with differential privacy (hosted at opacus.ai). Opacus is designed for simplicity, flexibility, and speed. It provides a simple and user-friendly API, and enables machine learning practitioners to make a training pipeline private by adding as little as two lines to their code. It supports a wide variety of layers, including multi-head attention, convolution, LSTM, GRU (and generic RNN), and embedding, right out of the box and provides the means for supporting other user-defined layers. Opacus computes batched per-sample gradients, providing higher efficiency compared to the traditional "micro batch" approach. In this paper we present Opacus, detail the principles that drove its implementation and unique features, and benchmark it against other frameworks for training models with differential privacy as well as standard PyTorch.
△ Less
Submitted 22 August, 2022; v1 submitted 25 September, 2021;
originally announced September 2021.
-
Antipodes of Label Differential Privacy: PATE and ALIBI
Authors:
Mani Malek,
Ilya Mironov,
Karthik Prasad,
Igor Shilov,
Florian Tramèr
Abstract:
We consider the privacy-preserving machine learning (ML) setting where the trained model must satisfy differential privacy (DP) with respect to the labels of the training examples. We propose two novel approaches based on, respectively, the Laplace mechanism and the PATE framework, and demonstrate their effectiveness on standard benchmarks.
While recent work by Ghazi et al. proposed Label DP sch…
▽ More
We consider the privacy-preserving machine learning (ML) setting where the trained model must satisfy differential privacy (DP) with respect to the labels of the training examples. We propose two novel approaches based on, respectively, the Laplace mechanism and the PATE framework, and demonstrate their effectiveness on standard benchmarks.
While recent work by Ghazi et al. proposed Label DP schemes based on a randomized response mechanism, we argue that additive Laplace noise coupled with Bayesian inference (ALIBI) is a better fit for typical ML tasks. Moreover, we show how to achieve very strong privacy levels in some regimes, with our adaptation of the PATE framework that builds on recent advances in semi-supervised learning.
We complement theoretical analysis of our algorithms' privacy guarantees with empirical evaluation of their memorization properties. Our evaluation suggests that comparing different algorithms according to their provable DP guarantees can be misleading and favor a less private algorithm with a tighter analysis.
Code for implementation of algorithms and memorization attacks is available from https://github.com/facebookresearch/label_dp_antipodes.
△ Less
Submitted 29 October, 2021; v1 submitted 7 June, 2021;
originally announced June 2021.
-
Privacy Impact on Generalized Nash Equilibrium in Peer-to-Peer Electricity Market
Authors:
Ilia Shilov,
Hélène Le Cadre,
Ana Bušic
Abstract:
We consider a peer-to-peer electricity market, where agents hold private information that they might not want to share. The problem is modeled as a noncooperative communication game, which takes the form of a Generalized Nash Equilibrium Problem, where the agents determine their randomized reports to share with the other market players, while anticipating the form of the peer-to-peer market equili…
▽ More
We consider a peer-to-peer electricity market, where agents hold private information that they might not want to share. The problem is modeled as a noncooperative communication game, which takes the form of a Generalized Nash Equilibrium Problem, where the agents determine their randomized reports to share with the other market players, while anticipating the form of the peer-to-peer market equilibrium. In the noncooperative game, each agent decides on the deterministic and random parts of the report, such that (a) the distance between the deterministic part of the report and the truthful private information is bounded and (b) the expectation of the privacy loss random variable is bounded. This allows each agent to change her privacy level. We characterize the equilibrium of the game, prove the uniqueness of the Variational Equilibria and provide a closed form expression of the privacy price. In addition, we provide a closed form expression to measure the impact of the privacy preservation caused by inclusion of random noise and deterministic deviation from agents' true values. Numerical illustrations are presented on the 14-bus IEEE network.
△ Less
Submitted 18 January, 2021;
originally announced January 2021.
-
Risk-Averse Equilibrium Analysis and Computation
Authors:
Ilia Shilov,
Hélène Le Cadre,
Ana Busic
Abstract:
We consider two market designs for a network of prosumers, trading energy: (i) a centralized design which acts as a benchmark, and (ii) a peer-to-peer market design. High renewable energy penetration requires that the energy market design properly handles uncertainty. To that purpose, we consider risk neutral models for market designs (i), (ii), and their risk-averse interpretations in which prosu…
▽ More
We consider two market designs for a network of prosumers, trading energy: (i) a centralized design which acts as a benchmark, and (ii) a peer-to-peer market design. High renewable energy penetration requires that the energy market design properly handles uncertainty. To that purpose, we consider risk neutral models for market designs (i), (ii), and their risk-averse interpretations in which prosumers are endowed with coherent risk measures reflecting heterogeneity in their risk attitudes. We characterize analytically risk-neutral and risk-averse equilibrium in terms of existence and uniqueness , relying on Generalized Nash Equilibrium and Variational Equilibrium as solution concepts. To hedge their risk towards uncertainty and complete the market, prosumers can trade financial contracts. We provide closed form characterisations of the risk-adjusted probabilities under different market regimes and a distributed algorithm for risk trading mechanism relying on the Generalized potential game structure of the problem. The impact of risk heterogeneity and financial contracts on the prosumers' expected costs are analysed numerically in a three node network and the IEEE 14-bus network.
△ Less
Submitted 6 April, 2020;
originally announced April 2020.
