PANORAMIA: Privacy Auditing of Machine Learning Models without Retraining
Authors:
Mishaal Kazmi,
Hadrien Lautraite,
Alireza Akbari,
Mauricio Soroco,
Qiaoyue Tang,
Tao Wang,
Sébastien Gambs,
Mathias Lécuyer
Abstract:
We introduce a privacy auditing scheme for ML models that relies on membership inference attacks using generated data as "non-members". This scheme, which we call PANORAMIA, quantifies the privacy leakage for large-scale ML models without control of the training process or model re-training and only requires access to a subset of the training data. To demonstrate its applicability, we evaluate our…
▽ More
We introduce a privacy auditing scheme for ML models that relies on membership inference attacks using generated data as "non-members". This scheme, which we call PANORAMIA, quantifies the privacy leakage for large-scale ML models without control of the training process or model re-training and only requires access to a subset of the training data. To demonstrate its applicability, we evaluate our auditing scheme across multiple ML domains, ranging from image and tabular data classification to large-scale language models.
△ Less
Submitted 12 February, 2024;
originally announced February 2024.
