-
Second-Order Information Matters: Revisiting Machine Unlearning for Large Language Models
Authors:
Kang Gu,
Md Rafi Ur Rashid,
Najrin Sultana,
Shagufta Mehnaz
Abstract:
With the rapid development of Large Language Models (LLMs), we have witnessed intense competition among the major LLM products like ChatGPT, LLaMa, and Gemini. However, various issues (e.g. privacy leakage and copyright violation) of the training corpus still remain underexplored. For example, the Times sued OpenAI and Microsoft for infringing on its copyrights by using millions of its articles fo…
▽ More
With the rapid development of Large Language Models (LLMs), we have witnessed intense competition among the major LLM products like ChatGPT, LLaMa, and Gemini. However, various issues (e.g. privacy leakage and copyright violation) of the training corpus still remain underexplored. For example, the Times sued OpenAI and Microsoft for infringing on its copyrights by using millions of its articles for training. From the perspective of LLM practitioners, handling such unintended privacy violations can be challenging. Previous work addressed the ``unlearning" problem of LLMs using gradient information, while they mostly introduced significant overheads like data preprocessing or lacked robustness. In this paper, contrasting with the methods based on first-order information, we revisit the unlearning problem via the perspective of second-order information (Hessian). Our unlearning algorithms, which are inspired by classic Newton update, are not only data-agnostic/model-agnostic but also proven to be robust in terms of utility preservation or privacy guarantee. Through a comprehensive evaluation with four NLP datasets as well as a case study on real-world datasets, our methods consistently show superiority over the first-order methods.
△ Less
Submitted 13 March, 2024;
originally announced March 2024.
-
FLTrojan: Privacy Leakage Attacks against Federated Language Models Through Selective Weight Tampering
Authors:
Md Rafi Ur Rashid,
Vishnu Asutosh Dasu,
Kang Gu,
Najrin Sultana,
Shagufta Mehnaz
Abstract:
Federated learning (FL) has become a key component in various language modeling applications such as machine translation, next-word prediction, and medical record analysis. These applications are trained on datasets from many FL participants that often include privacy-sensitive data, such as healthcare records, phone/credit card numbers, login credentials, etc. Although FL enables computation with…
▽ More
Federated learning (FL) has become a key component in various language modeling applications such as machine translation, next-word prediction, and medical record analysis. These applications are trained on datasets from many FL participants that often include privacy-sensitive data, such as healthcare records, phone/credit card numbers, login credentials, etc. Although FL enables computation without necessitating clients to share their raw data, determining the extent of privacy leakage in federated language models is challenging and not straightforward. Moreover, existing attacks aim to extract data regardless of how sensitive or naive it is. To fill this research gap, we introduce two novel findings with regard to leaking privacy-sensitive user data from federated large language models. Firstly, we make a key observation that model snapshots from the intermediate rounds in FL can cause greater privacy leakage than the final trained model. Secondly, we identify that privacy leakage can be aggravated by tampering with a model's selective weights that are specifically responsible for memorizing the sensitive training data. We show how a malicious client can leak the privacy-sensitive data of some other users in FL even without any cooperation from the server. Our best-performing method improves the membership inference recall by 29% and achieves up to 71% private data reconstruction, evidently outperforming existing attacks with stronger assumptions of adversary capabilities.
△ Less
Submitted 25 May, 2024; v1 submitted 24 October, 2023;
originally announced October 2023.
-
FLShield: A Validation Based Federated Learning Framework to Defend Against Poisoning Attacks
Authors:
Ehsanul Kabir,
Zeyu Song,
Md Rafi Ur Rashid,
Shagufta Mehnaz
Abstract:
Federated learning (FL) is revolutionizing how we learn from data. With its growing popularity, it is now being used in many safety-critical domains such as autonomous vehicles and healthcare. Since thousands of participants can contribute in this collaborative setting, it is, however, challenging to ensure security and reliability of such systems. This highlights the need to design FL systems tha…
▽ More
Federated learning (FL) is revolutionizing how we learn from data. With its growing popularity, it is now being used in many safety-critical domains such as autonomous vehicles and healthcare. Since thousands of participants can contribute in this collaborative setting, it is, however, challenging to ensure security and reliability of such systems. This highlights the need to design FL systems that are secure and robust against malicious participants' actions while also ensuring high utility, privacy of local data, and efficiency. In this paper, we propose a novel FL framework dubbed as FLShield that utilizes benign data from FL participants to validate the local models before taking them into account for generating the global model. This is in stark contrast with existing defenses relying on server's access to clean datasets -- an assumption often impractical in real-life scenarios and conflicting with the fundamentals of FL. We conduct extensive experiments to evaluate our FLShield framework in different settings and demonstrate its effectiveness in thwarting various types of poisoning and backdoor attacks including a defense-aware one. FLShield also preserves privacy of local data against gradient inversion attacks.
△ Less
Submitted 10 August, 2023;
originally announced August 2023.
-
Spectra of s-neighbourhood corona of two signed graphs
Authors:
Tahir Shamsher,
Mir Riyaz ul Rashid,
S. Pirzada
Abstract:
A signed graph $S=(G, σ)$ is a pair in which $G$ is an underlying graph and $σ$ is a function from the edge set to $\{\pm1\}$. For signed graphs $S_{1}$ and $S_{2}$ on $n_{1}$ and $n_{2}$ vertices, respectively, the signed neighbourhood corona $S_{1} \star_s S_{2}$ (in short s-neighbourhood corona) of $S_{1}$ and $S_{2}$ is the signed graph obtained by taking one copy of $S_{1}$ and $n_{1}$ copies…
▽ More
A signed graph $S=(G, σ)$ is a pair in which $G$ is an underlying graph and $σ$ is a function from the edge set to $\{\pm1\}$. For signed graphs $S_{1}$ and $S_{2}$ on $n_{1}$ and $n_{2}$ vertices, respectively, the signed neighbourhood corona $S_{1} \star_s S_{2}$ (in short s-neighbourhood corona) of $S_{1}$ and $S_{2}$ is the signed graph obtained by taking one copy of $S_{1}$ and $n_{1}$ copies of $S_{2}$ and joining every neighbour of the $i$th vertex of $S_{1}$ with the same sign as the sign of incident edge to every vertex in the $i$th copy of $S_{2}$. In this paper, we investigate the adjacency, Laplacian and net Laplacian spectrum of $S_{1} \star_s S_{2}$ in terms of the corresponding spectrum of $ S_{1}$ and $ S_{2}$. We determine $(i)$ the adjacency spectrum of $S_{1} \star_s S_{2}$ for arbitrary $S_{1} $ and net regular $ S_{2}$, $(ii)$ the Laplacian spectrum for regular $S_{1} $ and regular and net regular $ S_{2}$ and $(iii)$ the net Laplacian spectrum for net regular $S_{1} $ and arbitrary $ S_{2}$. As a consequence, we obtain the signed graphs with $4$ and $5$ distinct adjacency, Laplacian and net Laplacian eigenvalues. Finally, we show that the signed neighbourhood corona of two signed graphs is not determined by its adjacency (resp., Laplacian, net Laplacian) spectrum.
△ Less
Submitted 4 May, 2023;
originally announced May 2023.
