A Survey on Federated Unlearning: Challenges, Methods, and Future Directions

Existing MU research papers predominantly rely on the following unlearning principles to make the distribution of the model $\mathcal{M}(D)$ identical to the distribution of the model $\mathcal{M}(D_{r})$ (Bourtoule et al., 2021).

Retraining. is a process training from a model free from the influence of data from $D_{u}$ on the dataset $D_{r}$, essentially starting from scratch. In this method, a newly trained model $\mathcal{M}(D_{r})$ does not have any information about $D_{u}$. However, this process is both time-consuming and resource-intensive because it discards the model $\mathcal{M}(D)$ on $D$ containing the contribution of $D_{r}$ dataset.

Fine-tuning. uses the remaining dataset $D_{r}$ to optimize the model $\mathcal{M}(D)$ and reduce the impact of data from $D_{u}$. However, this process involves multiple iterations, leading to increased computational and communication costs.

Gradient ascent. represents a reverse learning process. In machine learning, the model $\mathcal{M}(D)$ is trained by minimizing the loss using gradient descent. Conversely, the unlearning process involves the application of gradient ascent to maximize the loss. However, this method can easily lead to catastrophic forgetting. As a result, many studies introduce constraints to preserve memory.

Multi-task unlearning seeks to not only eliminate the influence of $D_{u}$ but also to reinforce the acquisition of knowledge from the remaining data $D_{r}$. In the course of these endeavors, most studies aim to strike a balance between the erasure effect and the retention effect.

Model scrubbing. applies a “scrubbing” transformation $\mathscr{H}$ to the model $\mathcal{M}(D)$ to ensure that the unlearned model closely approximates the perfectly retrained model with only $D_{r}$, as expressed by $\mathscr{H}(\mathcal{M}(D))\approx\mathcal{M}(D_{r})$ (Ginart et al., 2019). When defining the scrubbing method $\mathscr{H}$, most approaches rely on a quadratic approximation of the loss function. Specifically, for model parameters $\theta$ and $\phi$, the gradient of the loss function of a given data point $D_{x}$ satisfies

where $\mathcal{H}_{D_{x}}(\theta)$ is positive semi-define. The scrubbed model becomes the new optimum by setting $\nabla f_{D_{r}}(\mathscr{H}_{D_{r}}(\theta))=0$, yielding the equation:

$\mathscr{H}$ can perform a Newton step and can be derived under various theoretical assumptions (Fraboni et al., 2022) (Golatkar et al., 2020). However, the challenge of this method lies in computing the Hessian matrix, which is infeasible for high-dimensional models. Therefore, some approaches aim to compute an approximation of the Hessian.

Synthetic data. is a method that replaces certain data with synthetic data to help the model ”forget” specific information. An example of this approach involves generating synthetic labels for the data within $D_{u}$ and then combining them with the data in $D_{u}$ for training to accomplish unlearning. This method disentangles the impact of certain data from the model, hel** to eliminate the influence of specific information while retaining the model’s overall performance.

Verification methods aim to confirm whether data intended for deletion has indeed been effectively unlearned. Currently, these methods can be classified as outlined below:

Model performance. The most straightforward approach is to evaluate the model performance on the target client’s data and test data to assess how effectively the data has been unlearned and how robustly the unlearned model is maintained. The evaluation metrics encompass accuracy, loss, and statistical errors.

Model discrepancy. Another approach to assess unlearning performance is by evaluating the discrepancy between the trained model and the unlearned model. This discrepancy can be measured using metrics such as Euclidean distance, KL-divergence, L2 distance, Wasserstein distance, and angle-based distance.

Execution efficiency. In addition, the time taken for the unlearning process, measured in terms of rounds, runtime, or speed-up ratio compared to a baseline, as well as memory consumption, can be used to evaluate the efficiency of the unlearning algorithm.

Attack performance. As introduced in Section 3.4, membership inference attacks can be used to determine whether a particular data was used during the training of a model. Therefore, by executing MIA on the unlearned model over unlearned data, the attack success rate (ASR) can be used to evaluate how effectively the data has been unlearned. Poorer performance by the MIA indicates that the influence of the unlearned data on the global model has diminished. Similarly, in the context of backdoor attacks, by injecting backdoors into the unlearned data and following the unlearning procedure, effective unlearning should disrupt the relationship between the trigger pattern and the backdoor class. The ASR can also be used to evaluate how effectively the backdoor is removed by unlearning. An empirical study on these metrics can be referred to in (Nguyen et al., 2024).

The participants involved in federated learning (Kairouz et al., 2021; McMahan et al., 2017; Yang et al., 2019) can be categorized into two categories: (i) a set of $n$ clients denoted as $\mathcal{U}={u_{1},u_{2},\dots,u_{n}}$, where each client $u_{i}\in\mathcal{U}$ possesses its local dataset $\mathcal{D}_{i}$, and (ii) a central server represented as $S$. A typical FL scheme works by repeating the following steps until training is stopped (Kairouz et al., 2021). (i) Local model training: each FL client $u_{i}$ trains its model $\mathcal{M}_{i}$ using the local dataset $\mathcal{D}_{i}$. (ii) Model uploading: each FL client $u_{i}$ uploads its locally trained model $\mathcal{M}_{i}$ to the central server $S$. (iii) Model aggregation: the central server $S$ collects and aggregates clients’ models to update the global model $\mathcal{M}$. (iv) Model updating: the central server $S$ updates the global model $\mathcal{M}$ and distributes it to all FL clients.

The revelation of a participant’s local model poses a direct threat to the fundamental privacy guarantee of standard federated learning (Zhu et al., 2019). Thus, privacy-preserving aggregation protocols (Bell et al., 2020; Guo et al., 2021; Bonawitz et al., 2017; Liu et al., 2023b) are essential to maintain the security and privacy of the model aggregation process in Step iii of FL. Additionally, FL is susceptible to poisoning attacks (Lyu et al., 2020) (see Section 3.4 for more details). In these attacks, malicious clients manipulate the global model by sending poisoned model updates to the server during Step ii, to affect global model performance or inject backdoors. Therefore, malicious-client detection mechanisms (Li et al., 2020a; Cao et al., 2021; Zhang et al., 2022a; Shen et al., 2016) are imperative to differentiate between malicious and benign clients.

First proposed by Shokri et al. (Shokri et al., 2017), the fundamental idea behind MIA is to determine whether a particular record was used during the training of a target model. This is predicated on the observation that data samples present in the training set will lead the model to produce outputs with higher confidence scores. Consequently, an adversary can train a separate model for binary classification, designating outputs as either “member” (indicating that the data was part of the training set) or “non-member” (indicating that the data was not part of the training set). This potential to distinguish between member and non-member records poses a threat to data privacy. Remarkably, MIA does not require knowledge of the target model’s specific architecture or the distribution of its training data. Relying on the shadow models, a series of shadow training datasets $D^{\prime}_{1},\cdots,D^{\prime}_{k}$ and disjointed shadow test datasets $T^{\prime}_{1},\cdots,T^{\prime}_{k}$ can be synthesized to mimic the behavior of the target model so as to train the attack model. Evaluating the unlearning effectiveness can be achieved by performing MIA on the unlearned model with the unlearned data. The ASR serves as an indicator of how well the data has been unlearned. A decrease in the MIA’s performance suggests that the influence of the unlearned data on the global model has been successfully reduced.

Backdoor attacks embed a distinct pattern or “trigger” into portions of the training data (Li et al., 2022). The trigger can be a small patch or sticker that is visible to humans (Gu et al., 2017; Liu et al., 2018), or the value perturbation of benign samples indistinguishable from human inspection (Li et al., 2020b; Bagdasaryan and Shmatikov, 2021; Saha et al., 2020). When the model is subsequently trained or fine-tuned on this, its behavior remains typical for standard inputs. Yet, upon detecting an input that contains this covert trigger, the model will yield malicious behaviors that align with the attacker’s intentions. Backdoor attacks are particularly concerning because they can remain dormant and undetected until the attacker chooses to exploit them. In the context of unlearning, injecting backdoors into the unlearned data and then applying the unlearning procedure should effectively disrupt the relationship between the trigger pattern and the backdoor class. The ASR can be utilized to assess the effectiveness of the unlearning process in removing the backdoor. A lower ASR would indicate that the backdoor has been successfully eliminated.

As previously mentioned, standalone server unlearning typically depends on the utilization of stored historical data, which may include gradients, global models (Wu et al., 2023d, 2022b; Zhang et al., 2023e; Guo et al., 2023; Jiang et al., 2024; Huynh et al., 2024), contribution information (Zhang et al., 2023e), or intermediate information necessary for constructing a random forest (Liu et al., 2021a). This category necessitates a significant amount of memory on the server, potentially limiting its practical application in large-scale FL systems with complex ML models.

In the case of FedRecovery (Zhang et al., 2023e), the server retains historical data from all clients and quantifies their contributions in each round based on gradient residuals. When a target client requests to leave, the server systematically removes its contributions from all FL rounds through a fine-tuning process. Based on FedRecovery (Zhang et al., 2023e), Crab (Jiang et al., 2024) achieves a more efficient recovery based on (i) selective historical information rather than all historical information and (ii) a historical model that has not been significantly affected by malicious clients rather than the initial model. Additional constraints can be introduced to further guide the recovery process, such as a penalty term based on projected gradients (Shao et al., 2024; Fu et al., 2024), randomly initialized degradation models (Zhao et al., 2023), and estimated skew (Huynh et al., 2024). The approach of eliminating the contribution of the target client is more straightforward in (Wu et al., 2023d, 2022b), where the server directly averages the models of the remaining clients. Strategic retraining based on the change of sampling probability is adopted for fast and efficient recovery (Tao et al., 2024). To mitigate the potential decrease in accuracy due to the averaging process in the averaged model, knowledge distillation is employed. This technique facilitates the transfer of information from the trained model to the unlearned model, hel** to preserve performance. Consequently, these designs adhere to a multi-task unlearning approach. In VERIFI (Gao et al., 2022b), after receiving the gradients from all clients, including those from the target client, are uploaded to the server. The server then amplifies the gradients from the remaining clients and diminishes the gradients of the target client, to reduce the impact of target clients, hence achieving unlearning.

Apart from the above-mentioned works, RevFRF (Liu et al., 2021a), as shown in Figure 6, focuses on federated random forest training. To remove a client, the server first identifies nodes affected by the target client and subsequently eliminates these affected nodes until reaching the leaf node. Subsequently, the server reconstructs the affected branches using previously stored intermediate information, rather than instructing the remaining clients to undergo retraining. In particular, in the worst-case scenario where the revoked node is the root node, the server has to reconstruct the entire random decision tree. Unlearning within a federated clustering setting is explored in SCMA (Pan et al., 2022), where each client maintains a vector to denote its local clustering result. These vectors are then aggregated by the server to form a global clustering outcome. Eliminating a client is straightforward by assigning a zero vector to the unlearned client and then re-aggregating all vectors.

Limitations. Server-standalone unlearning, which relies on historical data stored on the server, lacks real-time input from remaining clients during the unlearning process. This limitation may result in slightly lower unlearning performance compared to client-aided unlearning. This characteristic could impede the applicability of server-standalone unlearning in complex ML models or Non-IID FL settings, where there is a notable bias, affecting the overall efficacy of the unlearning process. Furthermore, server-standalone unlearning may lack responsiveness to changes in data and client behavior, as it solely relies on historical information. This can limit its adaptability in dynamic environments where real-time data and client interactions are crucial.

Unlearning performed by the server and remaining clients typically offers greater potential compared to standalone server unlearning. This is because the remaining clients contribute valuable information about the remaining data, which enables the server to enhance its unlearning process. In this context, the server may or may not have access to historical information.

This research direction is arguably pioneered by the design of FedEraser (Liu et al., 2021b), as shown in Figure 7. The core concept of FedEraser is that the current global model can be reconstructed using only the initial model and historical clients’ gradients at each round. Consequently, unlearning boils down to eliminating the influence of the target client on the historical gradients, i.e., calibrating historical gradients. To achieve this goal, for a historical FL round $i$ with the stored gradients $G_{i}$ and a calibrated global model $M_{i-1}$ for the previous round, each remaining client $u_{j}$ calculates a local calibration direction $c_{i}^{j}$ based on its local data $D_{j}$ and $M_{i-1}$. The local calibration directions are then aggregated by the server to derive a global calibration direction $c_{i}$, which enables the server to calculate calibrated historical gradients $G_{i}^{\prime}$ and to obtain a calibrated global model $M_{i}$ via $G_{i}^{\prime}$. This iterative process continues round-by-round until all historical gradients are successfully calibrated, resulting in the server obtaining a final calibrated global model, eliminating the influence of the target client. To enhance unlearning efficiency, the server stores clients’ gradients at intervals of every $\Delta t$ rounds, leading to a trade-off between unlearning performance and resource consumption in terms of memory and computation. Improve upon FedEraser, Crab (Jiang et al., 2024) and Fast-FedUL (Huynh et al., 2024) optimize storage efficiency by selectively storing important gradients, while Sharding Eraser (Lin et al., 2024a) compress storage using coding-based techniques. A similar idea is adopted in (Wang et al., 2024b) focusing on ranking tasks instead of classification tasks. Building upon the unlearning concept introduced in FedEraser, an efficiency-enhancing technique is employed in FRU (Yuan et al., 2023) for federated recommendations. In FRU, only the important updates to clients’ item embeddings are stored. In line with FedEraser (Liu et al., 2021b) and FRU (Yuan et al., 2023), FedRecover (Cao et al., 2023) also entails the storage of historical gradients and global models. In FedRecover, to prevent the remaining clients from computing exact model updates for fine-tuning, which can lead to significant computational overhead, the server calculates updates for the remaining clients using historical gradients and global models, as described below:

where $\mathbf{H}_{t}^{i}=\int_{0}^{1}\mathbf{H}\left(\overline{\boldsymbol{w}}_{t}+z\left(\hat{\boldsymbol{w}}_{t}-\overline{\boldsymbol{w}}_{t}\right)\right)dz$ is an integrated Hessian matrix for the $i$th client in the $t$th round. Denote the global-model difference in the $t$th round as $\Delta\boldsymbol{w}_{t}=\hat{\boldsymbol{w}}_{t}-\overline{\boldsymbol{w}}_{t}$ and the model-update difference of the $i$th client in the $t$th round as $\Delta\boldsymbol{g}_{t}^{i}=\boldsymbol{g}_{t}^{i}-\overline{\boldsymbol{g}}_{t}^{i}$. The Hessian matrix $\tilde{\boldsymbol{H}}_{t}^{i}$ for the $i$th client in the $t$th round can be approximated as

where $\Delta\boldsymbol{W}_{t}=\left[\Delta\boldsymbol{w}_{b_{1}},\Delta\boldsymbol{w}_{b_{2}},\cdots,\Delta\boldsymbol{w}_{b_{s}}\right]$ and $\Delta\boldsymbol{G}_{t}^{i}=\left[\Delta\boldsymbol{g}_{b_{1}}^{i},\Delta\boldsymbol{g}_{b_{2}}^{i},\cdots,\Delta\boldsymbol{g}_{b_{s}}^{i}\right]$ are L-BFGS buffers (Nocedal, 1980) maintained by the server. Nonetheless, these approximations introduce estimation errors over rounds. Therefore, the remaining clients are periodically tasked with computing their exact model updates to correct these approximations, based on an adaptive abnormality threshold. A more straightforward retraining-based method is employed in SIFU (Fraboni et al., 2022). In SIFU, the fundamental concept of unlearning is to identify the most recent global model using a bounded sensitivity metric calculated from historical contributions. Subsequently, the remaining clients retrain based on the identified model.

Other approaches concentrate on unlearning without reliance on historical updates. In SFU (Li et al., 2023d), upon receiving an unlearning request, the server refines the global model using gradient information provided by the target client and representation matrix information provided by other clients. In KNOT (Su and Li, 2023), clients are grouped into clusters based on their training time and model sparsity. Clients within the same cluster collectively aggregate and update the global model asynchronously. When a client requests to leave, only clients within the same cluster require retraining. A similar structure is adopted in FedCIO (Qiu et al., 2023) where clients are clustered according to their data distribution. Differing from conventional FL systems, HDUS (Ye et al., 2023) operates without a central server, as shown in Figure 8. Instead, each client possesses their own neighboring distilled models, referred to as seed models. When a client requests to leave, the adjacent clients simply delete the seed model of the unlearned client. For predictions, an ensemble learning method is employed to combine the outputs of the primary model with those of the seed models. Incentive mechanisms along with game theoretical analysis in FU systems is presented in (Ding et al., 2023; lin2024incentive，ding2023strategic).

Client-aided unlearning inherently depends on the involvement of remaining clients and their updates, which can be a vulnerability in dynamic environments where client participation fluctuates. Additionally, this unlearning method can be slow, as it relies on all remaining clients, often resource-constrained devices, and is limited by the bandwidth of the FL system. This could lead to inefficiencies, particularly in cross-device scenarios with frequent client turnover or limited system resources.

To unlearn the partial data of the target client, retraining is the most straightforward approach. To mitigate the computational cost of starting from scratch with retraining, one solution is to roll back the global model to a state where it has not been significantly influenced by the target client. From this point, all FL clients can conduct the retraining process. For instance, in Exact-Fun (Xiong et al., [n. d.]), where FL models are quantized, when a client requests to leave, the client calculates a new model based on the remaining data. If the original model matches the new quantized model, signifying that the removal has no impact, the FL model remains unchanged. Otherwise, retraining is required to eliminate the influence of the unlearned data. In ViFLa (Fan et al., 2022), which is essentially a machine unlearning scheme, training samples are segmented into different groups, with each group representing an FL client. Hence, ViFLa can simulate an FU process in this context. The local model is trained using ring-based SQ-learning for LSTM, and weighted aggregation is determined by KL-attention scores. The historical model parameters represented by states over a ring are stored. To unlearn partial data, each client removes unlearned data and computes the new updates. Based on these new updates, the server identifies a previous state from which the remaining clients continue their training. A similar concept of identifying the optimal previous state for retraining is also present in SIFU (Fraboni et al., 2022), as discussed earlier in Section 4.1.2. In SCMA (Pan et al., 2022), a straightforward approach to unlearning partial data involves naive retraining. Each client maintains a vector representing its local clustering result, and these vectors are aggregated by the server to create a global clustering outcome. To unlearn partial data, SCMA entails each client calculating a new local vector, i.e., retraining, and then re-aggregating all vectors.

Fine-tuning and multi-task learning are popular approaches for FU as well. As an example, in FRAMU (Shaik et al., 2023a), the server aggregates fine-tuned local models and attention scores. Using these scores, it filters out irrelevant data points and updates the global model. The attention scores are acquired through local reinforcement learning applied to dynamic data. In FedLU (Zhu et al., 2023), designed for FL over knowledge graphs where embeddings are aggregated instead of gradients as in standard FL, the unlearning of partial data is accomplished through iterative optimization of local embeddings. This process follows the multi-task unlearning concept, involving unlearning over the local model and learning over the global model. In FedME² (Xia et al., 2023), clients engage in multi-task learning to optimize the loss of the local model, the loss from an MIA-like evaluation model, and a penalty term that accounts for the difference between the local model and the global model. In (Wang et al., 2024e), unlearning is conducted by optimizing model performance on the remaining dataset while considering bias caused by the unlearned data.

Model scrubbing-based methods are commonly employed for unlearning partial data. In (Liu et al., 2022c), the model scrubbing technique is applied to the target client to locally unlearn the partial data, involving Hessian matrix computations, with enhanced computational efficiency through an approximate diagonal empirical Fisher Information Matrix (FIM). In Forsaken (Liu et al., [n. d.]), dummy gradients are computed to align the confidence vector of the unlearned model with that of a perfectly unlearned model. Forsaken+ (Ma et al., 2022) minimizes the distance between the posteriors of the data to be forgotten and those of non-member data for unlearning. FedAU (Gu et al., 2024b) relies on the linear combination to approximate the unlearned model utilizing a pre-computed auxiliary model during the learning process. (Gu et al., 2024a) focus on feature unlearning by minimizing local feature sensitivity through model scrubbing. A similar approach of local unlearning followed by aggregation is described in CONFUSE (Meerza et al., 2024) for multi-task unlearning at different levels. FFMU (Che et al., 2023) treats data removals as perturbations on the dataset, employing random smoothing (RS) (Cohen et al., 2019) to obtain a smoother model to simulate an unlearning process. In particular, FFMU aligns with the fundamental idea presented in PCMU (Zhang et al., 2022b), which involves randomized gradient smoothing combined with gradient quantization as follows.

where $\mathcal{D}=\mathcal{N}\left(0,\sigma^{2}I\right)$ is a Gaussian distribution, $Q$ is a gradient quantization to map each dimension of the continuous gradient $G(x,y)\in\mathbb{R}^{T}$ over a discrete three-class space $\{-1,0,1\}$, for mimicking the classification in the randomized smoothing for certified robustness. $S$ is a smooth version of $Q$, and returns whichever gradient classes $Q^{t}$ is most likely to return when $\bar{G}$ is perturbed by noise $\varepsilon$. Extending FFMU from PCMU poses a challenge due to FL’s privacy requirements, limiting server access to clients’ local training data and affecting the certified data removal radius and budget in the global model. Therefore, by leveraging the theory of nonlinear functional analysis, the local MU models $g(x;q)$ in FFMU are reformulated as output functions of a Nemytskii operator $O(q)(x)$ where $q=Q(\bar{G})+\varepsilon$. In this way, the global unlearned model with bounded errors can maintain the certified radius and budget of data removals of the local unlearned models within a distance $\frac{(K-1)Cd}{\sqrt{2\pi}K\delta}$ (see (Che et al., 2023) for more details).

Some FU approaches are proposed with the use of synthetic data and gradient ascent principles. For instance, in UKRL(Xu et al., 2023a), unlearning is conducted by training on perturbed unlearned data. In FedAF (Li et al., 2023a), shown in Figure 9, synthetic labels are generated for the data to be unlearned. A trusted third party creates random teacher models, and ensemble predictions from these models to provide synthetic labels for the unlearned data. Training is then conducted using this data with synthetic labels to achieve unlearning. Note that a multi-task approach is also employed in FedAF, where an additional task is introduced to retain the memory of the remaining data. In another work (Alam et al., 2023), which focuses on how adversaries can stealthily perform backdoor unlearning to evade server detection, the gradient ascent method is employed. The loss from the local benign dataset is utilized to constrain unbounded losses during the unlearning process based on gradient ascent.

Unlearning is also applicable in Bayesian federated learning systems. In Forget-SVGD (Gong et al., 2022a), when a client requests to leave, the target client leverages the remaining data to compute the posterior probability, approximated through variational inference (VI), and performs an extra FL round for model updating. Similar unlearning methods are employed in (Gong et al., 2022b; Gong, 2023). BVIFU (Gong et al., 2021) shares a core concept with Forget-SVGD, employing exponential family distributions in VI to approximate posterior probability. Unlike retraining-based unlearning in Bayesian FL, BFU (Wang et al., 2023c) introduces a multi-task unlearning approach. It employs a parameter self-sharing method to balance between forgetting the unlearned data and remembering the knowledge learned by the original model, where probability distributions are approximated by a neural network.

In addition to the aforementioned approaches, the work in (Wang et al., 2022) specializes in unlearning a specific type of partial data, particularly focusing on a category within the training dataset, i.e., a class. Typically initiated by the server, this unlearning process involves the application of quantization and pruning techniques. Specifically, the locally trained CNN model takes private images as input and produces feature map scores that assess the relationship between each channel and category. These scores are transmitted to the central server and aggregated into global feature map scores. The server utilizes TF-IDF to evaluate the relevance scores between channels and categories and creates a pruner to perform selective pruning on the most distinguishing channels of the target category. Subsequently, normal federated training proceeds with the exclusion of training data associated with the target category.

Limitations. Unlearning partial data through active unlearning is notably complex among all unlearning targets. It requires the removal of specific data while maintaining model performance on the remaining data. This complexity often results in more intricate algorithms, leading to increased computational and communication costs. Furthermore, this setting is susceptible to attacks based on over-unlearning, as identified in (Hu et al., 2023b), where adversaries can exploit the unlearning process to enhance the effects of remaining poisoned data, thus facilitating poisoning attacks. This vulnerability underscores the need for careful consideration in implementing active partial data unlearning.

As mentioned earlier, unlearning through gradient ascent is a versatile approach suitable for both partial data and target client unlearning. As detailed in (Halimi et al., 2022), the target client employs gradient ascent to maximize the local loss, subject to constraints determined by the reference model provided by the remaining clients. Specifically, during FL training, a client’s objective is to address the following optimization problem:

where $\mathcal{L}(w;(x_{i},y_{i}))$ represents the loss function, which calculates the prediction error for an individual example $(x_{i},y_{i})$ from the dataset $D$, using the model parameters $w$. The unlearning method designed in (Halimi et al., 2022) is to reverse this learning process. That is, during unlearning, instead of learning model parameters that minimize the empirical loss, the client $i$ strives to learn the model parameters to maximize the loss. Additionally, to prevent the process of gradient ascent from producing an arbitrary model similar to a random model, the average of the other clients’ models, i.e., $w_{ref}=\frac{1}{N-1}\sum\limits_{i\neq j}w_{j}$, is used as a reference model, and an $\ell_{2}$-norm ball around the reference model is employed to limit the unbounded loss. Thus, during unlearning, the client solves the following optimization problem:

Similarly to constraints, concurrent boost training on the remaining data (Wang et al., 2024c) and reference models (Bano et al., 2023) can also be adopted as mitigation methods. To reduce the computation overhead, unlearning is conducted based on gradient ascent on distilled dataset (Dhasade et al., 2023). Apart from the unlearning principle based on gradient ascent, 2F2L (** et al., 2023) and Appro-Fun (Xiong et al., 2024) adopt model scrubbing methods similar to previous works such as (Liu et al., 2022c) and (Cao et al., 2023). To enhance computational efficiency, the complex task of Hessian matrix inversions is approximated using a pre-trained deep neural network and Taloy expansion.

Limitations. Unlearning an entire client through active learning poses limitations in scenarios where the unlearning request comes from the client to be unlearned. In such cases, the client must remain in the system for unlearning, conflicting with the common “request then leave” behavior in FL systems. Moreover, this method necessitates the unlearned client’s continued participation alongside remaining clients, which is not cost-effective, especially in large-scale and resource-constrained FL systems.

To ensure the “right to verify,” it is imperative that federated unlearning schemes provide clients with the ability to confirm the successful unlearning of their data. Regrettably, this aspect has received limited attention in existing FU literature. In EMA (Huang et al., 2022), the target client employs various metrics, including correctness, confidence, and negative entropy, to assess the performance of the audited dataset concerning the global model. These metrics are then ensembled to determine whether they meet a predefined threshold, serving as an indicator of whether the target client’s data has been effectively unlearned.

Another noteworthy contribution in this domain is VERIFI (Gao et al., 2022b). VERIFI introduces two non-invasive verification methods that distinguish themselves from invasive techniques involving the injection of backdoors or watermarks, which manipulate the original data. In contrast, the verification methods proposed in VERIFI operate without modifying the data itself. These methods revolve around tracking a subset of unlearned data known as “markers,” selected based on specific criteria. The criteria encompass two primary categories: (i) Forgettable Memory, where markers are identified as a representative subset incurring a high variance of local training loss, and (ii) Erroneous Memory, which designates markers as incorrectly predicted samples labeled as erroneous. By actively monitoring the unlearned model’s performance on these markers, clients can effectively verify whether the unlearning process has successfully removed the unlearned data.

The outcome of the verification conducted by the server plays an important role in determining when to stop the unlearning process within the FU system. For instance, FRAMU (Shaik et al., 2023a) terminates unlearning when the difference between two consecutive global models becomes smaller than a predefined parameter. KNOT (Su and Li, 2023) concludes unlearning based on the required validation accuracy and the standard deviation across a recent history of such validation accuracies. ViFLa (Fan et al., 2022) and SCMA (Pan et al., 2022) end unlearning when the model converges. FedLU (Zhu et al., 2023) relies on prediction results derived from knowledge, while FFMU (Che et al., 2023) assesses whether data removals exceed a certified budget.

While the remaining FU works reviewed in this survey do not explicitly introduce verification mechanisms, the verification metrics employed in these works to assess unlearning performance could provide valuable insights for future research. We have summarized these metrics adopted in reviewed FU works in Table 3.

Accuracy-based metrics over unlearned data are the most commonly utilized metrics in the reviewed FU works. For example, they are used in works such as (Wang et al., 2023c; ** et al., 2023; Zhang et al., 2023e; Fraboni et al., 2022; Cao et al., 2023; Liu et al., 2021b; Halimi et al., 2022; Xia et al., 2023; Li et al., 2023a; Wu et al., 2023d; Alam et al., 2023; Liu et al., 2022c; Zhu et al., 2023). Metrics based on running time are employed to assess the efficiency of unlearning algorithms, as demonstrated in (Wang et al., 2023c; Zhang et al., 2023e; Fraboni et al., 2022; Cao et al., 2023; Liu et al., 2021b; Xia et al., 2023; Liu et al., [n. d.]; Ma et al., 2022; Li et al., 2023a; Liu et al., 2022c; Pan et al., 2022). Furthermore, some works rely on verification through backdoor attacks, as in (Li et al., 2023d; Cao et al., 2023; Yuan et al., 2023; Halimi et al., 2022; Ma et al., 2022; Wu et al., 2023d; Alam et al., 2023), while others use membership inference attacks, as seen in (** et al., 2023; Zhang et al., 2023e; Liu et al., 2021b; Ma et al., 2022). The difference between the unlearned model and the retrained model is adopted to evaluate unlearning performance in (Wang et al., 2023c; Ma et al., 2022; Gong et al., 2022b).

By surveying the existing FU literature, we can make several observations regarding verification as follows:

• •

  Only a few research studies on FU take “who-verify” into account, and the verification in almost all FU schemes remains at an experimental assessment level.

• •

  Most FU methods rely on the assumption that verification is conducted by the server rather than the clients.

• •

  There are no standard or widely adopted methods for proof of unlearning.

These observations indicate the need for research into ”who-verify” and the development of efficient and robust verification methods conducted by different participants, especially client-verify methods. For instance, when considering MLaaS with unlearning services, clients must be allowed to verify if their data has been unlearned and its impact on the FL model has been removed. Only when the data removal adheres strictly to the specified unlearning request can the trustworthiness of the federated unlearning system be maintained.

In many FU works, historical information is essential for facilitating the unlearning process. Nevertheless, this implies that the server needs to store a substantial volume of data, leading to significant memory consumption. This historical information can be gradients and global models (Cao et al., 2023; Fraboni et al., 2022; Liu et al., 2021b; Yuan et al., 2023; Wu et al., 2023d; Halimi et al., 2022), gradient residuals (Zhang et al., 2023e), specific state (Shaik et al., 2023a), or some intermediate results (Liu et al., 2021a).

Approaches to mitigation. Memory consumption reduction can be achieved by selectively storing historical information. For instance, in FedEraser (Liu et al., 2021b), the server stores clients’ gradients at specific intervals of FL rounds or based on the importance of gradients (Jiang et al., 2024). Similarly, in FRU (Yuan et al., 2023), only important updates to clients’ item embeddings are stored. In addition, by adopting coding-based techniques, the storage can be further compressed as demonstrated in (Lin et al., 2024a).

Unlearning in the FU system typically necessitates FL clients to transmit extra information, such as gradients, to the server to facilitate the process. For example, in FRAMU (Shaik et al., 2023a), clients additionally send attention scores to the server. In FedLU (Zhu et al., 2023), loss information is transferred for mutual knowledge distillation between the server and clients. In SFU (Li et al., 2023d), the remaining clients need to send their representation matrix to the server. In (Halimi et al., 2022), the remaining clients’ models are sent to the target client as references.

Approaches to mitigation. Reducing the communication cost can be achieved by minimizing the size of the model to be transferred, which may involve methods like quantization (Xiong et al., [n. d.]), and by reducing the number of unlearning rounds. For retraining-based unlearning, the FU system may roll back the global model to a state where it has not been significantly influenced by the target client. From this point, all FL clients can conduct the retraining process (Xiong et al., [n. d.]; Fan et al., 2022; Fraboni et al., 2022), thus reducing the unlearning rounds and enhancing communication efficiency. Besides, clustering is used to divide FL users into groups, each with its own model. The final inference is determined by a majority vote from these sub-models. This method confines unlearning processes to individual clusters, eliminating the need for participation from all users, thus improving efficiency (Su and Li, 2023; Qiu et al., 2023; Liu et al., 2024c; Wang et al., 2023a).

FU often requires clients to engage in additional computational tasks compared to standard FL. These tasks can involve generating dummy gradients (Liu et al., [n. d.]), conducting online reinforcement learning (Shaik et al., 2023a), seed model generation (Ye et al., 2023), or computing Hessian matrices (** et al., 2023; Cao et al., 2023; Li et al., 2023a; Liu et al., 2022c). It’s important to note that some of these computational tasks can be resource-intensive, which could pose challenges for deploying unlearning mechanisms in FL systems.

Approaches to mitigation. To enhance computational efficiency, approximation methods are commonly employed to accelerate certain components of FU algorithms. For example, the computation on the Hessian matrix can be approximated using techniques like a pre-trained deep neural network with Taylor expansion (** et al., 2023), the L-BFGS algorithm (Cao et al., 2023), or the Fisher information matrix (Li et al., 2023a; Liu et al., 2022c). Additionally, other optimizations are based on different FU structures. For instance, some works conduct training and unlearning simultaneously (Che et al., 2023), similar to the approach used in (Zhang et al., 2022b) in an MU setting. Orthogonally, computation tasks can be outsourced to a trusted third party, as demonstrated in (Li et al., 2023a). Another approach is transferring the majority of tasks to the server for estimation while executing only a small part of computation tasks for calibration (Cao et al., 2023), or reducing the computational tasks in the unlearning process by involving some pre-computation during the learning phase (Gu et al., 2024b). In (Dhasade et al., 2023), dataset distillation is adopted to compress the size of the dataset while preserving the unlearning performance, hence reducing the computational overhead.