The Implications of Decentralization in Blockchained Federated Learning: Evaluating the Impact of Model Staleness and Inconsistencies

Enabled by the advances in edge computing, the decentralization of artificial intelligence (AI) and machine learning (ML) unlocks a prominent paradigm where intelligence is brought closer to end-users. Decentralized AI allows real-time and near real-time applications to meet the ever-increasing latency requirements [1] thanks to the delay reduction achieved by eliminating the communication with a central node performing model inference. And not only that, but decentralized AI can potentially save energy by reducing the burden of massive data transferring, and also distributing the workload across many sites, rather than hosting it in a data center [2].

To enable edge intelligence and address scalability issues in ML, federated learning (FL) emerged in 2016 as a powerful tool to train ML models in a distributed manner [3]. In FL, a set of participants (also referred to as FL clients or FL devices) train an ML model collaboratively by exchanging model parameters, rather than by exchanging training data explicitly. Following this approach, an FL algorithm—see, e.g., Federated Averaging (FedAvg) [4]—can potentially reduce the overheads of ML training and also enhance the privacy of its centralized counterpart. FL was initially defined around the operation of a central server, which is responsible for orchestrating the ML training procedure by iteratively retrieving ML model updates from FL clients, computing global ML model updates, and distributing the outputs back to FL devices. Given the potential weaknesses of the centralized setting [5], including security, bottlenecking, or straggling issues, alternative decentralized architectures for FL have been recently proposed [6].

To enable the decentralization of FL, blockchain technology [7] stands as an appealing approach, as blockchains provide secure, immutable, and trustworthy decentralized storage. The decentralized realization of FL through blockchain, referred to as blockchained FL (sometimes it is also referred to as FLchain [8]), provides trust via cryptographic proof to federated ecosystems where multiple (often unreliable) parties cooperate to train a shared model. The blockchained FL framework does not only address centralization issues (e.g., single point of failure) but also provides complementary mechanisms that may boost FL settings, including effective ways of incentivizing FL participants (e.g., through native tokenization) to undertake ML model training [9].

Although blockchain enables security and trustworthiness in decentralized FL applications, its inherent decentralization entails some important implications that must not be disregarded. Figure 1 summarizes the blockchained FL operation by graphically showing the following main procedures (highlighted by the numbered circles in the figure):

1. 1.

   On-device computation at end devices: FL clients generate local ML model updates by training a model using their local data. The resulting local models are encapsulated in blockchain transactions.

2. 2.

   Exchange of transactions: The local model updates are submitted by FL devices to the blockchain. Blockchain nodes (e.g., miners) maintain a shared pool of transactions synchronized by exchanging and broadcasting transactions through peer-to-peer (P2P) messages.

3. 3.

   Blockchain writing: Blockchain miners take transactions from the shared pool to generate new blocks that update the status of the ledger. Each block corresponds to a newly computed aggregate model, built from FL client model updates.

4. 4.

   Block propagation and consensus: Blockchain nodes exchange blocks and enforce consensus rules to ensure that the ledger is consistent across the entire P2P network. The latest blocks are provided to FL devices to continue training the FL model iteratively.

Under ideal conditions in which information is instantaneously propagated without errors, the blockchain would keep track of the overall FL optimization process as in centralized FL, thus storing a global update in each block (equivalent to performing FL rounds). However, such ideal conditions are unfeasible and do not hold in reality. First, the delays associated with information propagation in the blockchain, together with the decentralized nature of consensus, make the ledger inconsistent in different parts of the blockchain network. Apart from that, FL devices have heterogeneous capabilities in terms of computation and communication, thus contributing to the inconsistent usage of models for training.

In this work, we focus our attention on the two following issues derived from the operation of blockchained FL:

• •

  Issue #1 - Ledger inconsistencies: A ledger inconsistency (e.g., a fork in the main chain) arises as a result of the concurrent and decentralized mining operation. The effects of ledger inconsistencies on the FL operation are exemplified in the right-top part of Fig. 1. In the provided example, Miner #4 generates a valid block at $t_{1}$, but before such a block is propagated at $t_{3}$, Miner #5 generates another valid block at $t_{2}$ (notice that $t_{3}>t_{2}$) with potentially different model updates. Until Miner #5’s chain is correctly updated (i.e., after Miner #4’s block is reaffirmed by another mined block), the forked block (shown in red) is used by Client #5 to generate a new local update at time $t_{4}$, thus leading to a model inconsistency.

• •

  Issue #2 - Model staleness: The delays for computing and securing local updates on the blockchain lead FL devices to potentially use outdated models for model training, i.e., models that are trained insufficiently compared to newer client updates but which are still to be processed by the blockchain. Model staleness may raise concerns in specific datasets and scenarios, provided that old model updates can negatively impact the global model’s accuracy. As an example of model staleness, in the right-bottom part of Fig. 1, Client #2 uses the model in Block #1 to generate a new local update at $t_{1}$. However, before the update from Client #2 is computed and secured on the blockchain (which entails computation, communication, and mining delays), newer blocks with fresher models are generated as a result of the asynchronous operation of other FL clients. Then, the question is whether the stale model update generated by Client #2 is still valid or not.

As described above, the blockchained FL setting leads to a set of issues that can potentially affect the learning procedure therein. To the best of our knowledge, these issues have been barely studied in the literature.

This paper is an extended version of our previous work presented in [10]. It builds upon the same concepts and methodology to provide further characterization and insights into the blockchained FL setting and its inherent issues (highlighted in Section I-B). The specific contributions of this paper are as follows:

• •

  We provide a comprehensive and self-contained overview of blockchained FL, including its technological realization and the description of the issues associated with it.

• •

  We characterize the main implications that arise from blockchained FL, i.e., ledger inconsistencies and model staleness. In this regard, we compute the freshness of blocks and use it as a staleness metric in FL.

• •

  We provide a simulation tool that is new of its kind for realistically capturing the blockchained FL operation. This tool, which is named BlockFLsim [11], integrates BlockSim [12] with Pytorch [13], thus allowing to simulate blockchained FL applications.

• •

  We evaluate the blockchained FL approach through extensive simulations and assess the impact of ledger inconsistencies and model staleness on FL accuracy. Our evaluation is carried out for automatic image recognition in different scenarios. More specifically, we consider two different sub-problems, characterized by the MNIST [14] and the CIFAR-10 [15] datasets for the evaluation of blockchained FL.

The remainder of this paper is structured as follows. Section II provides an overview of the state-of-the-art solutions regarding decentralized realizations of FL and the integration of blockchain and FL. Section III describes background concepts on blockchain and FL as independent solutions and then delves into the blockchained FL paradigm. The system model is presented in Section IV, which is followed by the performance evaluation using simulations in Section V. Section VI provides insights on future research directions and Section VII concludes the paper with final remarks.

The decentralization of ML procedures has been popularized in the past years as a result of the increased computational and storage capabilities of handheld devices and the increasing reluctance to share private data with a server. A proposal to distribute in a P2P fashion the well-known stochastic gradient descend (SGD) algorithm, normally used for training purposes, is presented in [16]. In [17], a fully decentralized mechanism was proposed to train ML models by leveraging one-hop communications between neighbor nodes. Similarly, gossip-based communications [18] were leveraged in [19] to train ML models in a P2P network. As the authors of [19] showed, besides improving robustness, gossip learning significantly reduces energy consumption while leading to the same performance as in centralized learning.

The idea of gossip learning has also been applied in decentralized FL. In particular, the work in [20] leveraged gossip communications to allow the exchange of model segments among workers, which were used to provide ML model updates in a decentralized manner. A decentralized version of FedAvg whereby clients communicate with their neighbors was also proposed and analyzed in  [21]. A similar approach was presented in [22], where decentralized FL was realized through device-to-device (D2D) communications. In addition, the authors in [22] defined protocols for both analog and digital types of transmission modes, which were evaluated through simulations. Another decentralized FL solution was proposed in [23] to accommodate the specific needs of unmanned aerial vehicles (UAV) networks. An alternative mechanism for decentralizing FL can be found in [24], which was inspired by the BitTorrent protocol. More specifically, through the mechanism in [24], FL clients can request model updates to other devices on-demand.

Between centralized and fully decentralized FL, other solutions relied on cloud-edge computing hierarchies [25, 26] or clustering capabilities [27] in order to improve the performance of FL. These classes of hybrid methods are useful to boost efficiency and mitigate the single point of failure and scalability issues of centralized FL, and at the same time prevent synchronization issues and model inconsistencies as in fully decentralized FL. For a more comprehensive overview of existing decentralized ML and FL solutions, we refer the interested reader to the surveys in [28, 29, 30].

Blockchained FL was first introduced in [31, 8] as a prominent solution for decentralizing FL and replacing the figure of the centralized orchestrating server with a blockchain. Different types of solutions have been envisioned to realize blockchained FL [32, 33]. Important design considerations lie in where and how to deploy the blockchain, which largely impacts the cost and performance of the solution [34].

A common approach in the literature relies on mobile edge computing (MEC) servers, co-located within access points (APs) or base stations (BSs) connected to end-users, to perform blockchain operations such as transaction validation or mining. This approach is very convenient for fulfilling typical blockchain requirements, as edge servers are typically equipped with high computation and communication capabilities. In [32], for instance, a generic blockchained FL architecture based on edge computing was defined. Through this approach, FL devices (in charge of performing model training) can submit ML model updates to the closest AP/BS, where an edge server acts as a full blockchain node (it collects transactions and participates in the block mining). Other works following a similar approach can be found in [8, 35, 36, 37], while alternatives including cloud/fog computing solutions were considered in [38, 23, 39, 40].

Apart from the system’s architecture, the election of the blockchain type is critical to the desired performance and capabilities and, therefore, should be influenced by the underlying FL application and the degree of trust among participants. In this regard, public blockchains using consensus mechanisms such as Proof of Work (PoW) have been adopted in [37, 23] to accommodate fully decentralized applications. Conversely, consortium and private blockchains using Algorand or Practical Byzantine Fault Tolerance (PBFT) have been considered in [36, 41] to fit more restricted settings where a limited group of participants maintain the blockchain.

When it comes to the evaluation of blockchained FL applications, analytical models like the one in [31] have been widely adopted to derive the end-to-end latency model of a blockchained FL system. Similarly, end-to-end latency models were proposed in [42, 43] to characterize the communication, computation, and consensus delays in blockchained FL.

Analytical models like the abovementioned ones, while providing a good intuition of the performance of blockchained FL applications, typically assume unrealistic conditions like perfect synchronization or proper FL client scheduling (see, e.g., [6]). In contrast, blockchained FL implementations suit better an asynchronous setting in which FL clients operate independently. The asynchronous blockchained FL problem has been studied in [44, 45, 46] but, still, the impact that blockchain decentralization has on the FL operation has been little studied to date. For instance, in the literature, forks have been considered to affect the transaction confirmation time only (see, e.g., [31]), so their implications on the FL training procedure remain unclear. In this paper, we aim to cover this gap in the literature and study the issues associated with model staleness and ledger inconsistencies that naturally arise in practical blockchained FL realizations.

Blockchain is a distributed ledger technology where a set of P2P nodes holding their own copy of the ledger must agree on the history of timestamped transactions. Blockchain combines multiple concepts and technologies, including cryptography primitives, to achieve relevant properties such as security, privacy, and immutability in a decentralized data-sharing framework. To showcase the main components and operations of blockchains, we resort to the blockchain protocol stack illustrated in Fig. 2 (other definitions have also been proposed in [47, 48]), which includes the following layers:

• •

  Application: The application layer deals with the decentralized applications (e.g., finance, supply chain, IoT [49]) running on the blockchain. This includes all the infrastructure and tools for generating and handling the decentralized application data (e.g., transactions, smart contracts) hosted in the blockchain and may include the support from user graphical interfaces, APIs, wallets, etc.

• •

  Data: The data layer is related to the way data is structured and stored in the blockchain. This layer applies concepts such as the chaining of blocks or digital signatures to enforce privacy, security, immutability, and transparency properties.

• •

  Network: The network layer allows blockchain nodes communicate to exchange transactions and blocks, and also to enforce consensus. In this layer, procedures like node discovery or block/transaction exchanges are defined.

• •

  Consensus: The consensus layer establishes the rules to be applied by blockchain nodes in order to participate in the maintenance and update of the distributed ledger. Consensus mechanisms such as PoW establish the operations of blockchain miners for creating and accepting new blocks, thus being the core of the decentralized and concurrent synchronization of the ledger.

• •

  Infrastructure: Finally, the infrastructure layer defines the set of physical devices, connections, and operations within the underlying blockchain network. Depending on the type of consensus adopted, varying computational and storage capabilities might be required.

The data and the consensus layers constitute the core of blockchain technology, as they define the way information is structured and validated by participants, which is the key to providing security and immutability. As shown in Fig. 3, transactions (i.e., events updating the ledger) are grouped in blocks, each chained after the previous one and starting by the genesis block (with depth 0). Depending on the blockchain realization (e.g., Bitcoin, Ethereum, Hyperledger Fabric), blocks may carry different types of information, but in general, blocks include the following basic fields:

• •

  Nonce (number only used once): The number used to demonstrate a cryptographic proof within the PoW mining operation. In Bitcoin, a nonce is a 32-bit number that, when hashed with the rest of the headers of a block, meets the difficulty imposed by the consensus protocol (i.e., the solution of such a hash contains a predefined amount of initial zeros).

• •

  Hash: Output of the hash function (e.g., SHA-256) when combining the headers of the block. The hash is used as cryptographic proof to authorize a given miner to add a new block to the ledger.

• •

  Timestamp (TS): The time the block was created.

• •

  Merkle root (MR): Typically, for building the hash of a block, transactions are organized in a Merkle tree to further guarantee immutability. The root of the Merkle tree is included in the header.

• •

  Body of transactions: Set of transactions generated by the application layer and which are included in the block by the miner.

Blockchain consensus plays a major role in preserving the integrity of the data stored in a blockchain and it is required for blockchain participants to adopt the same history of the ledger. Particularly in permissionless blockchains,¹¹1In permissionless blockchains, different from permissioned ones, any party can participate in the consensus procedure. where miners work concurrently, mining can lead to forks, i.e., different (forked) versions of the ledger adopted by different blockchain nodes. In PoW, which has been widely used in public blockchains (e.g., Bitcoin, Ethereum), forks occur when two or more miners come up with a valid nonce before the winning miner shares its block with the rest of the miners. Forks are solved by enforcing consensus rules, such as adopting the longest chain (with more invested power) as the valid one. Following the consensus rules, a blockchain node in a forked chain eventually switches to the main chain, which occurs when the main chain obtains more confirmations than any other forked version.²²2In Bitcoin, for instance, up to six confirmations are required before a transaction (a payment) is considered to be secure.

In FL [50], a set of clients collaborate to train a global model in a distributed manner. To that purpose, instead of sharing raw data directly (as done in traditional centralized ML applications), FL devices sequentially exchange model parameters, obtained by training the latest received global model on local data. Broadly speaking, the goal of FL is to minimize a global finite-sum cost function, weighted by the contribution of each node (e.g., in terms of data samples with respect to the total length of the distributed dataset).

FedAvg (illustrated in Fig. 4 and described in Algorithm 1) is one of the most popular algorithms to carry out the FL operation [4]. In FedAvg, the parameter server (or central orchestrating server) selects a subset $S_{t}\subseteq\mathcal{K}$ of FL clients in each round $t$. The selected clients compute a local update $w_{t}^{(k)}$ from the current global model $w_{t}$ by running SGD. Then, the server aggregates the received local models from the selected clients to generate a new global model update $w_{t+1}$, which is sent to the next set of selected clients for further training.

The blockchained FL solution relies on a distributed ledger to securely store ML model updates from FL clients. This solution gets rid of the central orchestrating server and, instead, empowers a decentralized network of miners to maintain the status of the FL global model by updating the ledger based on FL clients’ updates. The ledger can be accessed asynchronously by the FL clients for either reading (e.g., downloading the latest global models) or writing (e.g., submitting fresh local model updates). Under such an asynchronous and uncoordinated setting where each FL client works independently and maintains its own FL model history, the concepts of FL training round and global model lose meaning.

To carry out decentralized FL through blockchain, we identify two types of logical entities, namely blockchain nodes and FL clients, each one with specific functionalities and logical components (illustrated in Fig. 5). Blockchain nodes are responsible for gathering, sharing, and verifying transactions and maintaining the distributed ledger, while FL clients perform a given federated optimization task by contributing with locally trained models. The set of functional blocks from blockchain nodes and FL clients is as follows:

• •

  Distributed data module: Used to store and keep track of the history of blocks and to maintain the pool of unconfirmed transactions.

• •

  Consensus engine: Deals with the enforcement of the distributed protocol through mining operations (e.g., PoW, PBFT), validation, and conflict resolution.

• •

  Communication module: It allows blockchain nodes to share either transactions or blocks among them. The communication module also provides an interface with FL clients for gathering transactions (local models) and exposing the ledger.

• •

  Blockchain adaptor: It allows clients to interact with the blockchain to either submitting transactions or retrieving the latest FL models. An incentive engine can be used to encourage the participation of FL clients in the distributed learning operation.

• •

  Federated learning engine: Allows training a model using on-device ML libraries (e.g., TensorFlow) and local data (it can be an offline process). In this paper, model aggregation is considered to be done by miners. Nevertheless, the envisioned architecture also allows performing model aggregation on the FL device side, as previously proposed in [31, 32].

The specific implementation of blockchain nodes and FL clients depends on the underlying FL application and the requirements therein. Based on the type of blockchain adopted (e.g., public vs. private, permissioned vs. permissionless blockchains), blockchain nodes and miners might need different hardware capabilities, thus leading to either specialized or general-purpose devices meeting certain storage and computation requirements. In public permissionless blockchains, blockchain nodes must possess high computational and storage capabilities to support computation-intensive mining and store a large history of transactions, respectively. In Bitcoin, for instance, blockchain nodes are typically specialized devices (e.g., ASIC miners) with tens to hundreds of terahash per second (TH/s) power and hundreds of GBs of memory [51]. When it comes to FL clients, they need enough storage to keep their local dataset (typically in the order of a few GB, depending on the application) and from low to moderate computational power to perform local training. FL clients are typically end devices like smartphones or laptops, but other solutions leveraging MEC-based computation offloading exist [26].

A set of $\mathcal{K}=\{1,2,...,K\}$ clients collaborate to train a global model $w$ by sharing locally trained model parameters $w^{(k)}\in\mathbb{R}^{d}$ through a blockchain. Starting from a global model provided by block $b$, $w^{(b)}$, each device $k$ attempts to minimize a local loss function $l^{(k)}(\cdot)$ by running $E$ epochs of SGD on its local data. Using a dataset $\mathcal{D}^{(k)}$ with $D^{(k)}=|\mathcal{D}^{(k)}|$ samples, a client $k$ updates the local model parameters $w^{(k)}$ as:

where $\eta$ is the learning rate and $\nabla l^{(k)}(w^{(b)},\mathcal{D}^{(k)})$ is the $k$-th client average loss gradient with respect to $w^{(b)}$. A client $k$ counts on $\rho^{(k)}$ computational power for training a local model.

The local model updates are aggregated and the resulting global model is included into a blockchain block. The aggregation of local updates, assumed to be done by the miner generating the block $b$, is computed as

where $D^{(b)}=\sum_{k\in b}D^{(k)}$. Following this approach, any client can retrieve the latest block $b$ from its closest miner $m$ and use the latest global model to continue training the federated model.

We consider a P2P network of $\mathcal{M}$ blockchain nodes acting as miners. The blockchain nodes are responsible for gathering transactions from FL devices, mining blocks, and broadcasting new transactions and blocks. The overall process, as described in the literature [31, 52], can be decomposed into different steps, including transaction/block propagation, block mining, and consensus resolution, which delay characterizations are described in the following sections.

To evaluate the performance of different blockchained FL realizations, we focus on the blockchain throughput and the model accuracy and staleness, defined in the following subsections.

First, we study the performance of different blockchain settings in Fig. 6, which shows the throughput in TPS achieved for different block interval ($BI$) values and numbers of transactions used per block ($\text{N}_{\text{t}}$). To showcase the performance for the most challenging task considered, Fig. 6 focuses on CIFAR-10, where a heavier CNN model is used.

As shown, the blockchain throughput decreases as the block interval $BI$ increases (e.g., from 7.429 TPS to 0.163 for $BI=1$ s and $BI=60$ s, respectively, when $\text{N}_{\text{t}}=10$), which is mostly motivated by the bottleneck created by block mining. Both $BI$ and $\text{N}_{\text{t}}$ are blockchain configuration parameters that allow for representing different types of blockchains (e.g., public, consortium, private). The block interval, for instance, is a fundamental aspect of the design of blockchains, provided that it determines the speed at which the blockchain validates and secures transactions.³³3In PoW mining, miners invest their computational power to prove the validity of the blocks they mine in a decentralized setting. Accordingly, large block intervals are derived from the necessity of enforcing a high mining difficulty in networks with a big computational power. Typically, the block interval is selected according to the characteristics of the targeted scenario (e.g., type of participation access) and the security requirements of the supported decentralized application. Whereas there are multiple aspects involved in the establishment of security and trust in a blockchain, the block interval encapsulates well the characterization of different types of blockchain. In particular, low block interval values such as $BI=1$ s are feasible in blockchains where the miners are trusted (e.g., a private blockchain controlled by a consortium of network operators), while higher values like $BI=60$ s are required in blockchains where miners are trustless (e.g., a public blockchain where any interested party can participate). As for the set of included transactions used to generate each block, we observe that higher $\text{N}_{\text{t}}$ lead to higher throughput (e.g., from 0.813 to 7.429 TPS for $\text{N}_{\text{t}}=1$ and $\text{N}_{\text{t}}=10$, respectively when $BI=1$ s), as blocks can carry more information. However, processing and validating a large volume of user transactions requires enough computational power from blockchain nodes. The blockchain throughput, as shown in more detail in the sequel, has an impact on the underlying application’s performance, so it is an important metric to be optimized.

We now focus on the performance of the FL models in each blockchained FL setting. First, Fig. 7 shows the temporal evolution of both the training (Fig. 6(a) and Fig. 6(c)) and the validation accuracy (Fig. 6(b) and Fig. 6(d)) achieved by the models carried in the blocks of the main chain, for the two different datasets. The evaluation also includes the comparison of two types of P2P links, namely $\text{C}_{\text{link}}=\{1,100\}$ Mbps. The maximum number of transactions per block, $\text{N}_{\text{t}}$, is set to 10.

Starting with MNIST, we observe in Fig. 6(a) and Fig. 6(b) that the performance is very similar for both training and validation accuracy, and this is due to the simplicity of the data employed (at which up to 99,7% performance can be achieved [53]). For CIFAR-10 (see Fig. 6(c) and Fig. 6(d)), in contrast, the discrepancies between training and validation accuracy are much more noticeable, being the training accuracy superior (up to 100% accuracy) to the validation one (up to 75%).

Regarding the different evaluation parameters, we find the following. First, the lower the block interval ($BI$), the worse the accuracy, especially in CIFAR-10, for which the model’s complexity is higher. Setting $BI=1$ s, while allowing to generate blocks very fast, leads to a low training accuracy (in CIFAR-10, up to 50% training/validation accuracy at the end of the training). As studied later with further detail, a short block generation time leads to a high number of forks, which is detrimental to the learning procedure. Increasing the block interval ($BI=10$ s and $BI=60$ s), instead, contributes to improving the accuracy significantly, as newly generated models are properly spread throughout the network, thus leading to a consistent federated training operation. Increasing the block interval, therefore, is one way of enforcing stability to FL training, but it leads to a higher training time.

When it comes to the number of FL participants ($K$), we observe that a higher performance is achieved as $K$ decreases in all the cases. This is directly related to the amount of training data available to FL participants, provided that the entire dataset is split among all the considered participants. Apart from that, we observe that the gap in the accuracy achieved for different values of $K$ becomes bigger as $BI$ increases, as setting a higher $BI$ value allows for collecting more transactions in each block. In settings where $BI$ is small ($BI=1$ s), the probability of collecting enough transactions per block is low if the number of FL participants $K$ is low ($K=10$). In contrast, deployments with more FL clients in place ($K=100$) allow gathering plenty of local models per block, as there are more participants operating concurrently. However, the quality of the provided model updates for $K=100$ is lower than for $K=10$, as data is split among a higher number of users. Therefore, there is a trade-off between the number of local models per block and the quality of the same, and such a trade-off becomes apparent when the block interval is not properly dimensioned according to the underlying FL application.

Finally, regarding the type of links used to transmit transactions and blocks, we observe different behaviors depending on the blockchain setting. For low block interval values, i.e., $BI=\{1,10\}$ s, there is a big gap between the accuracy achieved by $\text{C}_{\text{link}}=100$ Mbps (shown by the solid lines) and $\text{C}_{\text{link}}=1$ Mbps (shown by the light circles). This gap is due in large part to the need for propagating the updated models throughout the blockchain network consistently, thus enabling a robust FL training operation. Otherwise, ledger inconsistencies lead to conflicting and redundant efforts in iterating the global model, thus negatively affecting the overall model accuracy. The gap between the performance achieved by each $\text{C}_{\text{link}}$ value, however, is mitigated as $BI$ increases, thanks to the model consistency achieved in those cases.

As shown before, the type of blockchain adopted and its characteristics have a significant impact on the performance of the ML model, being the block interval ($BI$) and the P2P links capacity ($\text{C}_{\text{link}}$) some of the most critical parameters in that regard. Now, to further illustrate the impact of the blockchain characteristics on the federated models’ accuracy, in Fig. 8, we show the difference between the test accuracy achieved for $\text{C}_{\text{link}}=1$ Mbps (leading to a higher fork probability) and $\text{C}_{\text{link}}=100$ Mbps (granting more stability), accompanied by the fork probability in each case. The fork probability is a good indicator of the model inconsistencies, as it gives a good intuition of how many blockchain versions (i.e., global models) are concurrently used by different FL clients.

The results in Fig. 8 confirm the observations done above for Fig. 7, as the highest differences in the test accuracy are obtained for low block interval values ($BI=\{1,10\}$ s) and high numbers of FL participants ($K={50,100\\ }$ clients). In particular, a high fork probability ($\text{P}_{\text{fork}}\approx 0.8$ for the worst case in CIFAR-10) can lead to a decrease of up to 30% on the model’s accuracy when users’ diversity is high, i.e., for $K=100$ clients. In these situations, increasing the pace at which information is processed within the blockchain by increasing $BI$ is helpful to minimize the fork probability, thus mitigating the effects of model inconsistencies. For instance, for MNIST, $BI=10$ s provides a good trade-off between the block confirmation time and the fork probability. For CIFAR-10, which uses heavier modes, setting $BI=60$ s is required to keep the test accuracy difference below 5%.

Next, we focus on model staleness and its impact on ML model accuracy. To that end, Fig. 9 illustrates the temporal evolution of the model staleness and the validation accuracy experienced throughout the 50 first blocks of the main chain in each simulation. For the sake of reducing the impact of forks and focusing on model staleness, the results are shown only for $\text{C}_{\text{link}}=100$ Mbps, where the fork probability is kept low in all the scenarios. Apart from that, the maximum number of local updates aggregated in a block, $\text{N}_{\text{t}}$, is fixed to 10. In the subplots on the top of Fig. 9, both model staleness and validation accuracy are displayed together in a 3D grid, while the corresponding 2D projection of the validation accuracy is plotted at the bottom of each figure for the sake of clarity.

As shown in both Fig. 8(a) and Fig. 8(b), model staleness remains stable for $BI=1$ s, which indicates that the blockchain is able to process the model updates submitted by clients in time. In contrast, model staleness increases very fast for each block when $BI=60$ s, as more and more local model updates remain unprocessed at the unconfirmed pool of transactions, thus becoming outdated with respect to newer updates (the blockchain cannot process all the local model updates). For $BI=1$ s, the achieved accuracy is significantly lower than in the other cases because FL devices are not fast enough to compute and provide local updates in time (as previously shown in Fig. 7), thus leading to under-trained models at each block depth. As for $BI=10$ s and $BI=60$ s, both solutions provide very similar accuracy, even if model staleness is much higher in the second case due to the slow pace at which the blockchain processes users’ transactions (this is the opposite of what occurs with $BI=1$ s). This result suggests that model staleness is not detrimental to the model performance at all and, moreover, old local updates (i.e., updates computed from old global models) indeed contribute to sustaining the training of the federated models. Nevertheless, this conclusion is tied to the dataset used in this paper (CIFAR-10) and the distribution of the data over time, which in this case remains unchanged. However, other applications where data vary over time (which is associated with concept drift [54]) could be severely affected by staleness.

The analysis performed in Section V has demonstrated that the type of blockchain selected and its configuration have a high impact on the performance of the application running over it. In particular, model staleness and model inconsistencies, which are motivated by the mismatch between the application requirements and the blockchain performance, may result in significant performance degradation of FL if they are not kept under control. For that reason, blockchain optimization becomes particularly relevant for the FL use case. Whereas techniques like sharding or off-chain computation have been widely applied to improve current blockchains, other works have focused on optimizing blockchain configurations to better comply with the application requirements. In this regard, [31] focused on the block generation rate to minimize the end-to-end latency of blockchained FL applications, while [52] targeted the optimization of the block size as a function of the users’ activity.

Another important aspect for optimizing blockchained FL applications relates to the exchange of raw ML models between FL devices and blockchain miners. As shown in [2], this approach entails significant communication and storage overheads, especially for complex ML models like VGG-16, whose size is over 500 MB [55]. To alleviate the communication burden, other distributed learning approaches such as Knowledge Distillation (KD) [56], much lighter than FL in terms of communication, might be compelling for its integration with blockchain.

The performance of blockchained FL applications can be affected by the quality and legitimacy of the contributions from the FL devices. To address these issues, we first find in FL client selection a prominent solution to speed up the performance of FL, thus relieving the burden on the blockchain. With FL client selection, user heterogeneity (e.g., in terms of computation and/or communication capabilities) can be addressed by selecting the best-performing nodes [57]. Apart from FL client selection, blockchain inherent properties can be leveraged and extended to provide enhanced security and trust. Some prominent examples are enhanced authentication mechanisms [58] and trust evaluation and enforcement [59]. Finally, there is the opportunity to adopt mechanisms in the blockchain for selecting client model updates to be mined, based on their degree of staleness (is the model update fresh enough?).