SemantIC: Semantic Interference Cancellation
Towards 6G Wireless Communications

Abstract

This letter proposes a novel anti-interference technique, semantic interference cancellation (SemantIC), for enhancing information quality towards the sixth-generation (6G) wireless networks. SemantIC only requires the receiver to concatenate the channel decoder with a semantic auto-encoder. This constructs a turbo loop which iteratively and alternately eliminates noise in the signal domain and the semantic domain. From the viewpoint of network information theory, the neural network of the semantic auto-encoder stores side information by training, and provides side information in iterative decoding, as an implementation of the Wyner-Ziv theorem. Simulation results verify the performance improvement by SemantIC without extra channel resource cost.

I Introduction

How to implement the Wyner-Ziv theorem [1] in practical coding design? Academia and industry have struggled with this question for several decades, although the Wyner-Ziv theorem provides a prospect to enhance the information quality by providing side information at the decoder only. The challenge of practical implementation is to find a method to manage side information.

Thanks to recent development of semantic communications [2], which is a promising technological basis for the oncoming sixth-generation (6G) wireless networks [3], we can obtain the solution to representing side information by semantic neural networks. In semantic communications, the semantic encoder extracts and transmits the semantic information, i.e., the feature of the original information. Then, the receiver generates and reconstructs the original information from the semantic information by the semantic decoder [4]. By this means, the payload transmitted through the channel can be significantly reduced. From the viewpoint of network information theory, the semantic encoder/decoder pair learns and shares the same side information by joint training. Hence, when separately deploying the semantic encoder and decoder, the transmitter can compress the source information according to side information, and the decoder can recover the source information with the assistance of side information.

It is noticed that the function of semantic decoder can be regarded as generative artificial intelligence (AI) [5, 6]. The diffusion model [7] is a generative AI method effective for achieving super-resolution, which can be also regarded as a process of eliminating noise. Particularly, in the reverse diffusion process, noise is predicted by neural networks and subtracted from the image. Therefore, generative AI implicitly contains the capability for eliminating noise. Similarly, if we train the semantic encoder and decoder by adding noise between them, the semantic encoder and decoder will also work better in denoising.

Figure 1: The system model from the viewpoint of network information theory. $X^{n}$, $Y^{n}$ and $\hat{X}^{n}$ stand for the source information, the side information and the reconstructed information, respectively, with the sequence length of $n$. $M$ is the codeword satisfying the coding rate $R$.

Inspired by the Wyner-Ziv theorem and semantic communications, this letter proposes a breakthrough technique, namely semantic interference cancellation (SemantIC), to improve the information quality by updating the receiver structure only. The theoretical model of SemantIC is exactly the Wyner-Ziv model as illustrated in Fig. 1, where the side information (in practice, the knowledge of source) is learned by neural networks and stored in the form of the neural network parameters at the decoder. The proposed SemantIC technique naturally exploits the semantic information contained in the data, and hence it well matches the feature of 6G which focuses more on the semantic information of data.

In practical implementation, the SemantIC receiver concatenates a pair of semantic encoder/decoder with the channel decoder. The semantic encoder extracts the semantic information of the channel decoder output. In this way, the noise and interference can be eliminated to a certain extent when extracting information features. From the viewpoint of machine learning, the concatenated semantic encoder/decoder pair constructs an auto-encoder [8, 9], which has also been verified to be able to eliminate noise [10]. Subsequently, based on the denoised information features, the semantic decoder reconstructs the original information to input into the channel decoder as the a priori information for the next round of iterative decoding.

Note that the channel decoder concatenated with the semantic auto-encoder composes a turbo loop. Owing to the turbo principle [11], the decoding gain is alternately enhanced by the channel decoder and the semantic auto-encoder. Furthermore, since the side information is introduced into the turbo loop by the semantic auto-encoder, the final decoding output contains more information and hence achieves a higher quality. The contributions of this letter is summarized as follows.

• We propose a novel anti-interference technique, i.e., SemantIC, which can improve the recovered information quality without extra cost of channel resources.

• We design an iterative decoding algorithm for image transmissions with SemantIC by constructing a turbo loop concatenating the channel decoder and the semantic auto-encoder, which alternately eliminates noise in the signal domain and the semantic domain.

• Simulation results show that the system with SemantIC has a better performance in terms of bit error rate (BER), Euclidean distance (ED), and peak signal-to-noise ratio (PSNR), even with a very light neural network (42.1 kB).

II Semantic Interference Cancellation Principle

II-A Theoretical Basis

The theoretical basis of SemantIC is the Wyner-Ziv theorem. As illustrated in Fig. 1, the Wyner-Ziv system is required to transmit a source sequence $X^{n}$ under the rate constraint $R$. Therefore, an encoder compresses the source sequence $X^{n}$ into a codeword $M$ before transmission. Then, at the decoder, the noncausal side information sequence $Y^{n}$ is available to help better reconstruct the source sequence. The recovery $\hat{X}^{n}$ may be lossy if both the channel capacity and the side information is insufficient. Mathematically, the link rate $R$ should satisfy the rate-distortion function given a distortion requirement $D$, well known as the Wyner-Ziv theorem:

$$R(D)\geq I(X;U|Y),$$ (1)

where $U$ represents the compress information of $X$.

In practical systems, the mutual information gain provided by the knowledge base can be mathematically evaluated as

$$I_{gain}=I(X;\hat{X}_{S})-I(X;\hat{X}_{C}),$$ (2)

where $\hat{X}_{S}$ is the information recovered with the help of the semantic auto-encoder, and $\hat{X}_{C}$ is the information independently recovered by the channel decoder only. Both $I(X;\hat{X}_{S})$ and $I(X;\hat{X}_{C})$ can be calculated from the mutual information between the transmitted and the recovered information.

Although Wyner and Ziv have presented a coding scheme in the proof of the rate-distortion function in (1), their proposed algorithm requires the sequence length tends to $\infty$. Moreover, since their coding scheme utilizes a random codebook, it requires a huge memory to store the codebook, and the encoding/decoding process is of very high time-complexity due to exhaustive search over the random codebook.

To implement the Wyner-Ziv theorem in practical systems, we have to solve many tricky problems including how to obtain, store and represent side information, as well as how to exploit side information in decoding without using a random codebook. We find that neural networks can automatically learn the knowledge about the source by training, which is essentially the same as obtaining, storing and representing side information. It is also noticed that the side information stored in the form of neural network parameters is noncausal, which satisfies the condition of the Wyner-Ziv theorem. Now, we need to find the last piece of the puzzle, i.e., decoding with the side information stored in neural networks.

II-B Decoder Structure

Figure 2: The structure of SemantIC system.

To provide side information by neural networks for decoding, we need to design the decoder structure. Therefore, we propose the structure of the SemantIC system in Fig. 2, where the transmitter has a general structure without constraint. At the receiver, the channel decoder $\mathcal{F}_{C}(\cdot)$ processes the a priori log-likelihood ratio (LLR) to output the a posteriori LLR as

$$\mathrm{LLR}^{p}=\mathcal{F}_{C}(\mathrm{LLR}^{a}).$$ (3)

The SemantIC decoder contains two key elements as follows.

II-B1 Auto-Encoder

The auto-encoder is a neural network having the function to extract information features and then recover information from features. By this method, the semantic noise is filtered by the neural network. In the SemantIC system, the receiver utilizes a semantic auto-encoder, consists of a pair of semantic encoder/decoder, to eliminate the interference and noise contained in the channel decoder output. This process is referred to as denoising in the semantic domain.

II-B2 Turbo Loop

To enhance the decoding gain, the structure of iterative decoding is necessary. Therefore, we connect the output of the semantic auto-encoder $\mathcal{F}_{S}(\cdot)$ back to the channel decoder $\mathcal{F}_{C}(\cdot)$ as the a priori information.

$$\mathrm{LLR}^{a}=\mathcal{F}_{S}(\mathrm{LLR}^{p}).$$ (4)

The structure of (3) and (4) composes a turbo loop for eliminating noise and interference in the signal domain and the semantic domain alternately. The semantic auto-encoder performs semantic denoising from the input $\mathrm{LLR}^{p}$, and then output $\mathrm{LLR}^{a}$ to the channel decoder for denoising of signals in the following iteration round of decoding.

Based on the semantic auto-encoder and the turbo loop, the SemantIC system can further improve the reconstructed information quality without any change outside the receiver. The turbo loop reduces noise at the cost of introducing extra latency in each round of iterations by the semantic auto-encoder depending on the complexity of the neural networks.

III Decoder Implementation Design

In order to evaluate the performance of the SemantIC system, we design a decoding algorithm for image transmission simulations. Since the semantic encoder/decoder requires a dataset for training, we select the CIFAR-10 dataset [12] for performance evaluation. The CIFAR-10 dataset comprises a collection of 60,000 pictures, consisting of 50,000 training images and an additional set of 10,000 images specifically designed for testing purposes. The image size is scaled to $3\times 96\times 96$, i.e., 3 color channels, and 96 pixels in both width and height.

Figure 3: A simple structure of semantic encoder/decoder.

As depicted in Fig. 3, we concatenate two-dimensional (2D) convolutional layers (Conv2Ds) as the semantic encoder, and 2D transpose convolutional layers (TranConv2Ds) are concatenated to be the semantic decoder. Let $\{n_{F},(w_{K},h_{K}),s\}$ specify the parameters of Conv2D and TranConv2D, with $n_{F}$ denoting the number of filters, $(w_{K},h_{K})$ denoting the kernel size, and $s$ denoting the stride. The parameters are configured to gradually extract the semantic features with a rate approximate to $0.3$, i.e., the output size is $16\times 23\times 23$. As will be shown in the simulation results, even with such simple semantic encoder/decoder structure, the SemantIC decoder is effective with respect to improving information quality.

Our model is implemented using Pytorch, with training conducted on the NVIDIA GeForce RTX 3060 Ti by the Adam optimizer [13]. Compared to the 163 MB size of the CIFAR-10 dataset, the file for storing the semantic auto-encoder parameter only has a size of 42.1 kB.

Low-density parity-check (LDPC) codes [14] are selected as the channel code, with the codeword length being 900. The number of parity-check equations including a certain bit is set at 2, and the number of bits in the same parity-check equation is set at 3. The maximum iteration round is set at 7. The information of each image is divide into groups to match the LDPC codeword length for transmission. Since the LDPC codes work for binary sequences, the image information in the form of pixels is quantized into binary sequences before inputting into the channel decoder. After channel decoding, the output should also be dequantized back to pixels.

An example source code for the implementation of the proposed SemantIC system is presented in [15].

(a) BER.

(b) ED and PSNR.

Figure 4: The impact of SNR on BER, ED, and PSNR.

IV Simulation Results

In this section, we conduct a comparative analysis between the system with and without SemantIC by simulations. We employ the BER, the ED, and the PSNR as evaluation metrics to quantify the distortion between the original images $\boldsymbol{X}$ and the reconstructed images $\boldsymbol{\hat{X}}$. Additionally, we set the learning rate to 0.003, the batch size to 64, and the maximum training epoch to 200.

IV-1 Anti-Interference Performance

As shown in Fig. 4, we demonstrate the relationship between the BER, ED, PSNR and SNR of the proposed method in an additive white Gaussian noise (AWGN) channel. It is easy to observe from the figure that the SemantIC system outperforms the system without SemantIC in terms of lower BER and ED as well as higher PSNR at different SNR values. It is observed that the BER performance gain seems less visible than ED. This is because the loss function for training the semantic auto-encoder is designed based on ED instead of BER. Although the BER performance seems to be very small, this performance gain is obtained without extra channel resource cost, and the SemantIC neural network has a very simple structure and with a very small size of parameters. The system is expected to have a better anti-interference performance, if the neural network is sophisticatedly designed, e.g., introducing the attention mechanism to enlarge receptive field, and/or utilizing the diffusion model to predict the noise.

Particularly, SemantIC performs better in low SNR scenarios. This can be attributed to the fact that higher-level semantic features are more resilient to channel noise and fluctuations compared to lower-level semantic features. By extracting high-level semantic features through the semantic encoder/decoder, SemantIC aids the channel decoder in recovering lost semantic information during the physical channel transmission process, thereby enhancing system robustness. Another factor for lower gain by SemantIC is that LDPC codes have a strong capability for enhancing decoding gain. Hence, when SNR is large, LDPC codes have already significantly eliminated noise and interference. Especially when SNR$\geq 8$ dB, the BER by channel decoding only has reduced to $0$ at the first round of iteration. Therefore, no gain can be obtained for SNR$\geq 8$ dB.

(a) BER.

(b) ED.

(c) PSNR.

Figure 5: Convergence behaviour for BER, ED, and PSNR.

IV-2 Iterative Gain

Fig. 5 demonstrates the performance gain achieved through iterative decoding without extra channel resource cost. It is evident that as the number of iterations increases, SemantIC consistently outperforms the method without SemantIC, and the performance gap gradually becomes significant. This is because during the iterative decoding process, the semantic auto-encoder can filter out noise and interference to a certain extent and provide additional semantic information through multiple interactions. This observation verifies the effectiveness of the turbo loop in the SemantIC decoder. Consequently, it effectively reduces error propagation and enhances overall decoding performance. Moreover, the performance converges fast at the fourth round of iteration. This is due to the strong error correcting capability of LDPC codes. Hence, practical systems can adopt early stop** strategy to balance computational complexity with latency. Interestingly, as SNR becomes larger, the BER performance gain increases, while the performance gain of the ED and PSNR decreases.

(a) Lossless images.

(b) SNR$=-5$ dB.

(c) SNR$=0$ dB.

(d) SNR$=5$ dB.

Figure 6: Comparisons of the reconstructed image quality.

IV-3 Visual Effect

For the comparison of visual effect, we utilize the original image presented in Fig. 6(a) as an example, and the semantic image is obtained by inputting the original image to the semantic encoder concatenated with the semantic decoder. Fig. 6 visualizes the results of image reconstruction with and without SemantIC, subjectively verifying the fidelity of semantic information. In Fig. 6, Iter, IC/noIC, Sem, ED represent the iteration rounds, the image with/without interference cancellation, the image reconstructed by the semantic decoder, and ED.

In Fig. 6(b), the method without SemantIC fails to clearly depict the shape of the car window, whereas our approach successfully recovers some details of the window from the blurred original image. In Fig. 6(c), SemantIC enhances both the visual quality and semantic information of the image. Traditional algorithms often focus solely on visual quality, neglecting the preservation of semantic information. However, SemantIC simultaneously considers both image quality and semantic information, rendering objects and scenes in the image more realistic, recognizable, and comprehensible. If we zoom-in and compare the image of IC/noIC with Iter$=7$, we can see that hot pixels within the yellow circle is less for the IC case. The semantic image also gradually becomes clearer as the iteration round increases. In Fig. 6(d), as channel conditions improve, the reconstructed image becomes highly realistic, making it difficult to discern from the original image. Although the image quality is already very high without SemantIC, our approach can still further eliminate some noise. For example, if we zoom-in the image with Iter$=7$, we can see that one hot pixel pointed by the yellow arrow is corrected.

V Conclusion

This letter has proposed a breakthrough technique, referred to as SemantIC, for anti-interference towards the 6G wireless communications. The SemantIC decoder contains a turbo loop consists of concatenated channel decoder and semantic auto-encoder, which eliminate the noise and interference alternately in signal domain and semantic domain. We have also designed a decoding algorithm and implemented the SemantIC system for image transmissions. The simulation results indicate that the SemantIC system can enhance the reconstructed information quality without extra cost of channel resources, although we only utilize a very simple neural network structure. Even with a very low SNR, the image recovered by SemantIC shows a clearer object shape than that without SemantIC.