WIA-LD2ND: Wavelet-based Image Alignment for Self-supervised Low-Dose CT Denoising

Abstract

In clinical examinations and diagnoses, low-dose computed tomography (LDCT) is crucial for minimizing health risks compared with normal-dose computed tomography (NDCT). However, reducing the radiation dose compromises the signal-to-noise ratio, leading to degraded quality of CT images. To address this, we analyze LDCT denoising task based on experimental results from the frequency perspective, and then introduce a novel self-supervised CT image denoising method called WIA-LD2ND, only using NDCT data. The proposed WIA-LD2ND comprises two modules: Wavelet-based Image Alignment (WIA) and Frequency-Aware Multi-scale Loss (FAM). First, WIA is introduced to align NDCT with LDCT by mainly adding noise to the high-frequency components, which is the main difference between LDCT and NDCT. Second, to better capture high-frequency components and detailed information, Frequency-Aware Multi-scale Loss (FAM) is proposed by effectively utilizing multi-scale feature space. Extensive experiments on two public LDCT denoising datasets demonstrate that our WIA-LD2ND, only uses NDCT, outperforms existing several state-of-the-art weakly-supervised and self-supervised methods. Source code is available at https://github.com/zhaohaoyu376/WI-LD2ND.

1 Introduction

Computed tomography (CT) has become a widely utilized tool in medical diagnosis. However, increased usage has raised concerns regarding potential risks associated with excessive radiation exposure [11]. The widely recognized principle of ALARA (as low as reasonably achievable) [25] is extensively embraced to minimize exposure through strategies such as employing sparse sampling and reducing tube flux. Reducing the X-ray radiation dose, however, leads to poor-quality images with noticeable noise, which poses challenges for accurate diagnosis [4]. Therefore, the development of image denoising techniques [31] that can effectively handle CT modalities emerges as a critical and urgent need in clinical practice, to ensure both patient safety and diagnostic precision.

In recent years, advanced deep learning networks have proven to be highly effective in reducing noise in low-dose computed tomography (LDCT) than traditional denoising methods [3, 7]. Supervised denoising methods [16, 15], such as CTformer [27] and ASCON [2], learn the end-to-end map** from low-dose to normal-dose CT images. Generative adversarial networks (GANs) are also utilized in LDCT denoising task, which do not need paired data, but lots of unpaired data for training.[24, 1, 14, 12].

Despite impressive results, these methods encounter challenges as they require both LDCT and NDCT images [18], either paired images or a large amount of unpaired images, which are often unavailable in practice due to high costs, privacy, and ethical concerns. Therefore, it is essential to develop self-supervised methods that harness the potent capabilities of deep neural networks while minimizing the need for extensive labeled data. Several self-supervised methods have been proposed for LDCT denoising, including but not limited to Blin2Unblind [28], Noise2Sim [23], Neighbor2Neighbor [10] and FIRE [19] among others [26, 13, 28]. However, these methods primarily concentrate on spatial domain information, overlooking the critical importance of frequency domain details. The crucial distinction between low-dose CT and normal-dose CT in high-frequency components (see Fig. 1) is not well explored.

In this paper, we design a novel self-supervised LDCT denoising method, only using NDCT data, called WIA-LD2ND. We first analyze LDCT denoising task from the frequency perspective and then propose a module called Wavelet-based Image Alignment (WIA), which aligns LDCT with NDCT by mainly adding noise to the high-frequency components of both LDCT and NDCT. We also propose a module called Frequency-Aware Multi-scale Loss (FAM) to capture high-frequency components in multi-scale feature space.

Our WIA-LD2ND offers three major contributions as follows: 1) We analyze the LDCT denoising task from a frequency perspective, offering novel insight into its optimization. 2) We introduce a simple and efficient module to align NDCT and LDCT, facilitating self-supervised learning. 3) We propose a frequency-aware multi-scale loss, enabling the reconstruction network to effectively handle high-frequency components.

(a) Visualization of the results of NDCT and LDCT after Wavelet transform results

(b) LF components of NDCT

(c) LF components of LDCT

(d) HF components of NDCT

(e) HF components of LDCT

Figure 1: (a) Visualization of results of NDCT and LDCT after Discrete Wavelet Transform (DWT). The primary differences between NDCT and LDCT are at the high frequency components $[LH,HL,HH]$. (b-c) Visualize the normalized low-frequency (LF) component $LL$ features of NDCT and LDCT, while (d-e) display the normalized high-frequency (HF) component $[LH,HL,HH]$ features. We adopt the first residual block of pre-trained ResNet-18 [9] to extract image features.

Figure 2: Overview of our proposed WIA-LD2ND. NDCT image $x$ is passed through the WIA and then fed into the reconstruction network. The high-frequency components of the denoised CT $y$ and the input image $x$ are both fed into FAM to compute the loss capturing high-frequency components in multi-scale feature space.

2 Method

Figure 2 presents the overview of the proposed WIA-LD2ND, comprising of two novel modules: Wavelet-based Image Alignment (WIA) and Frequency-Aware Multi-scale Loss (FAM). NDCT image $x$ is passed through the WIA to destroy some high-frequency components and then fed into the reconstruction network. The high-frequency components of the reconstructed CT $y$ and input $x$, [$y_{LH}$, $y_{HL}$, $y_{HH}$] and [$x_{LH}$, $x_{HL}$, $x_{HH}$], are relatively flat and hard to capture [5], as shown in Fig. 1 (a). To address this challenge, we propose integrating these high-frequency components into encoders to compute a loss $\mathcal{L}_{FAM}$ within the feature space to enhance the capability of the reconstruction network in capturing high-frequency components and detailed information more effectively. During training, we employ an alternating learning strategy to optimize the reconstruction network and FAM to improve learning efficiency and accuracy of results, which is similar to GAN-based methods [32]. We begin by analyzing the LDCT denoising task from a novel perspective, followed by detailed introduction of the two proposed modules.

2.1 Analysis of LDCT Denoising From Frequency Perspective

Images contain different frequency ranges and spatial locations information. The Discrete Wavelet Transform (DWT), using the Haar wavelet as in [17], is selected for frequency analysis for its simplicity and efficiency. DWT is commonly employed in the field of computer vision and offers a straightforward and computationally effective technique for dividing the input image into low-frequency sub-band and high-frequency sub-bands. It has four filters, ${LL^{T}}$, ${LH^{T}}$, ${HL^{T}}$ and ${HH^{T}}$, demonstrating the texture, horizontal details, vertical details, and diagonal information respectively [29], in which low and high pass filters are:

$$L^{T}=\frac{1}{\sqrt{2}}[1,1],H^{T}=\frac{1}{\sqrt{2}}[-1,1]$$ (1)

As shown in Fig. 1 (a), after DWT, the main differences between normal-dose CT (NDCT) and low-dose CT (LDCT) are observed in the high-frequency sub-images $[LH,HL,HH]$, with little difference in the $LL$. Fig. 1 (b-e) further support our conclusion, demonstrating that LDCT and NDCT have significant differences in the high-frequency components at the feature space, while differences in the low-frequency components are comparatively minor. In Fig. 3 (a-c) and Fig. 4, we find that previous LDCT denoising methods such as BM3D [3] performs poorly at reconstructing high-frequency components.

Based on these observations, we conclude that high-frequency components should be the main focus for LDCT denoising, where previous methods falter the most. This highlights the critical necessity for improved techniques in handling high-frequency components during the denoising process.

2.2 Wavelet-based Image Alignment

According to previous analysis, we employ the Discrete Wavelet Transform (DWT) to decompose the input image $x$ into two sets of components: low-frequency component denoted as $x_{LL}$, which captures and preserves the smooth surface and texture information, and high-frequency components denoted by [$x_{LH}$, $x_{HL}$, $x_{HH}$]. These high-frequency components are essential for capturing intricate texture details, representing the primary distinctions between low-dose CT (LDCT) and normal-dose CT (NDCT) images. Therefore, we make them similar in a simple and effective way, by mainly adding Gaussian noise into the high-frequency components of NDCT and LDCT images.

$$x_{LL}^{\prime}=x_{LL}+noise_{LL},\quad x_{LH}^{\prime}=x_{LH}+noise_{LH},$$ (2)

$$x_{HL}^{\prime}=x_{HL}+noise_{HL},\quad x_{HH}^{\prime}=x_{HH}+noise_{HH},$$

where $noise_{LL},noise_{LH},noise_{HL},noise_{HH}$ follow Gaussian distributions with mean 0 and variance $\sigma_{LL}^{2},\sigma_{LH}^{2},\sigma_{HL}^{2},\sigma_{HH}^{2}$, respectively. Notably, $\sigma_{LH},\sigma_{HL}$, and $\sigma_{HH}$ are larger than $\sigma_{LL}$. We then conduct inverse DWT (IDWT) on the modified components [$x_{LL}^{\prime}$,$x_{LH}^{\prime}$, $x_{HL}^{\prime}$, $x_{HH}^{\prime}$] to reconstruct $x^{\prime}$. As shown in Fig. 3 (d-e), after applying WIA module, NDCT and LDCT images share the same feature space, indicating successful alignment.

WIA eliminates the need for paired data. Instead, we only require NDCT images for training a model to denoise from $x^{\prime}$ (NDCT after WIA) to $x$ (original NDCT), thereby facilitating self-supervised learning.

(a)

(b)

(c)

(d)

(e)

Figure 3: (a-c) Visualization of NDCT, residual between NDCT and result of BM3D [3] (a classical denoising method), and high-frequency components in spatial domain. The residual is converted into a clean binary image for clarity. We filter high-frequency band from image and then convert the result into a binary image. (d-e) Visualization of the tSNE images of feature distribution on the NDCT, LDCT, and their respective transformations after applying WIA on Mayo-2016 dataset. We adopt the first residual block of pre-trained ResNet-18 to extract image features.

2.3 Frequency-Aware Multi-scale Loss

High-frequency components play a crucial role in low-dose CT images denoising, as analyzed in Sec. 2.1. However, the CNN-based and Transformer-based models tend to focus primarily on low-frequency representations, making it difficult for models to capture the high-frequency components [5]. Therefore, we design Frequency-Aware Multi-scale Loss (FAM) which is to focus the network more on the high-frequency components and detail information of the images.

The high-frequency components of denoised CT $y$ and NDCT $x$, [$y_{LH}$, $y_{HL}$, $y_{HH}$] and [$x_{LH}$, $x_{HL}$, $x_{HH}$], are fed into Online Encoder $E$ and Target Encoder $E^{\prime}$, respectively, both using the same lightweight architecture. Following previous studies [6, 8], the parameters of the Target Encoder $E^{\prime}$ are an exponential moving average of the parameters in the Online Encoder $E$. The process is as follows:

$$f_{1},f_{2},f_{3}=E(x_{LH},x_{HL},x_{HH}),f_{1}^{\prime},f_{2}^{\prime},f_{3}^{\prime}=E^{\prime}(y_{LH},y_{HL},y_{HH}).$$ (3)

We introduce a Frequency-aware Attention mechanism in the encoders, designed to selectively emphasize or de-emphasize areas within the input feature map based on their frequency content, as illustrated in Fig. 2 Given the input feature map ${f_{n}}_{\{n=1,2,3\}}$, we apply max and average pooling to extract prominent features. We then concatenate them and pass through a convolutional layer with Sigmoid activation to generate spatial attention weights.

Unlike previous studies [2, 30], our approach segments multi-scale features extracted from specific layers into patches $f_{n}^{(i)}$. We then select patches that are most similar to their adjacent counterparts, focusing on those that exhibit shared structural characteristics closely associated with high-frequency components. To this end, we use the cosine similarity $s$ on the feature space:

$$\text{s}(i,j)=f^{(i)^{\top}}f^{(j)}/\|f^{(i)}\|_{2}\|f^{(j)}\|_{2}.$$ (4)

We then select the similar feature patches $\{f_{n}^{(j)}\}_{j\in P^{(i)}}$, where $P^{(i)}$ is a set of feature patch indices of top-4 patches. For the $f^{\prime(i)}$ from the Online Network, we select the same positive feature patches $P^{(i)}$. We then aggregate positive patches $\{f_{n}^{(j)}\}_{j\in P^{(i)}}$ and $\{f_{n}^{\prime(j)}\}_{j\in P^{(i)}}$ using global average pooling (GAP) and multi-layer perceptron (MLP) yielding $g$ and $g^{\prime}$, respectively. Finally, the Frequency-Aware Multi-scale Loss $\mathcal{L}_{FAM}$ is given by:

$$\mathcal{L}_{FAM}=\|g-g^{\prime}\|_{2}^{2}.$$ (5)

The final loss is defined as $\mathcal{L}=\mathcal{L}_{pixel}(x,y)+\lambda\mathcal{L}_{FAM}$, where $\mathcal{L}_{pixel}$ consists of two common supervised losses: MSE and SSIM, defined as $\mathcal{L}_{pixel}(x,y)=\mathcal{L}_{MSE}(x,y)+\mathcal{L}_{SSIM}(x,y)$. $\lambda$ is set to 0.01 in this paper.

Table 1: Performance comparison on the Mayo-2016 [21] and Mayo-2020 [22] datasets. The best result is in bold, and the second best is underlined.

Methods	Mayo-2016		Mayo-2020		Avg
	PSNR	SSIM	PSNR	SSIM	PSNR	SSIM
BM3D (TIP2007) [3]	35.50	86.95	37.67	89.53	36.59	88.24
DIP (CVPR2018) [26]	37.25	85.94	40.16	95.89	38.71	90.92
Noise2Sim (TMI2022) [23]	38.51	90.15	39.16	90.90	38.84	90.53
Blind2Unblind (CVPR2022) [28]	35.84	81.15	40.96	94.84	38.40	88.00
Neighbor2Neighbor (TIP2022) [10]	35.27	86.79	36.78	94.06	36.03	90.43
ZS-N2N (CVPR2023) [20]	38.10	87.64	44.00	97.06	41.05	92.35
CycleGAN (ICCV2017) [32]	37.68	89.22	40.18	97.93	38.93	93.58
CUT (ECCV2020) [24]	37.96	89.93	41.11	97.61	39.54	93.77
WIA-LD2ND (ours)	38.15	90.00	44.64	98.31	41.40	94.16

Figure 4: Qualitative comparison of different methods on the Mayo-2020 dataset [22].

3 Experiments

3.1 Dataset and Training Details

We conduct experiments on two public LDCT denoising datasets, Mayo-2016 and Mayo-2020, from the NIH AAPM-Mayo Clinic Low-Dose CT Grand Challenge [21, 22]. We select 5410 image pairs (512×512) from 9 patients in Mayo-2016 for training and 526 for testing. We select the reconstruction parameter combination of {1mm, D45}. From Mayo-2020, 2082 pairs (512×512) from 12 patients are used for training, with 672 pairs from 4 patients for testing.

In all our experiments, we only use NDCT and choose a backbone identical to that used in [32, 24], along with employing the same data augmentation strategy as in [2]. For Mayo-2016, we set $\sigma_{LL}=100$, $\sigma_{LH}=200$, $\sigma_{HL}=200$, and $\sigma_{HH}=150$ in Eq. (2). For Mayo-2020, the noise variances are $\sigma_{LL}=25$, $\sigma_{LH}=50$, $\sigma_{HL}=50$, and $\sigma_{HH}=50$. We employ the Adam optimizer with the momentum parameters as $\beta_{1}$ = 0.9, $\beta_{2}$ = 0.99 and initial learning rate $1.0\times 10^{-4}$. Our network is trained over 200 epochs using a single NVIDIA GeForce RTX 3090.

3.2 Experiments Results

To evaluate the denoising efficacy of our WIA-LD2ND, we conduct comparative experiments against various denoising methods. The comparisons include traditional methods like BM3D [3], self-supervised methods including DIP [26], Noise2Sim [23], Blind2Unblind [28], Neighbor2Neighbor [10] and ZS-N2N [20] as well as weakly-supervised methods such as CycleGAN [32] and CUT [24]. We use two widely-adopted metrics, namely peak signal-to-noise ratio (PSNR) and structural similarity index measure (SSIM) to evaluate the performance.

Table 1 shows that our WIA-LD2ND, only using NDCT images, can achieve good performance on both the Mayo-2016 and Mayo-2020 datasets. Compared to the latest state-of-the-art self-supervised and weakly-supervised methods, our WIA-LD2ND achieves significant performance improvements.

Figure 4 presents the reconstruction results, with the subplots in the bottom left corner of the pictures showing the noise power spectrum (NPS), where blue indicates it is closer to the normal-dose CT. As illustrated, our WIA-LD2ND achieves the best results, reconstructing the most detailed information and exhibiting the bluest NPS. Conversely, the results of BM3D and DIP are over-smoothed and compromised with structured artifacts. Additionally, other deep-learning-based methods tend to remove noise aggressively. Our WIA-LD2ND prioritizes the preservation of informative details.

Table 2: Ablation studies are conducted to validate the effectiveness of each module on the Mayo-2016 [21] and Mayo-2020 [22] datasets. WIA^* represents directly adding Gaussian noise to NDCT, while FAM^* denotes the computation of high-frequency components of $y$ and $x$ at the feature level by directly employing the MSE loss.

Methods	#Params	Mayo-2016		Mayo-2020		Avg
		PSNR	SSIM	PSNR	SSIM	PSNR	SSIM
Baseline	11.37M	34.26	78.03	40.32	96.99	37.29	87.51
Baseline + WIA*	11.37M	35.49	87.73	42.00	98.26	38.75	93.00
Baseline + WIA	11.37M	37.85	89.77	42.73	98.26	40.29	94.02
Baseline + FAM*	13.83M	34.00	79.12	41.97	98.02	37.99	88.69
Baseline + FAM	13.83M	34.78	80.27	42.46	98.25	38.62	89.15
WIA-LD2ND	13.83M	38.15	90.00	44.64	98.31	41.40	94.16

Ablation Studies. To evaluate the effectiveness of our proposed modules, including WIA and FAM, we conduct ablation experiments on Mayo-2016 [21] and Mayo-2020 [22]. The results are shown in Table 2. WIA^* represents adding noise directly to NDCT, while FAM^* involves the computation of MSE loss directly for the feature of high-frequency components. Our designs achieve better performance than their variants. It reveals that both WIA and FAM are well-designed and contribute to performance gains. More ablation studies on additional hyperparameters and noise parameters are available in the supplementary materials. WIA-LD2ND incurs an increase of 2.46M parameters compared to the baseline. Considering the significant performance improvement over the baseline model without any extra inference time, this slight increase in training cost is acceptable.

4 Discussion and Conclusion

In this paper, we analyze the LDCT denoising task from a novel perspective and propose a self-supervised method called WIA-LD2ND. This method only utilizes NDCT images and incorporates two novel modules: Wavelet-based Image Alignment (WIA), which aligns NDCT and LDCT by destroying some high-frequency components, and Frequency-Aware Multi-scale Loss (FAM), which enhances the reconstruction network’s ability to capture high-frequency components and detailed information, thus improving denoising performance. Extensive experimental results demonstrate the superior performance and the effectiveness of our designs. It is noteworthy that WIA-LD2ND increases the number of parameters by 2.46M compared to the baseline during training, without requiring extra inference time. Exploring LDCT denoising from a frequency perspective presents a promising direction for future research.