Deep PCCT: Photon Counting Computed Tomography Deep Learning Applications Review

The original search identified 1,949 articles, 64 in PubMed and 1,885 in Scopus. After removing duplicate (218), ineligible records (1,275) and not English (seven), 449 articles were identified for screening. Ineligible records included encyclopedias, book chapters, conference abstracts, discussions, and editorials. Screening of 449 articles based on the titles and abstract resulted in excluding another 386 articles. These articles were excluded because the abstracts did not contain information about PCCT or information related to it. In total 63 articles were intended for inclusion in this manuscript. Finally another 11 articles were not accessible and therefore this resulted in a total of 52 final papers regarding topics related to PCCT. The PRISMA [7] diagram in Figure 1 provides a summary overview of the screening.

A typical radiomics study consists of numerous steps, each of which may be influenced by various factors. In the context of PCCT scans, the methodology typically involves the following key steps:

1. 1.

   Imaging acquisition

2. 2.

   Image segmentation - identify and delineate ROI in two dimensions or the VOI in three dimensions in PCCT scans. This often involves segmenting specific anatomical structures or lesions, which can be done manually with the help of radiologists, semi-automatically with standard image segmentation algorithms such as region growing or thresholding, or fully automatically with deep learning algorithms. Different open-source software solutions are available, such as 3D Slicer, ITK-SNAP, or ImageJ.

3. 3.

   Image Processing - attempts to homogenize the images from which the radiomics features will be extracted in terms of pixel spacing, grey-level intensities, and grey-level histogram bins. At this stage, it is essential to report each detail of the image processing step to ensure the reproducibility of the results.

4. 4.

   Feature Extraction - refers to the calculation of the characteristics used to quantify the intensity of gray levels (histogram), shape, texture, transformation and radial aspects within the ROI/VOI. It is recommended to follow the Imaging Biomarker Standardization Initiative guidelines due to the variety of methods used to calculate these features. PyRadiomics is a commonly used tool in this process. Radiomic features are of various types, such as intensity-based (histogram), shape-based, texture-based, transform-based and radial features. During feature extraction, different filters are often applied, such as wavelet or Gaussian filters.

5. 5.

   Feature Selection - identify the most informative features that significantly impact the task of diagnosis or prognosis through statistical analysis or cluster analysis.

In summary, the radiomics methodology for PCCT exams involves a systematic process of segmentation, image processing, feature extraction and feature selection. This final set of features is then used to create predictive models and extract valuable information to aid clinical decision-making, thus increasing the usefulness of PCCT scans in healthcare [15]. Dunning et al. developed a ML model, a SVM to automatically classify the risk of coronary plaques based on radiomic features extracted from different types of images obtained by PCCT [16], and Mundt et al. developed a logistic regression model that allows predicting the presence of calcification in coronary arteries by analyzing the features extracted by radiomics [17]. Landsmann et al. use a CNN-based approach to distinguish breast density, and therefore related health problems (higher tissue density, higher probability of breast cancer), through features derived from texture analysis [18]. Table 2 summarizes the articles that study and analyze radiomics in PCCT.

Mundt et al. investigated the relationship between the texture of periaortic adipose tissue, analyzed using radiomic features, and coronary artery calcification, since cardiovascular diseases are one of the main causes of global mortality, and therefore their understanding and identification remains crucial. The patients (mean age 56 years, 34 men, 21 women) underwent PCCT examinations. The periaortic adipose tissue surrounding the thoracic descending aorta was manually segmented on non-contrast axial images. Patients were stratified into three groups based on their coronary artery calcification status: Agatston score 0, Agatston score 1-99, and Agatston score $\geq$ 100. Pyradiomics was used to extract radiomic features (106 in total). R statistics facilitated the statistical analysis, including mean and standard deviation calculations, along with Pearson’s correlation coefficient for feature correlation. Random Forest classification helped with feature selection, and visualization was achieved through boxplots and heatmaps. The results highlighted two higher-order radiomic features, "glcm ClusterProminence" and "glcm ClusterTendency", which differed between patients without CAC and those with significant CAC according to Random Forest classification. "glcm ClusterProminence" emerged as the main differentiating feature. The results suggest that changes in the texture of periaortic adipose tissue may be correlated with coronary artery calcium score, suggesting a possible influence of inflammatory or fibrotic activity in perivascular adipose tissue [17].

Other studies investigating radiomic analysis in PCCT have been performed, such as Tharmaseelan et al. who studied the metabolic activity of perivascular adipose tissue and its association with aortic calcification and aortic diameter. The study explores the potential influence of periaortic adipose tissue on the development of aortic calcification. Semi-automatic segmentation of the periaortic adipose tissue was performed and Pyradiomics was used to extract radiomic features. Statistical analysis in R statistics involved calculating the mean and standard deviation, along with Pearson’s correlation coefficient for feature correlation. Random Forest classification was used for feature selection and an unpaired two-tailed t-test was applied to the final set of features. The results, presented as boxplots and heatmaps, covered data from 30 patients categorized based on the presence of local aortic calcification. Using Random Forest feature selection, a set of seven higher-order features was identified to differentiate periaortic adipose tissue texture between the two subgroups (with and without local aortic calcification). The subsequent t-test showed statistically significant discrimination for all characteristics. The results therefore suggest that textural changes in periaortic adipose tissue, associated with the presence of local aortic calcification, could potentially serve as the basis for identifying a biomarker for the development of atherosclerosis [19].

Staying focused on the heart region, Dunning et al. performed a study using PCCT images to classify coronary plaque risk using a ML model based on radiomics. Nineteen patients with coronary CT angiography were scanned on a PCCT system, generating five types of images: VMIs at 50, 70 and 100 keV, iodine maps and VNC. A total of 93 radiomic features were extracted from each image and a statistical analysis was carried out to identify significant features that distinguished low- and high-risk plaques. Two significant features were then input into a SVM for ML classification. The study used a LOOCV strategy, achieving classification accuracy. The results indicated that the ML model, particularly using 100 keV VMIs and VNC images, effectively differentiated between low- and high-risk coronary plaques in PCD-CTA [16].

The first studies, such as the studies by Dunning and Wolf, were performed on phantoms or organic samples. Dunning’s study demonstrated that, in organic samples, 14 radiomics features were obtained in both scans. Still, when they were compared, it was found that 13 of the 14 radiomics features studied were visibly altered by the improved resolution of the PCCT system. These results show that the features derived from high-resolution PCCT can potentially alter radiomics-based clinical interpretation [20]. As a complement, Wolf et al. studied the impact of different VMI reconstructions on radiomic features in in vitro and in vivo PCCT datasets. 12 organ phantoms and 23 patients were scanned. In the phantom scans, images were reconstructed at six keV levels and the results showed excellent repeatability of the radiomic features. For in vivo patients, the percentage of stable features was high among 90 keV $\geq$ VMIs. At the same time, comparison with lower levels of VMI led to fewer reproducible features, so it was concluded that there is a better reproducibility correlation between higher VMIs in myocardial texture [21].

Sharma et al. conducted a study using an anthropomorphic phantom with three different morphologies for lung lesions, comparing the EICT with the radiomic study. Both systems extracted 14 radiomic features. The findings indicate a reduced mean estimation error in PCCT, specifically in independent features (35.9% vs. 54.0%) and overall features (54.5% vs. 68.1%). Statistically significant error reductions were observed for eight out of the fourteen features. In the experiments, tests were performed to see the effect of changing parameters such as the radiation dose, the distance to the isocenter, the pixel size and the reconstruction kernel. For both systems, the accuracy of the estimate was minimally affected by the radiation dose administered and the distance from the isocenter of the model used, while the reconstruction kernel and pixel size had a relatively stronger effect on the results obtained [22].

Another study compared the properties of the radiomic features of the myocardium on CT with an EID and a PCD. In a total of 50 patients, they obtained similar results with regard to first-order features. However, the higher-order features showed a partially significant difference between PCCT and EICT, being clearer with PCCT. This indicates a possible impact of better spatial resolution, better low-energy photon detection and a better SNR on texture analysis in PCCT [23]. On the same topic, the same author wrote a study with 30 patients in which it was possible to differentiate between patients with coronary artery calcification and patients without coronary artery calcification using four different myocardial texture parameters ("gldm SmallDependenceHighGrayLevelEmphasis", "glcm ClusterShade", "glrlm LongRunLowGrayLevelEmphasis", and "ngtdm Complexity") extracted using radiomics. Another proof of the importance of radiomics analysis in PCCT [24].

Schwartz’s study compared the image quality of high-resolution, low-dose PCCT with standard chest EICT in the same patients. After all radiomics pipeline procedures, results showed that PCCT produces higher measured and perceived image quality at lower radiation doses than EICT, despite greater measured quantitative and qualitative noise by using a lower dose of radiation [25].

Regarding PCCT in oncology patients, Allphin applied transplanted soft tissue sarcomas with a high mutational load in Rag2+/- and Rag2-/- mice to model the variable load of lymphocytes. They were tested in EICT and PCCT, and the radiomics features were extracted. The radiomics features allowed differentiation between Rag2+/- and Rag2-/- tumors; in other words, tumors with different lymphocyte loads have slight modifications in their architecture that are not detected with conventional image-derived tumor metrics but which are better revealed in radiomics analysis of PCCT images with nanoparticle contrast [26].

In differentiation tasks, Kahmann studied the differences in the presence or absence of hypercholesterolemia in the radiomics analysis of 66 patients and verified the existence of two radiomics features. Patients with hypercholesterolemia had a higher concentration of high-density values. Analysis of pericoronary adipose tissue texture discriminated between patients with and without hypercholesterolemia. This is an important step in the development of new treatments for cardiovascular diseases [6].

Therefore, one of the main limitations of radiomics analysis is the reproducibility of radiomics features. To overcome this issue, a radiomics analysis was performed on phantom scans using a PCCT. The study aimed to assess the stability of radiomics analysis, focusing mainly on the consistency of the extracted radiomics parameters. The research revealed that most of the extracted radiomics features showed excellent stability, both in test-retest analyzes and in new analyzes after repositioning [28]. A similar study was carried out on human subjects and came to the same conclusion. These results highlight the robustness of radiomics analysis using PCCT data, marking a significant step towards integrating radiomics analysis into clinical practice [29]. These articles are summarized in Table 4.

The first CT scanners in the 1970s used iterative reconstruction algorithms, but their clinical use was limited due to computational constraints. It wasn’t until 2009 that the first commercial iterative reconstruction algorithms replaced conventional Filtered Back Projection (FBP). Reconstruction algorithms for CT vary in complexity, including hybrid, model-based, and fully iterative methods:

1. 1.

   Iterative Reconstruction: combines forward and backward projection steps to iteratively refine the image, for example, IRIS (Iterative Reconstruction in Image Space);

2. 2.

   Hybrid Approach: combines the speed of FBP with the refinement capabilities of iterative reconstruction. The initial steps often involve FBP, followed by iterative refinements to improve the image to take advantage of both methods. An example is the iDose4 algorithm;

3. 3.

   Model-based: uses a statistical model to represent the image and the data acquisition process. It improves image quality through iterative refinement and complex optimization algorithms, providing greater spatial resolution and lower noise levels, like SAFIRE (Sinogram Affirmed Iterative Reconstruction).

Due to the additional multi-energy information and detector elements, PCCT data is more complex than conventional CT data. Consequently, specialized reconstruction algorithms are required. Many authors choose DL methods based on CNN to learn the map** between images and data. This removes the need for explicit knowledge of the underlying statistical model. X-ray exposure per projection or the number of projections is minimized in order to reduce the image dose. However, this reduction leads to a lack of photons or undersampling artifacts in the image. In other words, the higher the radiation dose (the more harmful to the patient), less noise in the medical image [31].

Previous research has shown that noise reduction based on deep learning can provide performance equivalent to that of iterative reconstruction and a significant reduction of computing time. Several DL architectures have been used to remove noise in PCCT images, including U-networks [31], [32], [33], Pie-Net [34], encoder-decoder CNN [35], many models using CNN but then it varies inside it like the one that uses image-based noise insertion and PCCT images to train a U-Net via supervised learning [36], or just a simple CNN [37], or even a more extensive system involving CNNs named PKAID-net [38].

Another approach to image reconstruction in PCCT is QIR algorithms, which have been developed to satisfy the increasing hardware and software requirements of PCCT in terms of data complexity, spectral information, and noise profile. There are four strength levels of QIR (1-4) [39]. Recent research, including studies by Woeltjen and Sartoretti, has shown the impact of the QIR algorithm on enhancing the overall quality of lung images in PCCT. This reinforces the algorithm’s significance in medical imaging [40], [41], [42].

Yang et al. introduced an advanced CT spectral reconstruction method focusing on reducing radiation doses and resolving data noise. The approach uses the inherent correlation between channels acquired from the same object to improve reconstruction quality. By employing two iterative reconstructions with sparsity and low-degree dual constraints, the authors designed a heuristic strategy to address the optimization challenges. They incorporated a fusion framework using guided filters to fuse the results, obtaining satisfactory visualization, higher PSNR and SSIM values, and reduced material decomposition errors [43].

Gruschwitz et al. evaluated the influence of various vascular reconstruction kernels on the image quality of CT angiographies. The investigation incorporated a 1st generation PCCT and was matched with dose equivalent scans performed on a 3rd generation EICT. In particular, the reconstructions were meticulously carried out with various vascular cores, with a specific emphasis on aligning the individual modulation transfer functions between the two scanners. To assess the objective quality of the image, SNR and CNR were calculated. In addition, subjective assessments were made by six radiologists using a forced-choice pairwise comparison tool. An imperative characteristic to be analysed was intraluminal attenuation. The results of the study emphasised that the use of sharper convolution kernels in PCCT consistently resulted in superior image quality when compared to EICT, regardless of the specific reconstruction kernels used. This implies that PCCT has advantages in terms of intraluminal attenuation, noise reduction and overall image quality in the context of angiographies. The better subjective image quality obtained with PCCT further facilitates the application of sharper convolution kernels, thus potentially improving the assessment of vascular structures in clinical contexts [44].

Graafen et al. explored the application of PCCT for the non-invasive diagnosis of hepatocellular carcinoma (HCC). The research focused on assessing the improvements in image quality, including noise reduction and higher spatial resolution, offered by PCCT. The investigation involved phantom experiments to analyze different reconstruction kernels and a patient population study for HCC imaging. Results showed that softer reconstruction kernels (in image processing, a kernel refers to a small matrix used for various operations, such as blurring, sharpening, relief, and edge detection) yielded superior overall quality for HCC evaluation in PCCT, exhibiting the highest contrast-to-noise ratio in all contrast phases. There were no significant differences in image contrast or the lesion between the body and quantitative nuclei with equal sharpness levels. The findings support the preference for softer reconstruction kernels in HCC assessment with PCCT, with quantitative kernels showing potential for spectral post-processing without compromising image quality [30].

However, despite advances in DL-based denoising, significant challenges remain. One notable concern is the potential for overfitting to training data, resulting in limited generalization performance. The achievement of robust generalization is particularly critical in removing noise from PCCT images. This is due to the variations in PCD energy thresholds influenced by different contrast agents, thus affecting noise levels in various energy channels. Moreover, there is a clear need for reliable evaluation metrics to assess the performance of DL-based noise reduction algorithms accurately. Table 5 below shows some of the articles mentioned above.

These algorithms have been widely developed and improved with the advancements in DL methods. By automating the segmentation process, these algorithms eliminate the need for manual intervention, saving time and effort for radiologists and improving workflow efficiency. Segmentation algorithms in PCCT scans have been implemented using various approaches, such as automated region-growing algorithms [45] and CNNs [27]; besides, manual segmentation remains a viable option.

Do et al. evaluated the efficiency of energy thresholds and acquisition modes in minimizing metallic artifacts in PCCT and EICT. The study found that PCCT has higher noise levels than EICT regardless of the tube’s potential. This indicates that using high-energy compartments implies a trade-off between increased noise and decreased overall image contrast since the reconstruction process does not incorporate low-energy photons [47].

The challenge of noise reduction is inherent in most deep learning tasks, as networks must adapt and generalize their performance to training data that contains noise. In the context of noise reduction in PCCT, there is a correlation between the radiation dose - potentially harmful to patients - and the level of noise in the reconstructed images. To reduce radiation exposure, it is necessary to reduce either the X-ray exposure per projection or the number of projections. However, this reduction leads to challenges such as photon starvation or image subsampling artifacts [48].

Zhou et al. demonstrated the utility of high-energy compartments in reducing metal artifacts induced by implants in various body parts, such as the spine, shoulder, wrist, ankle, and elbows. Their study compared EICT with PCCT with tin filtering, observing reduced metallic artifacts, greater diagnostic confidence, and better anatomical recognition in images within the > 75 keV bin, emphasizing the importance of the PCCT use in medicine [49]. Another study on the same topic was carried out by Anhaus et al. using multiple phantoms, comparing monoenergetic images from 40 keV to 190 keV (in 10 keV increments) with and without iMAR. Their conclusions indicated that mono-energetic imaging alone did not effectively reduce metallic artifacts in materials with high atomic numbers, such as dental implants, the head of the hip, and the embolization coil. However, the inclusion of iMAR demonstrated a significant reduction in metal artifacts. Specifically, the ideal keV was identified for spinal implants as 100 keV and 120 keV when scanning with 120 and 140 kVp with tin filtering, respectively [50].

PCCT allows the reconstruction of mono-energetic images. Layer et al. studied polychromatic and monochromatic images with energy levels of 100 keV, 130 keV, 160 keV, and 190 keV, with and without iMAR, for unilateral and bilateral total hip arthroplasty patients. The researchers observed a reduction in metal artifacts and better diagnostic interpretation for bone, muscle, vessels, bladder, and soft tissue in polychromatic images with iMAR. They also found similar benefits in mono-energetic images ranging from 100 to 190 keV, with and without iMAR, with the best result observed in 100 keV mono-energetic images with iMAR [51]. Björkman et al. conducted an exhaustive evaluation, both subjective and objective, comparing the efficiency of PCCT and DECT with and without iMAR in the attenuation of metallic artifacts in a bovine knee with orthopedic plates and screws. The subjective evaluation by the researchers revealed higher scores for image quality and the assessment of fracture lines, bone structures, and the articular surface in images reconstructed with PCCT when compared to those obtained from DECT. Nevertheless, the objective analysis did not identify any statistically significant differences between the two imaging techniques [52].

In another area of application, Patzer et al. observed a reduction in metallic artifacts caused by dental implants and better visualization of soft tissues. They compared poly-energetic and mono-energetic images from 40 to 190 keV with iMAR and the same range without iMAR. In addition, they observed a more significant reduction in metallic artefacts and better visualization of soft tissue in mono-energetic images from 110 to 190 keV with iMAR, compared to poly-energetic images with iMAR [53].

These results contribute to understanding the optimization of imaging parameters to reduce metallic artifacts in CT scans, particularly with emerging technologies such as PCCT. Efficient denoising not only improves the detectability of smaller lesions and anatomical structures, but also plays a vital role in maintaining the reliability of quantitative imaging metrics. These metrics are crucial for monitoring therapy progress and evaluating treatment responses. Table 6 below shows some of the articles mentioned above regarding the material decomposition application of PCCT.

Material decomposition algorithms have been developed to provide additional information on tissue composition and make more informed diagnostic decisions while studying PCCT. These algorithms use advanced techniques, such as deep convolutional neural networks and segmentation algorithms, to accurately classify different groups of materials, including breast microcalcifications [54]. During phantom studies, PCCT demonstrated precise measurements of material concentrations for iodine, gadolinium, and calcium [55].

Wang et al. presents a thermometry algorithm for PCCT that can accurately predict the temperatures of unknown materials. This is done based on the spectrally resolved linear attenuation coefficients of a set of known base materials at various temperatures. This result is particularly relevant to surgery, as it offers a solution for dealing with the variability of tissue properties without the need to calibrate the tissue in the body in real-time [56].

In order to improve a limitation of PCCT related to spectral distortions that affect the accuracy of material decomposition, the article by Nadkarni et al. proposes and demonstrates a solution that uses a DL approach to compensate for these spectral distortions and improve the accuracy of material decomposition in PCCT: Iter2Decomp. This algorithm showed the best results compared to other methods, including matrix inversion decomposition [57].

Wu et al. conducted a study focusing on multi-material decomposition in spectral PCCT using a DL approach. The researchers proposed a method employing improved Fully Convolutional DenseNets to categorize and measure various materials. Their experimental results demonstrated that this approach outperformed traditional methods, particularly in high noise levels, effectively identifying bone, lung, and soft tissue compared to the basis material decomposition based on post-reconstruction space. The findings suggest that the proposed deep learning-based approach holds promise for improving spectral CT material decomposition and may contribute to establishing guidelines for future multi-material decomposition methods in this imaging context [58].

Bhattarai et al. explored the diagnostic potential of advanced PCCT for the assessment of osteoporosis. Traditional methods, such as dual X-ray absorptiometry, focus on limited parameters, whereas PCCT, with its high spectral sensitivity, provides qualitative and quantitative information on bone microarchitecture and composition. The study demonstrated the accuracy of PCCT in breaking down calcium content in hydroxyapatite phantoms and rats. It revealed substantial distinctions in morphological parameters between untreated and treated tibial bones. It demonstrated the efficacy of PCCT in both qualitative and quantitative assessment of bone conditions, enabling three-dimensional visualization of the distribution of calcium and water components in bone. PCCT offers an improved way to diagnose bone disease, monitor treatment progress and assess fracture risk [59].