Abstract

Background: There is an absence of open-source libraries in emission tomography that (i) use modern and popular backend code to encourage community contributions and (ii) offer support for the multitude of reconstruction algorithms available in recent literature, such as those that employ artificial intelligence.
Purpose: To create and evaluate a GPU-accelerated, open-source, and user-friendly image reconstruction library, designed to serve as a central platform for the development, validation, and deployment of various tomographic reconstruction algorithms.
Methods: PyTomography was developed using Python and inherits the GPU-accelerated functionality of PyTorch and parallelproj for fast computations. Its flexible and modular design decouples the system matrix from reconstruction algorithms, simplifying the process of integrating new imaging modalities using various python tools.
Results: Example use cases demonstrate the software capabilities in parallel hole SPECT and listmode PET imaging. Use of the OSEM, BSREM, KEM, and DIPRecon reconstruction algorithms are explored. System matrices incorporating breathing-motion correction are explored in SPECT imaging. PyTomography is validated against alternative vendor/open source software for OSEM based reconstruction.
Conclusions: We have developed and publicly shared PyTomography, a highly optimized and user-friendly software for medical image reconstruction, with a class hierarchy that fosters the development of novel imaging applications.

I. Introduction

Medical imaging forms a cornerstone in modern healthcare by providing visual and quantitative information about internal body structures and functions. It enables early detection¹, accurate diagnosis, and precise treatment planning for a wide range of medical conditions^{2, 3, 4}, contributing to improved patient outcomes and enhanced medical decision-making. Reconstruction algorithms are routinely used to generate 3D images that can be used for both research and clinical decision making ⁵.

While the development and validation of reconstruction techniques remains an active field of research^{6, 7, 8, 9}, it can often be difficult to share and disseminate one’s findings. Additionally, while manufacturers of tomographic imaging equipment such as Single Photon Emission Computed Tomography (SPECT) and Positron Emission Tomography (PET) cameras provide their own internal reconstruction software, users cannot access all implementation details; this limits reproducibility in scientific studies. These issues could be solved by moving to an open source paradigm for medical image reconstruction. Here, the imaging community could openly collaborate in the development of reconstruction techniques. Adoption of a singular open-source framework, as opposed to private frameworks developed independently by various research groups, would strengthen the ability to develop, share, and compare findings. This could ultimately help accelerate the development and translation of new diagnostic imaging capabilities into clinical practice.

A few tools have already been developed to try and address some of these issues. STIR ¹⁰ and CASToR ¹¹ are two C/C++ based open source and collaborative image reconstruction frameworks that support a variety of reconstruction algorithms for SPECT and PET. NiftyPET ¹² is a python library which employs GPU-accelerated code written in CUDA C for PET reconstruction. QSPECT¹³ and TIRIUS support use of the ordered subset expectation maximum (OSEM) algorithm for SPECT and PET reconstruction respectively. Other libraries, such as TomoPy ¹⁴ and the ASTRA toolbox ¹⁵, support a variety of GPU-accelerated image tomographic reconstruction algorithms, although they lack extensive SPECT/PET system modeling. Web links to all these tools, as well as a list of other available tools for image reconstruction, can be found on the NMMItools webpage of the Society of Nuclear Medicine and Molecular Imaging (SNMMI). The unique purpose of this research is to build upon recent efforts in GPU-accelerated reconstruction^{16, 17, 18, 19} and AI-based reconstruction^{7, 8, 9} to establish an open source medical image reconstruction programming framework written in a popular programming language that facilitates collaboration. Valuable past efforts have focused on CUDA programming to accelerate reconstruction times in SPECT¹⁶ and PET ^{16, 17} reconstruction through the use of multi-threading. However, potential contributors to an open-source framework may be unfamiliar with CUDA; any tool that prioritizes community contributions should employ familiar backend code. In Python, the library NumPy is widely known, but it lacks direct GPU acceleration support. PyTorch, however, emerges as a suitable alternative since it maintains a similar interface to NumPy while also facilitating GPU acceleration and direct interface with neural networks. Some potential contributors may wish to provide contributions using GPU-accelerated functionality offered by other libraries, such as CuPy; an optimal framework should thus also permit the ability to incorporate alternative implementations for particular tasks.

To this end, the python library PyTomography has been developed with the following priorities: (i) implementation of standard and traditional imaging modalities and reconstruction algorithms, (ii) implementation and dissemination of recent research developments, such as the deep image prior⁸, and (iii) encouragement of community involvement via extensive documentation and user-friendly tutorials. While the present focus of PyTomography is the development and validation of reconstruction algorithms for SPECT/PET, implementation of other imaging modalities and reconstruction algorithms can be added using the building blocks provided.

Overall, the aim of this study is to introduce the community to the PyTomography package, demonstrate its current capabilities and flexibility, and to encourage community involvement in the continuation of its development. To demonstrate the versatility of the software, example applications will be explored in (i) SPECT reconstruction using PyTorch based rotate+sum projectors, and (ii) listmode (LM) time of flight (TOF) PET reconstruction using Joseph’s projectors from the python library parallelproj¹⁹. In what follows, we elaborate on our methods and results for PyTomography, which is made publicly available to download; relevant links can be found in section VI..

II. Materials and Methods

II.A. Mathematical Description

A brief overview of the mathematical paradigm of medical imaging is presented in order to motivate the software architecture. The mathematical conventions used in this paper are as follows. Vectors are represented by lower case letters, while linear operators are represented by upper case letters. The product of two vectors $vw$ and division of two vectors $v/w$ are inferred to be point-wise operations. Components of a vector are given by $v_{i}$, and components of linear operators are given by $A_{ij}$.

II.A.1. Projectors

In the paradigm of tomographic medical imaging, there are two main vector spaces: the “projection” space $\mathcal{V}$ consisting of measured projection data $g$ and the “object” space $\mathcal{U}$ consisting of 3D objects $f$. Since measured and reconstructed data is digitized, $\mathcal{U}$ and $\mathcal{V}$ are assumed here to be finite dimensional vector spaces. In SPECT imaging, for example, $\mathcal{V}$ consists of a set of 2D pixelated images acquired by counting photons emitted by radiopharmaceuticals, and $\mathcal{U}$ represents the voxelized spatial distribution of radioactivity concentration in the object being imaged.

Certain imaging systems can be characterized by a system matrix $H$, which corresponds to a linear operator map** from object space to projection space. Its components $H_{ij}$ represent a contribution of voxel $j$ to detector element $i$. In SPECT/PET, analytical modeling of the data expectation $\bar{g}$ can be achieved via $\bar{g}=Hf+s$ where $s$ represents scatters and randoms. In CT, analytical modeling is achieved via $\bar{g}=be^{-Hf}$ where $b$ denotes the rate of emissions from the transmission source to a particular measurement. The operation of $H$ on an object vector is known as forward projection (FP), while the operation of $H^{T}$ on a projection vector is known as back projection (BP).

Efficient implementation of FP and BP depends on the particular imaging modality; this typically involves decomposing $H$ into a sequence of less computationally burdensome operations. In general, FP can be decomposed into the following sequence: (i) linear endomorphisms map** from $\mathcal{U}\to\mathcal{U}$, (ii) operators map**s from $\mathcal{U}\to\mathcal{V}$, and (iii) linear endomorphisms map** from $\mathcal{V}\to\mathcal{V}$ given by $\{B_{k}\}$. BP is given by the transpose of this operation sequence. In PyTomography, operators which map to the same vector space are defined as transforms, while operators that map between different vector spaces are defined as projectors. Example implementations available in PyTomography are listed below:

• Parallel Collimator SPECT: Modeling of primary detections in this imaging modality can be approximated using the “rotate+sum” technique¹⁶, whereby (i) consists of rotations and attenuation/collimator detector response modeling, (ii) consists of summation along an axis, and (iii) consists of adjustments for a camera with a finite field of view. System modeling is approximated using the following system matrix:

  $$H=\sum_{n}v_{n}\otimes\left(\mathcal{T}_{\text{fov}}^{(n)}~{}\mathcal{P}_{x}~{}\mathcal{T}_{\text{cdr}}^{(n)}\mathcal{T}_{\text{att}}^{(n)}~{}\mathcal{T}_{\text{trans}}^{(n)}~{}\mathcal{T}_{\text{rot}}^{(n)}\right)$$ (1)

  where $n$ designates the projection angle, $\mathcal{T}_{\text{rot}}^{(n)}$ (transform) implements a rotation, $\mathcal{T}_{\text{trans}}^{(n)}$ (transform) implements a translation that accounts for non-central projections, $\mathcal{T}_{\text{att}}^{(n)}$ (transform) implements the adjustment for photon attenuation, $\mathcal{T}_{\text{cdr}}^{(n)}$ (transform) implements depth-dependent collimator blurring, $\mathcal{P}_{x}$ (projector) is summation along the $+x$ axis, and $\mathcal{T}_{\text{fov}}^{(n)}$ (transform) applies a mask in projection space that accounts for a finite field of view, $v_{n}$ is a unit vector of length $N$ with 1 in component $n$, and $\otimes$ is the outer product.

• LM PET: Modeling in this modality can be approximated when (i) consists of object-space resolution modeling, (ii) consists of ray tracing between all detected lines of response (LOR), and (iii) consists of normalization/attenuation correction for each LOR. In this case, the system matrix can be written as

  $$H=\mathcal{T}_{\eta}~{}\mathcal{T}_{\omega}~{}\mathcal{P}_{\Sigma}~{}\mathcal{T}_{rr}$$ (2)

  where $\mathcal{T}_{rr}$ (transform) is used for object-space resolution modeling, $\mathcal{P}_{\Sigma}$ (projector) is used to compute line integrals, $\mathcal{T}_{\eta}$ (transform) adjusts for detector normalization, and $\mathcal{T}_{\omega}$ (transform) adjusts for attenuation effects. Due to cancellation in certain reconstruction algorithms, $\mathcal{T}_{\eta}$ and $\mathcal{T}_{\omega}$ need not always be included (though they are required for computing the sensitivity factor).

PyTomography can be used to construct projectors that extend projection space; such functionality is necessary, for example, when modeling multiple photopeaks in SPECT or when using motion correction. In such situations, the extended projection space has dimensionality $(N,...)$ where $...$ is the original dimensionality of projection space, and $N$ represents the number of photopeaks or time gates during acquisition. Mathematically, these projectors can be written as

$$H^{\prime}=\sum_{n}v_{n}\otimes B_{n}H_{n}A_{n}$$ (3)

where $\left\{A_{n}\right\}$ are object-to-object space transforms, $\left\{H_{n}\right\}$ are a sequence of system matrices, $\left\{B_{n}\right\}$ are a sequence of projection-to-projection space transforms, $v_{n}$ is a basis vector of length $N$ with a value of $1$ in component $n$ and $\otimes$ is an outer product. This can be utilized in the following examples:

• Periodic Motion Correction: $A_{n}$ represent spatial transformation operators corresponding to time gate $n$, $H_{n}$ represent system matrices for each motion transform (attenuation modeling may depend on object deformation), and $B_{n}=1$. This maps to an extended projection space $(N,...)$ where $N$ is the number of different time gates (e.g. during a breathing cycle).

• Multiple SPECT Photopeaks: In this scenario, $A_{n}$ are scaling factors that adjust for detector sensitivities and branching ratios, $B_{n}$ are identity map**s, and $H_{n}$ represents the system matrix corresponding to a particular photopeak. The extended projection space has dimensions $(N,...)$ where $N$ is the number of different photopeaks.

II.A.2. Likelihoods

A likelihood function $L(g|\bar{g})$ provides the likelihood of observing data $g$ given expectation $\bar{g}$. Since $\bar{g}$ is approximated via analytical models depending on $f$ and $H$, it follows that $L(g|\bar{g})=L(g|f,H,...)$, where the ellipsis “$...$” contains additional parameters such as a scatter estimate $s$; the ellipsis will assumed implicitly for the remainder of this paper. Many iterative reconstruction algorithms seek to maximize the likelihood function via gradient ascent using the gradient $\nabla_{f}L(g|f,H)$. Likelihood functions for different modalities are shown in Table 1.

Table 1: Likelihoods and corresponding gradients for various medical imaging modalities; $s$ represents a scatter estimate used in SPECT/PET while $b$ represents a scaling factor used in CT.

Likelihood	$L(g|f,H)$	$\nabla_{f}L(g|f,H)$
SPECT/PET	$\sum_{i}\left(g_{i}\log\left[Hf+s\right]_{i}-\left[Hf+s\right]_{i}\right)$	$H^{T}\left(\frac{g}{Hf+s}\right)-H^{T}1$
CT	$\sum_{i}\left(g_{i}\log\left[be^{-Hf}\right]_{i}-\left[be^{-Hf}\right]_{i}\right)$	$H^{T}(be^{-Hf}-g)$
Weighted MSE	$-\frac{1}{2}\sum_{i}\left[W\left(g-Hf\right)^{2}\right]_{i}$	$H^{T}W(g-Hf)$

II.A.3. Reconstruction Algorithms

An image reconstruction algorithm $A$ uses measured data $g$ (i.e. projections) to estimate a corresponding object $f$ that would produce projections $g$ given a system matrix $H$. Statistical algorithms find the image estimate $f$ which maximizes a likelihood function $L(g|H,f)$ characterizing the imaging system. This can be expressed as

$$f=A(L,...)$$ (4)

where the ellipsis “$...$”includes all additional hyperparameters required for the algorithm; the ellipsis will be assumed implicit for the remainder of this paper. The main algorithms in PyTomography are preconditioned gradient ascent (PGA) algorithms; these algorithms aim to maximimize the likelihood via iterative updates that take the form

$$f^{n+1}=f^{n}+C^{n}(f^{n},H_{n})\left[\nabla_{f}L(g^{n}|f^{n},H_{n})-\beta_{n}\nabla_{f}V(f^{n})\right]$$ (5)

where $n$ is the subiteration index, $f^{n}$ is the object estimate of subiteration $n$, $g^{n}$ specifies a subset of data used in subiteration $n$, $H_{n}$ is a system matrix projecting only to the current subset, $C^{n}(f^{n})$ is a preconditioner factor that depends on the object estimate, $V(f^{n})$ is a prior function used to regularize the algorithm, and $\beta_{n}=c_{n}\beta$ is a scaling factor for the prior function (scaled by the fraction of subset data $c_{n}$). These algorithms partition the data $g$ into $M$ approximately equal subsets; a single iteration of an algorithm is defined as the number of subiterations where all subsets have been used once. Various PGA algorithms available in PyTomography are shown in Table 2; they are characterized by their preconditioner factor. Additional reconstruction techniques can be developed using these algorithms. The kernel expectation maximum (KEM), for example, corresponds to (i) reconstructing a coefficient image $\alpha$ using $\alpha=A_{\text{OSEM}}(g,HK)$ where $K$ is a kernel transform and (ii) obtaining the image estimate via $f=K\alpha$.

Table 2: Sample algorithms available in PyTomography: shown are ordered subset expectation maximum (OSEM), ordered subset maximum a posteriori one step late (OSMAPOSL), block sequential regualized expectation maximum (BSREM), and kernelized expectation maximum (KEM).

Algorithm	Preconditioner factor $C^{n}(f^{n},H)$
OSEM20	$f^{n}/\left(H_{n}^{T}1\right)$ and $V(f^{n})=0$
OSMAPOSL21	$f^{n}/\left(H_{n}^{T}1+\nabla_{f}V(f^{n})\right)$
BSREM22	$f^{n}\alpha(n)/\left(\omega_{n}H^{T}1\right)$

Certain algorithm names in the literature correspond to specific realizations of the generic algorithms in Table 2 with particular likelihoods in Table 1. The OS-SART algorithm, for example, corresponds to the use of OSEM with the weighted MSE likelihood (with weights $W=1/(H1)$).

Prior functions $V(f)$ used in reconstruction often take the form

$$V(f)=\sum_{r}\sum_{s}w_{rs}\phi(f_{r},f_{s})$$ (6)

where $w_{rs}$ provides a weighting factor between the locations voxel $r$ and voxel $s$ and $\phi(f_{r},f_{s})$ characterizes a weighting factor between the observed image values at voxel $r$ and voxel $s$. When $w_{rs}$ is non-zero only for neighbouring voxels, the prior is known as a nearest neighbour prior; various nearest neighbour priors implemented in PyTomography are shown in Table 3.

Table 3: Sample nearest neighbour functions available in PyTomography; these priors are characterized by their image weighting function $\phi(f_{r},f_{s})$.

Prior	$\phi(f_{r},f_{s})$
Quadratic	$\left((f_{r}-f_{s})/\delta\right)^{2}/4$
Log-cosh	$\tanh((f_{r}-f_{s})/\delta)$
Relative difference penalty (RDP) 24	$\left((f_{r}-f_{s})^{2}\right)/\left(f_{r}+f_{s}+\gamma|f_{r}-f_{s}|\right)$

While the weighting factor $w_{rs}$ is typically proportional to inverse Euclidean distance, it is also possible to allow it to depend on an additional anatomical image $\chi$ whereby $w_{rs}=w_{rs}(\chi_{r},\chi_{s})$ Since anatomical information is used, this is commonly referred to as an anatomical prior (AP); variations of this technique have recently become popular in clinical practice due to enhanced lesion quantitation and detectability in bone SPECT/CT⁶. In this paper, use of an AP refers to using only the 8 most similar neighbours based on absolute differences of HU values in a CT scan or attenuation map.

PyTomography also offers support for the Deep Image Prior Reconstruction (DIPRecon) algorithm. Details regarding the implementation of this algorithm can be found in Algorithm 2 of Gong et. al⁸. The purpose of this reconstruction technique is to use anatomical images (CT/MR) to enhance functional images (SPECT/PET) through a sequence of neural network overfitting and mixing with OSEM updates. This reconstruction algorithm requires the use of an anatomical image $z$ and a neural network architecture $\mathcal{N}(\cdot;\theta)$ that can be trained to predict a reconstructed functional image $f$ given $z$ as input. This paper explores the use of DIPRecon on an ultra-high resolution PET/MR brain phantom.

II.B. Software Architecture

The key points of the last section are that (i) medical imaging consists of many different modalities that each require unique implementations for system modeling and (ii) tomographic reconstruction algorithms are modality agnostic. PyTomography has been carefully designed to permit arbitrary implementations of system modeling that can interface with the available reconstruction algorithms. The components of PyTomography, designed to facilitate the architecture described, are listed below.

1. Data. The standard data type used to represent images and raw projection data is the tensor class of PyTorch.

2. Metadata. Metadata classes contain all auxiliary information needed for image reconstruction pertaining to a particular imaging system, such as voxel size, spatial dimensions, and information pertaining to the raw data of a particular modality.

3. Transforms. Transform classes are used as utilities to model various features of the imaging process, such as attenuation and image blurring in SPECT. They are commonly used as components in a system matrix. Specifically, transforms are defined by the mathematical property that they are linear endomorphisms.

4. Projectors. Unlike transforms, projectors are mathematical operators that map between different vector spaces. The system matrix, which maps from object space to projection space and represents a model of the imaging system, is an example of a projector. While the input/output datatype to projectors is the PyTorch tensor class, internal code can utilize functionality from other libraries.

5. Likelihoods. Likelihoods specify the likelihood function that characterizes the acquired data. Likelihoods $L(g|H,f)$ are initialized using a system matrix $H$, projection data $g$, and any additional dependencies; once initialized they act as functions of an image estimate $f$.

6. Algorithms. Algorithms are used for image reconstruction, and are designed to interface with all imaging modalities. Statistical iterative algorithms require as input a likelihood instance, and may take as optional arguments prior functions and callbacks.

A visual representation of the software library along with sample code representing a SPECT reconstruction use case is shown in Figure 1. Metadata is first obtained via the raw vendor data stored in the path represented by the variable “file_NM”. The metadata is used as input to a “SPECTSystemMatrix” class, which also takes as input attenuation, PSF, and field of view cutoff transforms (creation of transforms not shown). The generated system matrix is used to build a likelihood instance, which also takes in as input the photopeak (creation not shown) and scatter as an additive term (creation not shown). The likelihood is then used to initialize an OSEM class for reconstruction; a reconstructed object is subsequently obtained using 4 iterations and 8 subset.

Figure 1: Basic overview of the workflow of PyTomography (top) with code representing a specific SPECT reconstruction use case (bottom). Sample classes and implementations are shown; these do not comprise the full set of classes available in the software.

II.C. Examples

Multiple examples are explored to highlight the capabilities of PyTomography. At present, reconstruction of parallel collimator SPECT and LM PET data is demonstrated, but it should be emphasized that these are purely intended as example use cases for particular imaging modalities. All computations were performed using a Microsoft Azure virtual machine (Standard NC6s v3) with a 6 CPUs (Intel(R) Xeon(R) CPU E5-2690 v4 @ 2.60GHz), 112 GB of RAM, and a TeslaV100 GPU. Python scripts and notebooks used to obtain these results can be found at https://github.com/qurit/PyTomography_paper_code.

II.C.1. Monte Carlo SPECT

Two use cases are explored for reconstruction of Monte Carlo generated SPECT image data from the SIMIND Monte Carlo simulation program²⁶. The system matrices used in each case correspond to equation 1, with each employing attenuation modeling and PSF modeling. The triple energy window technique²⁷ is used for scatter correction.

The first use case features reconstructions of digital phantom data representing a ¹⁷⁷Lu-PSMA-617 SPECT scan 38.6 minutes post injection. The digital phantom used for simulation was created using the extended-cardiac torso (XCAT) phantom²⁸, where organ concentrations were obtained using a physiologically-based pharmacokinetic PBPK model²⁹; the model includes major relevant physiological and molecular events and consists of 112 coupled ordinary differential equations. Relevant SIMIND acquisition parameters included 120 projections, a radial distance of $25~{}\text{cm}$, pixel spacing of $3~{}\text{mm}\times 3~{}\text{mm}$, and dimensions of 128x384. Reconstructions were performed for 120 iterations. Organ masks were obtained using bilinear downsampling from ground truth phantoms; voxels with greater than 50% organ volume were included in the mask. Images were reconstructed using each of the OSEM, BSREM, and KEM algorithms; relevant hyperparameters are shown in Table 4. A Callback class was created to track the RCs for each organ of interest at each iteration.

Table 4: Reconstruction parameters used for SPECT reconstruction in this paper. Neighbourhood size corresponds to the kernel size in KEM, and prior neighbourhood for BSREM. Nearest neighbours were obtained from attenuation maps.

Algorithm	OSEM	BSREM	KEM
Subsets	8	8	8
Neighbourhood Size	-	$3\times 3\times 3$	$5\times 5\times 5$
$k$ Nearest Neighbours	-	8	40
Kernel	-	-	Eq. 5 of Vuohijoki et. al. 6
Prior	-	RDP-AP	-

The second use case explores reconstruction of an XCAT phantom similar to the previous example with the following modifications: (i) dynamic breathing was simulated (ii) a lesion with activity $1~{}$MBq was inserted in the liver. The purpose of this use case is to demonstrate the capabilities of MC reconstruction algorithms in PyTomography and examine the impact on the RCs of the lesion. 4 static time points (representative of 4 different amplitudes throughout the breathing cycle) were simulated via the XCAT software corresponding to different periods of the breathing phase. SPECT acquisition in SIMIND was acquired for each of the 4 phases; parameters included 120 projections, a radial distance of $25~{}\text{cm}$, pixel spacing of $3~{}\text{mm}\times 3~{}\text{mm}$, and dimensions of 128x256. Motion correction transforms aligning each phase to a reference phase were obtained via a B-Spline fit on the ground truth attenuation maps using the SimpleITK toolkit ³⁰; motion transforms were converted into displacement vector fields for use in PyTomography. Reconstructed images were obtained using (i) no MC, (ii) separate reconstructions of each time-gate followed by MC and averaging post-reconstruction (iii) MC during reconstruction through use of the modified system matrix given by Equation 3. In each case, images were reconstructed using OSEM with a varying number of iterations and 8 subsets.

II.C.2. Clinical SPECT

This example highlights SPECT reconstruction on publicly available data from the Deep Blue data repository³¹ that was used for the SNMMI ¹⁷⁷Lu dosimetry challenge^{32, 33}. Patients received radiopharmaceutical therapy with ¹⁷⁷Lu-DOTATATE for neuroendocrine tumours (4 cycles of 7.4 GBq/cycle administered every 8 weeks); scans were taken at 4 different time points in the week following a therapeutic dose. Images consisted of 120 projections of shape $256\times 256$ with resolution $2.4~{}\text{mm}\times 2.4~{}\text{mm}$. The system matrix employed attenuation, PSF, and finite FOV modeling. Reconstructions using OSEM, BSREM, and KEM were performed for 50 iterations each; the TEW technique was used to correct for scatter in all cases.

II.C.3. Monte Carlo PET

This example demonstrates (i) the capabilities of LM TOF PET reconstruction on data generated by the GATE Monte Carlo simulation program and (ii) the DIP Recon technique. The simulated scanner consisted of $8$ rings of 338 mm radius with 32.25 mm ring spacing, and a TOF coincidence resolution of 550 ps. For simplicity, data acquisition in the simulation ignored dead-time effects and only considered true primary coincidence events; coincidences due to phantom-scatter and randoms were ignored. The digital phantom used in simulation corresponded to an ultra-high resolution ¹⁸F-FDG PET/ T1-weighted MRI brain image³⁴; the full simulated dataset was representative of a 3 min scan and consisted of $556$ million coincidence events. Data was reconstructed using a $128\times 128\times 96$ matrix size with voxel spacing of $2~{}$mm. The normalization $\eta$ was obtained via simulation of a thin cylindrical shell of radius $318~{}$mm that extended along the axial FOV of the scanner; normalization computation is available in PyTomography for basic polygonal PET geometries. Reconstruction was performed with normalization correction in 3 different ways:

1. The full dataset (556 million events) was reconstructed using OSEM (4it, 20ss) using the EM algorithm

2. 6.3% of the data set (35 million events) was reconstructed using OSEM (80it, 1ss) using the EM algorithm, representative of a low-count scenario.

3. 6.3% of the dataset (35 million events) was reconstructed using the DIPRecon algorithm. The prior image $z$ used was the corresponding T1-weighted MRI image; the initial network prediction $\mathcal{N}(z|\theta_{0})$ was obtained via fitting to the previous low count reconstruction. The neural network was a UNet³⁵ with similar architecture to the original DIP paper ⁸; more information on the network structure and reconstruction hyperparameters can be found in the publicly available code.

III. Results

The time required for reconstruction in the first SPECT use case was 412.8 s (3.44 s/iteration) for OSEM, 434.8 s (3.62 s/iteration) for BSREM, and 601.8 s (5.02 s/iteration) for KEM; sample reconstructions with corresponding RC curves are shown for each algorithm in Figure 2. BSREM clearly produced the least noisy images, and had the best RCs for the lungs, liver, kidneys, and bladder. KEM produced an image estimate with an intermediate level of noise, and had the best recovery coefficient for the salivary glands. OSEM yielded the noisiest images, and had the worst recovery coefficients in all regions except the salivary glands. Despite the high performance of BSREM, it was unable to accurately predict the regions of low uptake within the liver visible in the ground truth.

Figure 2: Reconstructions of full body ¹⁷⁷Lu-PSMA-617 Monte Carlo projection data; ground truth phantoms were obtained using the XCAT software and SPECT simulation was achieved using the SIMIND Monte Carlo program. Left: Central coronal slices corresponding to truth, OSEM (120it/8ss), BSREM (120it/8ss), and KEM (120it/8ss). Right: Recovery coefficient vs. iteration number for sample organs; line widths are representative of error estimates based on reconstructions of 5 independent noise realizations.

Sample reconstructions and RC curves for the SPECT motion correction use case are shown in Figure 3. The lesions appear sharper when motion correction is used; motion correction applied during reconstruction via Equation 3 produced lesions of higher uptake than when correction was applied post-reconstruction. This is quantitatively observed in the RC curves of Figure 3, where motion correction applied during reconstruction tends to produce the highest RCs throughout all iterations. After 100 iterations, the RCs were larger when MC was used during reconstruction than when it was used post reconstruction ($p<10^{-6}$).

Figure 3: Reconstructions of full body ¹⁷⁷Lu-PSMA-617 Monte Carlo projection data with breathing motion simulated; ground truth phantoms were obtained using the XCAT software and SPECT simulation was achieved using the SIMIND Monte Carlo program. Reconstruction was performed with OSEM (8ss) up to 100 iterations using attenuation/PSF modeling and TEW scatter correction; images shown correspond to reconstruction after 20 iterations. From left to right: ground truth phantom, reconstructed image without MC, reconstructed image with MC applied after reconstruction, reconstructed image with MC applied during reconstruction, and lesion recovery coefficients as a function of iteration number. Green boxes are drawn around the lesion in the liver.

The time required to reconstruct the clinical SPECT patient data was 464.7s (9.29 s/iteration) for OSEM, 453.1s (9.06 s/iteration) for BSREM, and 1825s (36.50 s/iteration) for KEM. Central coronal slices are shown in Figure 4; the patient had a region of high uptake value in the liver, indicative of a liver lesion. BSREM and KEM appear to confine activity within the boundary of the kidneys, with BSREM exhibiting slightly more of this behaviour. In addition, OSEM produced the most observed noise in the liver. A common artifact of both BSREM and KEM, however, is the noise in the liver lesion; use of anatomical nearest neighbours may cause the lesion activity noise profile to resemble the CT noise profile. The mean predicted counts in the kidneys were 1.98 (OSEM), 2.07 (BSREM), and 2.02 (KEM); these relative proportions are consistent with the RCs of Figure 2, suggesting that BSREM may yield the most accurate recovery coefficients for the kidneys.

Figure 4: Coronal slices of activity concentration (colored) and Hounsfield Unit values (greyscale) reconstructed using OSEM (50it/8ss), BSREM (50it/8ss), and KEM (50it/8ss). Data corresponds to the first time point of patient 4 from the “Lu-177 DOTATATE Anonymized Patient Datasets: Multi-Time Point Lu-177 SPECT/CT Scans” dataset³¹.

Reconstructing the LM TOF PET GATE output data required multiple steps. While initial processing of the ROOT listmode data is time consuming, PyTomography can save this data in an accessible format for reuse.

• Opening the ROOT list mode data ($\sim$ 50 GB) of the normalization scan and computation of the normalization factor $\eta$ (approximately 50 minutes). When the normalization factor is saved, it requires approximately 5 s to read.

• Opening the ROOT list mode data ($\sim$ 74 GB) of the phantom scan (approximately 1 hour). When saved in an accessible format, reading requires approximately 5 s.

• Initial configuration of the system matrix ($7.8$ s).

• Reconstruction of the PET listmode data. The high count scenario using OSEM (4it 20ss) required 185.6 s (46.4 s per iteration), the low count scenario using OSEM (80it 1ss) required 201.0 (2.5 s per iteration), and the low count scenario using DIPRecon required 3135.6 s (31.3 s per iteration)

Corresponding phantom reconstructions are shown in Figure 5; bias and noise in grey and white matter is shown as a function of iteration number in Figure 6. Due to finite resolution of the imaging system causing activity “leakage”, activity quantification in the grey matter region was consistently underestimated while activity quantification in the white matter region was consistently overestimated. The DIPRecon on the low count data had similar noise levels in grey matter to OSEM on the high count data, demonstrated both qualitatively and quantitatively in each Figures 5 and 6 respectively.

Figure 5: PET reconstruction of GATE simulated data corresponding to a ultra-high resolution ¹⁸F-FDG PET/ T1-weighted MRI brain image³⁴. From top to bottom: axial, coronal, and sagittal slices. From left to right: ground truth MRI (non contrast adjusted), ground truth PET uptake, high count OSEM reconstruction (20it/4ss), low count OSEM reconstruction (80it/1ss), and low count DIP reconstruction.

Figure 6: Variation in absolute bias and standard deviation throughout iterative reconstruction corresponding to reconstructed images of Figure 5. Subplots correspond to the grey matter region (left) and white matter region (right). For the low count rate scenario, points are plotted every 10 iterations. For the low count rate scenario with OSEM, the first point corresponds to the 10th iteration. The bias was negative in grey matter and positive in white matter.

IV. Discussion

This technical report presented the software architecture of PyTomography, and showed use cases on both SPECT and PET data. The short times required for reconstruction using this tool can enable researchers to perform extensive studies on phantom and patient data that may include multiple patients, noise realizations, and reconstruction algorithms. Validation of PyTomography against alternative reconstruction libraries is shown in the appendix.

Example applications in ¹⁷⁷Lu-SPECT imaging, which are relevant for currently FDA approved radiopharmaceutical therapies, explored the use of the OSEM, BSREM, and KEM, reconstruction algorithms. It should be emphasized that the purpose of this technical report was not to rank the different algorithms, but rather to demonstrate the capabilities of the software library. While some evidence was shown to favour BSREM over OSEM and KEM for kidney and liver dosimetry based on better recovery coefficients (Figure 2), a truly comprehensive study that aims to quantitatively compare algorithms should involve (i) multiple XCAT activity and anatomical configurations, (ii) inclusion of patient motion, (iii) effect of misalignment between SPECT and CT images, (iv) different hyperparameters in each reconstruction algorithm and (iv) should seek to use real patient data to validate any phantom studies. In addition, recovery coefficients alone are not sufficient to rank algorithms. Studies should also consider source-to-background ratios as well as error analysis based on intra- and inter-patient variabilities, and clinical task performance. While the presented software framework enables such studies to be performed, it is out of scope for this publication.

While the RDP-AP prior was featured heavily in this study, the framework of PyTomography is designed to foster the development and validation of other novel prior functions as well. In addition, while KEM was featured using anatomically-based basis functions, the library permits use of any external image for construction of the basis functions. Different priors and basis functions may also have significant implications for (i) quantitative PET/SPECT imaging in dosimetry, and (ii) qualitative observer based studies in lesion detection and classification.

The motion correction example considered a simplified case of 4 discrete phases where motion correction transforms were obtained using ground truth attenuation maps. In practice, however, breathing motion is continuous, and motion correction transforms will need to be derived using alternative source of data. The scope of this technical report is merely to demonstrate that PyTomography contains the functionality for incorporating motion correction.

Example applications in ¹⁸F-FDG PET imaging demonstrated the capabilities of LM reconstruction using the OSEM and DIPRecon algorithms. As demonstrated qualitatively in Figure 5 and quantitatively in Figure 6, reconstruction of the low count data using DIPRecon was able to reduce noise in the grey matter regions while preserving cortical structures. Once again, the purpose of this paper is merely to demonstrate the capabilities of PET LM reconstruction and use of the DIPRecon algorithm provided in the software library; extensive research on the validity of the DIPRecon method should seek to explore, for example, the effects of different neural network architectures, hyperparameters, and misalignment between the PET/MRI images.

While this paper has demonstrated implementations for parallel collimator SPECT and LM TOF PET, there exist many other modalities not presently included in the software, including (but not limited to) diverging/converging/pinhole SPECT, sinogram-based PET reconstruction, and various forms of computed tomography (CT) and magnetic resonance (MR) imaging. As displayed in Figure 1, projectors for different imaging modalities can be implemented using a variety of python based tools: it is hoped that the structure and flexibility of PyTomography will encourage contributions from various research groups interested in particular imaging systems. Since the library is open source, newly developed functionality can be easily tested and verified by many independent research groups with their own data. The immediate goals for future development in PyTomography at the time of publication include:

1. Fast PSF modeling for high energy SPECT isotopes, such as ¹³¹I and ²²⁵Ac.

2. Incorporation of scatter and random estimation in PET reconstruction.

3. Sinogram-based PET reconstructions.

4. Development of a 3D Slicer³⁶ extension.

In summary, we hope that we have made a compelling case for the benefit that this tool can bring to the medical imaging community. It can be used as a framework for studies that require fast reconstruction times, such as those that explore a wide range of reconstruction hyperparameters. It is well validated and may be used for accurate and reproducible quantitative imaging in both PET and SPECT; we particularly envision applications in the field of theranostics. A variety of developments will assist PyTomography in reaching the full potential of precision medicine. We thus highly encourage and welcome open-source contributions to assist in the items listed above; those who wish to contribute are encouraged to read the developers’ guide on the documentation website.

V. Conclusion

This work describes the python library PyTomography and highlights specific use cases in SPECT and PET imaging. The software architecture facilitates the development of different imaging modalities that can all interface with the same reconstruction algorithms. The purpose of PyTomography is to create a well-documented and computationally-efficient library for medical image reconstruction, where novel system modeling and reconstruction techniques are implemented, shared, and explored by experts in the community.

VI. Appendix

VI.A. Relevant Links

Links to the various webpages of PyTomography are contained in Table 5: GitHub contains the source code, Readthedocs contains all relevant documentation, PyPI is the host of the built source files, and Discourse is a community forum for users and contributors.

Table 5: Relevant Webpages

Website	Link
Github	https://github.com/qurit/PyTomography
Readthedocs	https://pytomography.readthedocs.io/en/latest/
PyPI	https://pypi.org/project/pytomography/
Discourse	https://pytomography.discourse.group/

VI.B. Validation of PyTomography

This section contains two examples used to validate PyTomography against other reconstruction software:

1. Monte Carlo SPECT Validation With STIR: Radioactivity concentrations in a digital phantom were selected to correspond to a 24-hour post injection prostate cancer patient (1700 MBq total activity) imaged via the prostate-specific membrane antigen (PSMA)-targeting radiopharmaceutical, ¹⁷⁷Lu-PSMA-617. Simulated data was reconstructed with OSEM (2 iterations, 8 subsets) using (i) STIR and (ii) PyTomography. Coronal slices and 1D profiles of reconstructions are shown in Figure 7 and are nearly identical. The image noise, quantified by the standard deviation of predicted counts in the liver, was 0.789 counts (STIR) and 0.793 counts (PyTomography). The mean predicted counts in the kidneys were 1.764 (STIR) and 1.761 (PyTomography). The reconstruction time in PyTomography (5.4 s) was significantly faster than the time required in STIR (51542.2 s), owing to the speed-up factor of GPU implementation.

2. Clinical SPECT Validation With Vendor: A Jaszczak phantom was filled with Lu-177 and scanned at 180 projections ($128\times 128$ with a resolution of $4.8\text{mm}\times 4.8\text{mm}$) using a Siemens Symbia T Series SPECT/CT scanner with Medium Energy collimators. Reconstruction was performed with OSEM (4 iterations, 10 subsets) using (i) the manufacturer scanner software and (ii) PyTomography. Reconstructions and corresponding 1 dimensional profiles are shown in Figure 8; they appear similar. 3D slicer was used to obtain sample segmentations of the hot and cold regions; the mean source-to-background ratios were 20.97 (vendor) and 21.76 (PyTomography). The image noise, quantified by the standard deviation of predicted counts in the cold region, were 0.217 counts (vendor) and 0.178 counts (PyTomography).

3. Monte Carlo PET Validation With CASToR: The ultra-high resolution ¹⁸F-FDG PET/ T1-weighted MRI brain image³⁴ used in the PET example (556 million events) was reconstructed using LM TOF OSEM (2 iterations, 21 subsets) with both PyTomography and CASToR. Resulting reconstructions are shown in Figure 9. The mean activity ratio in grey to white matter was 2.05 (PyTomography) and 2.03 (CASToR). The noise in the image was estimated as the standard deviation of counts in white matter divided by the mean activity in white matter; the values were 0.329 (PyTomography) and 0.321 (CASToR). Apart from processing the ROOT files and building the sensitivity image, the time required for reconstruction in PyTomography was 14 s to build the system matrix and required 59 s per iteration for reconstruction. CASToR reconstruction was ran on a separate HP Z8 Computer Desktop with 2 Intel Xeon Gold 6136 CPUs at 3.00 GHz with a total of 48 CPU cores. The reconstruction was performed with 24 CPU cores and reconstruction required approximately 455 s per full iteration.

The images shown in this section are displayed using the “nipy_spectral” colormap of matplotlib because it has significant contrast. Since this colormap is neither perceptually uniform nor monotonically increasing in lightness, images in this section should be viewed in color.

Figure 7: Central coronal slices (left) and 1D profiles (right) of reconstructed SIMIND SPECT projection data using OSEM (2it/8ss) with attenuation/PSF modeling; images were reconstructed using STIR and PyTomography. 1D profiles correspond to the horizontal lines shown on the coronal slices.

Figure 8: Sample axial slices (top) and 1D profiles (bottom) of a ¹⁷⁷Lu Jaszczak phantom reconstructed using OSEM (4it/10ss) with attenuation/PSF modeling and triple energy scatter correction. Reconstructions were performed using (i) the scanner software, and (ii) PyTomography. 1D profiles correspond to the horizontal lines shown on the axial slices.

Figure 9: Sample coronal slices (top) and 1D profiles (bottom) of the high resolution brain phantom reconstructed using CASToR and PyTomography with LM TOF OSEM (2it/21ss). The two images were normalized such that the sum of their voxels was identical. 1D profiles correspond to the horizontal lines shown on the coronal slices

References