Self-Supervised k-Space Regularization
for Motion-Resolved Abdominal MRI Using Neural Implicit k-Space Representations

Abstract

Neural implicit k-space representations have shown promising results for dynamic MRI at high temporal resolutions. Yet, their exclusive training in k-space limits the application of common image regularization methods to improve the final reconstruction. In this work, we introduce the concept of parallel imaging-inspired self-consistency (PISCO), which we incorporate as novel self-supervised k-space regularization enforcing a consistent neighborhood relationship. At no additional data cost, the proposed regularization significantly improves neural implicit k-space reconstructions on simulated data. Abdominal in-vivo reconstructions using PISCO result in enhanced spatio-temporal image quality compared to state-of-the-art methods. Code is available at https://github.com/vjspi/PISCO-NIK.

1 Introduction

Motion, i.e. induced by respiration in the abdomen, results in non-negligible artefacts in magnetic resonance imaging (MRI). Therefore, motion-sensitive applications such as radiation therapy planning [18] or free-breathing high-resolution diagnostic imaging [22] rely on dynamic MRI reconstruction techniques. These techniques reconstruct temporally resolved images by binning the data into different respiratory motion states (MS). However, reconstructing multiple MS inherently results in a reduction of data per temporal image, leading to undersampling artifacts due to violation of the Nyquist theorem. Common approaches address this by applying regularization across the temporal dimension to exploit redundancies [6, 19]. Yet, the total number of MS is limited, resulting in motion blurring due to low temporal resolution.

Recently, neural implicit k-space representations (NIK) have shown promising results for blurring-free motion-resolved reconstructions [17, 9]. Based on the acquired k-space trajectory and a surrogate signal for the current motion state, a multi-layer perceptron (MLP) is trained to predict the k-space signal for a given spatio-temporal input. At inference, any coordinate can be sampled, allowing for flexible spatial sampling patterns at high temporal resolutions. Exclusive training in k-space eliminates the need for costly domain transforms such as non-uniform Fast Fourier Transformations (NUFFT) within each iteration.

Similar to other accelerated reconstruction methods [2, 7], NIK learns to fill missing data points within k-space. Yet, without any kind of regularization, the reconstruction model may be prone to overfitting and noisy reconstructions. General learning-based MRI reconstruction methods counteract overfitting by including a regularization, usually enforced on the estimated image [1, 8, 10]. However, image-based regularization methods are impractical for NIK, as they necessitate sampling of the entire k-space within each training iteration to obtain the image that requires regularization. While k-space-based regularization would be effective, translation of image-based constraints to k-space is not trivial.

Exploring the parallel imaging concept of Generalized Autocalibrating Partially Parallel Acquisitions (GRAPPA) [7] reveals a potential spatial neighborhood relationship within k-space itself. This relationship needs to be estimated on a fully-sampled calibration set to be subsequently applied to undersampled k-space regions. Learning-based reconstruction methods already utilize this neighborhood relationship for k-space refinement [12, 16]. However, similar to GRAPPA, they first require explicit determination of the k-space relationship This is impractical for motion-resolved imaging, since calibration needs to be acquired and conducted for every MS.

In this work, we reformulate the concept of k-space relationship independent of calibration data. We exploit the inherent global k-space relationship without the need for explicit determination. Our contributions are three-fold:

1. We introduce PISCO: a novel Parallel Imaging-inspired Self-Consistency for self-supervised k-space refinement that operates independently of any additional training or calibration data.

2. Incorporating the concept of PISCO, we present the first k-space-based regularization loss for neural implicit k-space representations (NIK).

3. Based on a realistic motion simulation as well as in-vivo data, we quantitatively and qualitatively demonstrate the potential of PISCO for motion-resolved abdominal MR reconstruction using NIK.

2 Methods

2.1 k-Space Interpolation using GRAPPA

In MRI, a k-space signal $y=\{y_{i}\in\mathbbm{C}^{N_{c}}|i=1,...,N_{x}\cdot N_{y}\}$ is acquired with $N_{c}$ coils at spatial k-space coordinates $k=\{k_{i}\in\mathbbm{R}^{2}|i=1,...,N_{x}\cdot N_{y}\}$ (no time dimension for simplicity). The desired image ${x}\in\mathbbm{C}^{N_{x}\cdot N_{y}}$ is then reconstructed by solving the inverse problem $y=\mathrm{A}x$, where the forward operator $\mathrm{A}=\mathcal{F}\mathbf{S}$ consists of the coil sensitivity maps $\mathbf{S}\in\mathbbm{C}^{N_{x}\cdot N_{y}}$ and Fourier transform $\mathcal{F}$. For acceleration purposes the acquired k-space is often undersampled, i.e., $y$ is acquired at fewer $k_{i}$ than required to fulfill the Nyquist theorem. To avoid undersampling artefacts, parallel imaging methods such as GRAPPA [7] leverage the multi-coil nature of MRI data to derive missing values from its neighboring k-space values.

For GRAPPA, a patch of $N_{n}$ neighboring coordinates is sampled around a missing target $k^{T}_{i}\in\mathbbm{R}^{2}$, resulting in a coordinate patch $k^{P}_{i}\in\mathbbm{R}^{N_{n}\cdot{2}}$. The target signal value $y^{T}_{i}\in\mathbbm{C}^{N_{c}}$ is then obtained by linearly combining the neighboring signal values $y^{P}_{i}\in\mathbbm{C}^{N_{n}\cdot N_{c}}$ (Fig. 1A). Stacking $N_{m}$ pairs of targets and patches, the linear combination can be written as $\mathrm{T}=\mathrm{P}\mathrm{W}$, where $\mathrm{T}=[y^{T}_{1},...,y^{T}_{N_{m}}]\in\mathbbm{C}^{[N_{m}\times N_{c}]}$, $\mathrm{P}=[y^{P}_{1},...,y^{P}_{N_{m}}]\in\mathbbm{C}^{[N_{m}\times N_{n}\cdot N_{c}]}$ and $\mathrm{W}\in\mathbbm{C}^{[N_{n}\cdot N_{c}\times N_{c}]}$ is the global weight matrix with a total of $N_{w}=N_{n}\cdot N_{c}\cdot N_{c}$ weights. To determine $\mathrm{W}$, GRAPPA splits the signal space $y$ into a fully sampled auto-calibration signal $y_{\mathrm{ACS}}$ and the remaining undersampled k-space $y_{\mathrm{US}}$. Within the fully sampled $y_{\mathrm{ACS}}$, pairs of $\mathrm{T_{\mathrm{ACS}}}$ and $\mathrm{P_{\mathrm{ACS}}}$ can be used to solve the least-squares problem:

$$\mathrm{W_{ACS}}=\arg\min_{\mathrm{W}}\lVert\mathrm{P_{ACS}}\mathrm{W}-\mathrm{T_{ACS}}\rVert_{2}^{2}+\alpha\lVert\mathrm{W}\rVert_{2}^{2}\text{\quad s.t. }\mathrm{T}_{ACS},\mathrm{P}_{ACS}\subseteq y_{ACS}$$ (1)

where $\lVert\cdot\rVert_{2}^{2}$ applies the L2-norm element-wise and $\alpha$ weighs the Tikhonov regularization. Subsequently, undersampled points $y^{T}_{US}$ are derived using the calibrated weights to compute $y^{T}_{US}=\mathrm{W_{ACS}}\cdot y^{P}_{US}$, leading to improved reconstructions [7].

2.2 From GRAPPA to PISCO

Figure 1: Overview. A: Multiple pairs of targets and surrounding neighbors $y_{i}^{T},y_{i}^{P}$ are sampled and randomly sorted into subsets and solved for the linear relationship $W_{s}$ (Eq. 2). PISCO aims to minimize the distance between all $\mathrm{W_{s}}$. For simplicity, the coil dimension $N_{c}$ is not visualized, but included in matrix dimensions. B: PISCO for regularization of NIK. Any points can be sampled, but $L_{DC}$ can only be compared to measured k-space $y_{meas}$ (gray lines). $L_{PISCO}$ refines independent of $y_{meas}$.

Acquiring a fully sampled region $y_{ACS}$ to calibrate a weight set $\mathrm{W}$ is not always feasible, particularly in dynamic imaging. Hence, we reframe GRAPPA’s concept of global spatial k-space relationship to a calibration-free condition based on the following assumption: If a weight set $\mathrm{W}_{\mathrm{ACS}}$ calibrated on $y_{\mathrm{ACS}}\in y$ models the linear relationship for the whole k-space $y$, then a weight set $\mathrm{W}_{\mathrm{s}}$ derived from a random subset $y_{\mathrm{s}}\in y$ should result in the same linear relationship. Consequently, weight sets $\mathrm{W}_{\mathrm{s}}$ determined from multiple random subsets $\{y_{\mathrm{s}}\in y|s=1,...,N_{s}\}$ are expected to converge to the same solution, i.e. $\mathrm{W}_{i}=\mathrm{W}_{j},\forall i\neq j$. Thus, we propose to refine an interpolated k-space without access to a calibration set by enforcing Parallel Imaging-inspired Self-Consistency (PISCO) as follows:

$$\mathrm{W}_{s}=\arg\min_{\mathrm{W}}\lVert\mathrm{P}_{s}\mathrm{W}-\mathrm{T}_{s}\rVert^{2}_{2}+\alpha\lVert\mathrm{W}\rVert^{2}_{2}\text{\quad s.t. }\mathrm{T}_{s},\mathrm{P}_{s}\subseteq y_{s}$$ (2)

$$\textbf{PISCO: }\sum_{i=1}^{N_{s}}\sum_{j=1}^{N_{s}}\text{distance}(\mathrm{W}_{i},\mathrm{W}_{j})\xrightarrow{!}0$$ (3)

2.3 Dynamic MRI Reconstruction using NIK with PISCO

In dynamic MRI reconstruction, the image reconstruction problem from Sec. 2.1 is expanded by a temporal dimension $t$, i.e. k-space coordinates $k$ consist of $k_{i}=[k_{x},k_{y},t]$ and result in multiple temporal images $x=\{x_{t}|t=1,2,...,N_{t}\}$. NIK learns a map** $G_{\theta}:k\xrightarrow{}y$ based on acquired data points [9]. For abdominal reconstruction, $t$ can be derived from a self-navigator reflecting the current respiratory state [17]. At inference, a fully-sampled spatiotemporal k-space $\tilde{k}$ can be queried to reconstruct the temporal images ${x}=\mathrm{A}G_{\theta}(\tilde{k})$.

Within each training iteration of NIK, a coordinate batch $k_{B}=\{{k_{i}}\}^{N_{b}}_{i=1}$ is sampled from acquired coordinates $k_{meas}$ to predict $y_{B}^{pred}=G_{\theta}(k_{B})$(Fig. 1B). Then, the model is optimized with a data consistency loss $\mathcal{L}_{DC}$ comparing the predicted $y_{B}^{pred}$ and measured signal $y_{B}^{meas}$. This limits NIK’s training strategy to actually acquired points. By adding the concept of PISCO during training we enable further self-supervision to coordinates that have not been acquired, which is especially beneficial for addressing large gaps in the outer regions of k-space in radial trajectories. As in Sec. 2.1, we assign $k_{B}$ as targets $k_{B}^{T}$ and sample spatial neighbors $k_{B}^{P}$. To account for the dynamic nature, we sort the coordinates along $t$ to allow for separation into $N_{s}$ temporally adjacent subsets [3] with $N_{m}$ coordinates each. The resulting $\mathrm{T}_{s}$ and $\mathrm{P}_{s}$ are used to solve for $[\mathrm{W}_{s}]_{s=1}^{N_{s}}$ (Eq. 2). To fulfill Eq. 3, we introduce the PISCO loss:

$$\mathcal{L}_{PISCO}=\frac{1}{N_{s}^{2}}\sum_{i=1}^{n_{s}}\sum_{j=1}^{n_{s}}\mathcal{L}_{dist}(\mathrm{W}_{i},\mathrm{W}_{j})$$ (4)

where $\mathcal{L}_{dist}$ is chosen to be the complex difference $\mathcal{L}_{\mathbbm{C}}^{1}=||\mathit{Re}(W_{i}-W_{j})||_{1}+||\mathit{Im}(W_{i}-W_{j})||_{1}$. After pre-training NIK for $E_{pre}$ epochs, the self-supervised consistency is enforced by optimizing the model’s weights $\theta$ with $\mathcal{L}_{DC}$ and $\mathcal{L}_{PISCO}$ in an alternating manner, with the latter’s impact weighted by $\lambda$. Additional robustness against outliers in T and P is enhanced with Thikonov regularization weight $\alpha$ when solving for $W$. Moreover, the number of samples per equation system is increased to $N_{m}=f_{od}\cdot N_{w}$, with $f_{od}>1$ determining the ratio of unknowns $N_{w}$ to available equations $N_{m}$. The integration of $\mathcal{L}_{PISCO}$ in NIK’s training procedure is summarized as pseudo-code in Alg. 1.

Algorithm 1 Neural Implicit k-Space Representation with PISCO

1:Acquired k-space coordinates $k^{meas}$ and signal values $y^{meas}$, total epochs $E$, pretraining epochs $E_{pre}$, NIK architeture $G_{\theta}$, initial model parameters $\theta_{0}$

2:for $e=0$ to $E$ do

3:     for $\text{batch ($k_{B},y_{B}$) in }(k^{meas},y^{meas}$) do $\triangleright$ Sample batch of coordinates

4:         $y_{B}^{pred}\leftarrow G_{\theta_{e}}(k_{B})$ $\triangleright$ Predict k-space with NIK

5:         $\theta_{e+1}\leftarrow L_{{DC}}(y_{B}^{pred},y_{B})$ $\triangleright$ Update model with data consistency loss

6:         if $e>E_{\text{pre}}$ then $\triangleright$ Apply PISCO regularization

7:              $k_{B}^{T},k_{B}^{T}\leftarrow k_{B},\text{patches around }k_{B}$ $\triangleright$ Sample coordinate patches

8:              $y_{B}^{T},y_{B}^{P}\leftarrow G_{\theta_{e}}(k_{B}^{T}),G_{\theta_{e}}(k_{B}^{P})$

9:              $[P_{s}]_{s=1}^{N_{s}},[T_{s}]_{s=1}^{N_{s}}\leftarrow$ divide ${y_{B}^{T},y_{B}^{P}}$ into multiple subsets $\triangleright$ Sec. 2.2

10:              $[W_{s}]_{s=1}^{N_{s}}\leftarrow$ solve each subset for weights with regularization $\alpha$ $\triangleright$ Eq. 2

11:              $\theta_{e+1}\leftarrow\lambda\cdot L_{{PISCO}}([W_{s}]_{s=1}^{N_{s}})$ $\triangleright$ Eq. 4               

12:return Learned NIK model $G_{\theta}$

3 Experimental Setup

3.1 Data

Due to the lack of ground-truth data for motion-resolved imaging [16], we create a dynamic free-breathing simulation using the XCAT phantom [13]. We generate water/fat/susceptibility maps [5, 11] for 100 time points $t_{MS}$ within one breathing cycle and simulate complex images $\textbf{x}_{t}$ based on a water-fat model [21] (echo time $T_{e}$=1.4ms). After applying six simulated coil sensitivity maps, radial motion-free multi-coil k-space data $Y_{mf}$ is obtained using NUFFT for acceleration factors R=1,2,3. Then, $Y_{mf}$ is merged into one motion-affected k-space $Y_{ma}$ based on a respiratory navigator $t$ extracted from the lung-liver edge.

For in-vivo validation, datasets were acquired using a pseudo golden angle stack-of-star trajectory (voxel size=1.5x1.5x3mm³, flip angle=10°) at 3T (Ingenia Elition X, Philips Healthcare) after local ethics committee approval. We obtained a dynamic free-breathing dataset of the abdomen, as well as a respiratory-gated acquisition from the same subject as reference. To validate the regularization independent of motion, a static dataset of the thigh region was acquired and retrospectively accelerated (R=1,2,3) . Each k-space consists of 26 coils ($N_{C}$), 536 frequency encoding steps ($N_{FE}$), 1341/537/537 spokes ($N_{PE}$) for dynamic/gated/static. Coil sensitivities are estimated with ESPIRiT [20].

3.2 Training and Inference

We adapt NIK’s [9] architecture using 4 layers, 512 hidden features, high-dynamic range loss as $\mathcal{L_{DC}}$, SIREN activations [15], batch size of 10k and use STIFF feature encoding [4]. We correct the respiratory signal $t$ for linear drifts and rescale it to [0,0.5]. For PISCO-NIK, $L_{\text{PISCO}}$ is included after $E_{pre}=200$ and weighted to match the magnitude of $L_{\text{DC}}$, i.e., $\lambda$=0.01/0.1 for simulation/in-vivo data. For the neighbors P, a kernel of size [3,3] with $\delta x/\delta y=N_{FE}^{-1}$ is sampled around the target T resulting in $N_{n}$ = 8, as shown in Fig. 1. Hence, $N_{w}$ for the phantom is 288 $(8\cdot 6\cdot 6)$ and for the subject would be 5408 $(8\cdot 26\cdot 26)$. To increase the number of possible sets for the subject ($N_{s}\propto N_{w}^{-1}$), we reduce the number of output coils solved for in each iteration, thereby lowering $N_{w}$ to 624 $(8\cdot 26\cdot 3)$. Weight solving for $L_{\text{PISCO}}$ is regularized with $\alpha$=1e-4 and $f_{OD}$=1.1. Both losses are optimized with separate Adam optimizers (lr=3e-5). All models are run for 1000 epochs total (NVIDIA RTX A6000, using Python 3.10.1/PyTorch 1.13.1).

3.3 Evaluation

Dynamic reconstructions using $L_{\text{PISCO}}$ (PISCO-NIK) are compared to vanilla NIK (same hyperparameters, only $L_{\text{DC}}$), inverse NUFFT for 1 or 4MS (INUFFT/INUFFT4) and state-of-the-art motion-resolved XD-GRASP [6] using the standard 4 MS as well as 50 MS to match the temporal resolution of NIK (XD-GRASP4 and XD-GRASP50). Quantitative metrics are computed in reference to the motion-resolved $k_{mf}$ (simulation) and INUFFT-R1 (static). Calculated metrics include peak signal-to-noise ratio (PSNR) and spatial feature similarity (FSIM) of $xy$-reconstructions at 50 time points. To evaluate temporal performance, FSIM-t is calculated for the simulation on $xt/yt$ with $y/x$ fixed.

4 Results

Figure 2: Quantitative results for 20 motion-affected simulation slices accelerated by R=1,2,3. All comparisons, except those marked with ”N”, are statistically significant (Wilcoxon signed rank test with False Discovery Rate correction at p$<$0.05).

Figure 3: Static in-vivo reconstructions (thighs) for accelerations R=1,2,3. Exclusion of the temporal component enables comparison to INUFFT-R1, showing increased PSNR using PISCO regularization. PISCO-NIK sharpens reconstructions (blue arrows and increased FSIM) and reduces ringing artefacts of NIK-R2 (green arrows).

Quantitative reconstruction results of 20 slices of the simulation for R=1,2,3 are shown in Fig. 2. Videos of dynamic reconstructions are included in Suppl. Fig. 1. Both neural implicit representations, NIK and PISCO-NIK, clearly outperform XD-GRASP50 (i.e. with same temporal resolution). XD-GRASP4 results in higher PSNR due to lower temporal resolution, but significantly lower spatial FSIM due to residual blurring. Particularly with increasing acceleration (R=2/R=3), PISCO-NIK results in an significant increase of PSNR (up to 1.1dB), FSIM and FSIM-t (up to 0.01/0.02).

Figure 4: Dynamic in-vivo reconstructions for two abdominal slices, with spatial $xy$-image (fixed $t$) and temporal $yt$-image (fixed $x$ at white dotted line). Complete reconstruction videos in Suppl. Fig. 2. Temporally resolved reconstructions are not available for the gated reference and limited for INUFFT4/XD-GRASP4 (blue arrows). XD-GRASP50 results in streaking, but more precise vessel structure, likely due to less motion blurring (white arrow). NIK enables improved temporal resolution, whereas PISCO additionally temporally smoothens (blue arrow) while maintaining spatial sharpness (white arrow).

Static reconstruction results are shown in Fig. 3. Similar to the simulation, including PISCO results in higher PSNR and leads to sharper results, specifically with increasing acceleration factors. Note that for the static case all data are considered as one MS, therefore naturally resulting in lower undersampling for INUFFT compared to the dynamic case with multiple MS. In Fig. 4, motion-resolved reconstructions of the dynamic in-vivo data demonstrate the benefit of neural implicit representations regarding temporal resolution ($yt$ images of NIK and PISCO-NIK, also visible in videos in Suppl. Fig. 2). Since no ground truth reconstruction is available for dynamic imaging, an unpaired gated scan is included as comparison. It results in sharp spatial images, but no temporal resolution. XD-GRASP4 smoothens results compared to INUFFT4, but residual motion blurring and discrete jumps in the temporal dimension persist (blue arrow). XD-GRASP50 encounters these visible jumps, but results in streaking. NIK allows for high temporally resolved but noisy reconstructions and the proposed PISCO regularization enforces smoother results with sharper vessel structures.

5 Discussion and Conclusion

In this work, we have presented the novel concept of parallel imaging-inspired self-consistency (PISCO), which leverages global k-space relationship without the use of calibration data to explicitly determine it. We have shown the seamless integration of PISCO as self-supervised k-space regularization in NIK-based reconstructions, eliminating the need for additional data or training steps. Both simulated and in-vivo dynamic reconstruction results illustrate that PISCO enhances spatial and temporal denoising while preserving temporal resolution and sharp vessel structures. Additionally, we have verified PISCO’s spatial denoising potential on a static example, highlighting its capacity to learn improved neural representations resulting in enhanced image quality.

Generally, NIK’s flexibility in sampling and PISCO’s calibration-free nature allow for a flexible kernel design. This is particularly beneficial for radial acquisitions, where GRAPPA calibration with a radial kernel would demand multiple ACS acquisitions due to the limitation of neighborhood relationships to small radial sections [14]. In contrast, our work illustrates how the proposed PISCO-NIK approach effectively utilizes Cartesian relationships to refine radial acquisitions.

Adaptation of kernel design to the undersampling pattern may further enhance the effectiveness of PISCO. Notably, improved performance can be observed for accelerated reconstructions, particularly for R=2. This improvement can be attributed to the kernel design used, which samples one adjacent neighbor in each direction to the available acquisition points. Thereby, it effectively covers points not present with R=2. For higher acceleration factors, larger k-space gaps persist, potentially requiring a larger kernel size to focus PISCO’s attention during model training at these gaps. Kernel design as well as hyperparameter tuning for PISCO are subject to further investigation.

By introducing a regularization term for neural implicit k-space representations for the first time, we have improved its potential for motion-resolved abdominal imaging. Yet, training times and the need for re-training of neural implicit k-space representations remain a challenge for MR reconstruction [16]. Moreover, the reliability of the learned representation depends on a reliable surrogate motion signal, introducing uncertainties. Future research addresses these uncertainties as well as methods to accelerate the training procedure.

Concluding, we have demonstrated how a conventional parallel imaging concept can be leveraged as self-supervised regularization for learning-based reconstruction. Due to its calibration-free and flexible design, our proposed method PISCO can be seamlessly integrated into the clinical workflow, making it an attractive regularization method for application in further anatomies or k-space-based reconstruction methods.