Low-Complexity CSI Feedback for FDD Massive MIMO Systems via Learning to Optimize

In order to solve Problem (5), fully data-driven DL-based methods [9, 10, 11] have been developed. The map** $\mathcal{E}_{\theta_{\mathrm{e}}}(\cdot)$ and $\mathcal{D}_{\theta_{\mathrm{d}}}(\cdot)$ can be instantiated as DL-based encoder and decoder, and jointly trained via end-to-end learning [9, 10, 11]. Fully data-driven DL-based approaches obtain better performance than traditional CS-based methods, especially at low compression ratios. This is because of the powerful representation ability and universal approximation of neural networks. However, most of the existing works improve the reconstruction accuracy at the cost of higher neural network complexity, e.g., larger kernels, deeper neural networks, or complicated attention mechanism, which is not affordable for resource-constrained devices, e.g., mobile phones. For example, assume that the compression ratio is 1/16 and the CSI feedback and recovery period is 1 millisecond. The computational overhead required by the TransNet encoder is about 17.07 G floating point operations per second (FLOPS). Note that Kirin 659, one of the mid-end mobile systems on chip (SoC), has a total peak computation capability of 57.6 G FLOPS [25]. If the TransNet is deployed in practice, around $30\%$ of the mobile’s computational power is used for CSI feedback, which cannot be acceptable. Although TransNet achieves SOTA performance, the extensive computational demands and memory requirements hinder its practical deployments.

By taking the physical meaning of encoding and decoding process into consideration, deep unfolding methods were proposed for CSI feedback [19, 20, 26]. It is shown in classic CS theory that when a signal exhibits a certain sparsity in a specific transform domain, we can obtain a small number of codewords (observations) through linear projection and use nonlinear recovery map** to get an accurate estimation of the original signal [13]. By amalgamating the CS knowledge, deep unfolding-based methods implement a linear learnable encoding process to reduce the signal dimension at the encoder side. The projected codeword can be expressed as

where $\mathbf{W}$ is the sampling matrix and $\mathbf{h_{\text{vec}}}\in\mathbb{R}^{2N_{a}N_{t}\times 1}$ is the vectorized channel matrix $\mathbf{H}^{\prime\prime}$ stacking the real and imaginary part. The decoding process at the BS can be regarded as solving an inverse problem. The dimensionality reduction in (6) makes the signal recovery notably ill-posed. A regularization term is typically added to the optimization function to make use of known prior information about the optimal solution, which is expressed as

where $R(\mathbf{x})$ is the regularization term. Typically, $l_{1}$-norm is utilized as a regularizer, i.e., $R(\mathbf{x})=\lambda||\Psi\mathbf{x}||_{1}$, where $\Psi$ is a certain sparse transformation. Problem (7) is then written as

Iterative Shrinkage-Thresholding Algorithm (ISTA) is a classic iterative method to solve Problem (8), and the follow-up model-driven DL methods for CSI feedback are inspired by ISTA-based algorithms. At the $t$-th step of ISTA, the iterative process is expressed as

where $\mathbf{u}^{[t]}$, $\alpha$, and $\theta$ are the intermediate variable, step size, and thresholding parameter, respectively. In [27], a model-driven DL method, ISTA-Net, is proposed. It is designed to learn optimal parameters, i.e. thresholds, step sizes as well as nonlinear transforms, without hand-crafted settings, in an end-to-end manner. The ISTA-Net method adopts CNN to approximate the nonlinear sparse transformation and improves the recovery performance compared to conventional CS algorithm. As a deep unfolding method for CSI feedback, TiLISTA [19] utilizes a sparse auto-encoder to learn the sparse transformation in the spatial domain. Nevertheless, due to the truncation of the ISTA algorithm into a finite and fixed number of layers for both training and inference stages, ISTA-Net and TiLISTA struggle with scaling to accommodate a variable number of iterations and face challenges in guaranteeing convergence upon implementation. These problems motivate us to propose a new model-driven DL-based network for CSI feedback with provable convergence guarantee.

Since wireless channels are typically non-stationary, traditional fixed domains, e.g., DFT wavelet transformation, usually result in poor reconstruction performance. In practice, the spatial angles are continuous rather than discrete, which makes the sparsity of the channel coefficients after DFT transformation still insufficient [19]. To demonstrate this conclusion, we plot the gray-scale channel visualizations in angular-delay domain in Fig. 3(a). We can observe from Fig. 3(a) that due to the multipath effect, there is a high level of sparsity in the delay domain, i.e., only a few elements in each column of channel matrx $\mathbf{H}^{\prime\prime}$ contains significant values. However, in angular domain (each row of the channel matrix), the sparsity level is still unsatisfactory. This reveals that the signals after the DFT transformation is still not strictly sparse when the number of antennas is not large enough. Besides, due to the complicated outdoor communication surroundings, the sparsity level of outdoor scenario is less satisfactory compared to the indoor scenario. In this paper, considering the characteristic of wireless channels, we design a learnable angular domain sparse transformation for CSI feedback.

The details of the sparse transformation and the inverse transformation are shown in Fig. 3(b). To enhance the sparsity level in the angular domain, each row of the channel matrix $\mathbf{H}^{\prime\prime}$ is selected and fed into the neural network individually. We employ an MLP with three fully-connected layers as $f_{t}(\cdot)$. $N_{i}$ denotes the output dimension of $f_{t}(\cdot)$. In order to obtain strictly sparse signals, only the largest $G$ values of the output of $f_{t}(\cdot)$ are retained and all the other values are set zero, referring to the top $G$ activation. By doing this, the proposed learning-based sparse transformation function transforms angular domain channels into another domain with sparse features. The inverse transformation $f_{i}(\cdot)$ exhibits a reverse structure compared to $f_{t}(\cdot)$. It maps the channels in the learned sparse domain back to the angular domain. The rows of $\mathbf{H}^{\prime\prime}$ are processed in parallel. After obtaining the output of $f_{i}(\cdot)$, the estimated channel matrix can be constructed by stacking rows into a whole matrix.

The proposed $f_{t}(\cdot)$ and $f_{i}(\cdot)$ guarantee sparsity in the transformed signals and strive to ensure that the signals, when inversely transformed, closely resemble the original ones. $f_{t}(\cdot)$ and $f_{i}(\cdot)$ are trained end-to-end with other learning components and the training loss is

where $\mathbf{H}^{\prime\prime}_{i}$ denotes the $i$-th channel matrix in the traning dataset, $D$ denotes the total number of training samples, and $\beta$ denotes the balancing term between channel recovery MSE and the sparse transformation MSE. The proposed sparse transformation effectively overcomes the shortcomings of manually designed transformations for wireless channels. It seeks to discover a sparse basis specifically within the angular domain of the CSI matrix. Moreover, the sparse transformation and inverse transformation learned from the numerous CSI training data is more consistent with the data of the specific channel model [19]. Therefore, the learnable sparse transformation can obtain a more effective sparse representation of CSI, which improves the reconstruction accuracy of the proposed network.

Let $F(\mathbf{x})$ denote the objective function of (10). Conventional CS algorithms, e.g., ISTA, solve Problem (10) via proximal gradient descent. However, the use of fixed update rule and manually designed optimization parameters leads to a large number of iterations and high computational cost. In contrast to ISTA, we propose to learn the update rule from data to boost decoder-side convergence. The proposed method is designed to determine the update directions by taking the current estimate, i.e., $\mathbf{x}^{[t]}$, and the gradient of the objective function, i.e., $\nabla F(\mathbf{x}^{[t]})$, as inputs. The general update rule of the $t$-th iteration is written as:

where $\mathbf{d}^{[t]}:\mathcal{Z}\to\mathbb{R}^{2N_{a}N_{t}}$ denotes the update direction, $\mathbf{z}^{[t]}\in\mathcal{Z}$ is the input vector, and $\mathcal{Z}$ is the input space. The input vector involves dynamic information, for example $\{\mathbf{x}^{[t]},F(\mathbf{x}^{[t]}),\nabla F(\mathbf{x}^{[t]})\}$. We assume that the update rule $\mathbf{d}^{[t]}(\cdot)$ is differentiable with respect to the input $\mathbf{z}^{[t]}$ and its Jacobian is bounded by a scalar $C$. Formally speaking, the space of update rules is as follows.

In practice, training the deep neural network that is parameterized from $\mathbf{d}^{[t]}(\cdot)$ will require the derivatives of $\mathbf{d}^{[t]}(\cdot)$. Therefore, the differentiablility and bounded Jacobian of the update direction are important. Note that many existing L2O approaches, e.g., LSTM in [24, 30], achieve $\mathbf{d}^{[t]}(\cdot)\in\mathcal{D}_{C}(\mathcal{Z})$. The definition of the update rule space will help guanrantee the convergence of the proposed L2O scheme, which is proved in Section IV-A.

Note that the objective function in Problem (10) contains a smooth fidelity function $f(\mathbf{x})=\frac{1}{2}||\mathbf{s}-\mathbf{Wx}||_{2}^{2}$ and a non-smooth regularization function $r(\mathbf{x})=\lambda||f_{t}(\mathbf{x})||_{1}$. For the smooth part, $\mathbf{x}^{[t]}$ and $\nabla f(\mathbf{x}^{[t]})$ are taken as the input to the update rule. For the non-smooth part, a subgradient $\mathbf{g}^{[t]}$ of $r(\mathbf{x})$ can be utilized. However, the convergence of subgradient descent is generally unstable, and it will not converge to the solution if the step size is constant [31]. Proximal Point Algorithm (PPA) [32] converges faster and more stably than the subgradient descent method. While subgradient descent adopts explicit update, the PPA method takes implicit update rule, i.e.,

where $\alpha_{\text{PPA}}$ denotes the step size of PPA algorithm. Inspired by PPA, we select $\mathbf{x}^{[t+1]}$ and $\mathbf{g}^{[t+1]}$ to be the input to the update rule $\mathbf{d}^{[t]}(\cdot)$.

In addition to $\mathbf{x}^{[t]}$, $\nabla f(\mathbf{x}^{[t]})$, $\mathbf{x}^{[t+1]}$, and $\mathbf{g}^{[t+1]}$, we also introduce an auxiliary input $\mathbf{y}^{[t]}$ to $\mathbf{d}^{[t]}(\cdot)$ which contains information about the past estimates and is able to accelerate convergence. Recall that the update schemes (9) of existing deep unfolding methods introduced in previous section explicitly depend on only the current status $\mathbf{x}^{[t]}$. Therefore, they lose the ability to capture dynamics between iterations and tend to memorize the datasets. To address this drawback, in the proposed method, we introduce an auxiliary variable $\mathbf{y}^{[t]}$ that encodes historical information through an operator $\mathbf{m}$:

where in addition to the current estimate $\mathbf{x}^{[t]}$, the past $K$ iterations estimates are also taken into consideration. To facilitate parameterization and training, we assume $\mathbf{m}$ is differentiable, i.e., $\mathbf{m}\in\mathcal{D}_{C}(\mathbb{R}^{(T+1)\times 2N_{a}N_{t}})$. With the help of $\mathbf{y}^{[t]}$, we are able to infuse more information into the update rule. We set the current estimate, the gradient, the current auxiliary variable, and the gradient of the auxiliary variable as the inputs of the update rule $\mathbf{d}^{[t]}$. The update rule is then shown as [21]

Follow the derivation in [21, Theorem 4], a good update rule should satisfy asymptotic fixed point condition and global convergence condition. We then derive a math-structured update rule from generic update rule (15), i.e., for any bounded matrix sequence $\{\mathbf{B}^{[t]}\}_{t=1}^{\infty}$, there exist

for all $t=1,2,\cdots$, with $\{\mathbf{P}_{1}^{[t]},\mathbf{P}_{2}^{[t]},\mathbf{A}^{[t]}\}$ being bounded, and $\mathbf{b}_{1}^{[t]}\to\mathbf{0},\mathbf{b}_{2}^{[t]}\to\mathbf{0}$ as $t\to\infty$. If we further assume $\mathbf{P}_{1}^{[t]}$ is uniformly symmetric positive definite, then we can substitute $\mathbf{P}_{2}^{[t]}{\mathbf{P}_{1}^{[t]}}^{-1}$ with $\mathbf{B}^{[t]}$ and obtain

where $\operatorname{prox}_{r,\mathbf{P}_{1}^{[t]}}(\cdot)$ denotes the proximal operator and is defined as

The norm $\|\cdot\|_{\mathbf{P}^{-1}}$ is defined as $\|\mathbf{x}\|_{\mathbf{P}^{-1}}:=\sqrt{\mathbf{x}^{\top}\mathbf{P}^{-1}\mathbf{x}}$.

In the update scheme (17), $\mathbf{b}_{1}^{[t]}$ and $\mathbf{b}_{2}^{[t]}$ are biases; $\mathbf{A}^{[t]}$ is an accelerator term which can be viewed as an extension of Nesterov momentum; $\mathbf{P}_{1}^{[t]}$ is the preconditioner that plays a similar role as step size in the gradient descent; $\mathbf{B}^{[t]}$ is a balancing term between $\hat{\mathbf{x}}^{[t]}$ and $\hat{\mathbf{y}}^{[t]}$. If $\mathbf{B}^{[t]}=\mathbf{0}$, then $\mathbf{x}^{[t+1]}$ only depends on $\mathbf{x}^{[t]}$ and if $\mathbf{B}^{[t]}=\mathbf{1}$, then $\mathbf{x}^{[t+1]}$ only depends on $\mathbf{y}^{[t]}$ explicitly. Note that ISTA is a special case of update rule (17). When $\mathbf{B}^{[t]}=\mathbf{A}^{[t]}=\mathbf{b}_{1}^{[t]}=\mathbf{b}_{2}^{[t]}=\mathbf{0}$, (17) reduces to ISTA. Therefore, (17) provides more degrees of freedom and is able to enhance reconstruction performance.

To obtain a better balance between performance and efficiency, in our Csi-L2O decoding module, $\mathbf{P}_{1}^{[t]}$, $\mathbf{B}^{[t]}$, and $\mathbf{A}^{[t]}$ are implemented as diagonal matrices, i.e.,

where $\mathbf{p}^{[t]},\mathbf{b}^{[t]},\mathbf{a}^{[t]}\in\mathbb{R}^{2N_{a}N_{t}\times 1}$. The proximal operator is set a scaled soft-thresholding operator, which is expressed as

where $\theta^{[t]}$ denotes the soft-thresholding parameter in the $t$-th iteration. Update rule (17) then becomes:

To generate the most appropriate decoding algorithm, the optimization parameters $\mathbf{p}^{[t]}$, $\mathbf{a}^{[t]}$, $\mathbf{b}^{[t]}$, $\mathbf{b}_{1}^{[t]}$, and $\mathbf{b}_{2}^{[t]}$ are not selected mannually but learned from a large amount of data. Note that $\mathbf{p}^{[t]},\mathbf{a}^{[t]},\mathbf{b}^{[t]},\mathbf{b}_{1}^{[t]},\mathbf{b}_{2}^{[t]}\in\mathbb{R}^{2N_{a}N_{t}\times 1}$. In FDD massive MIMO systems, $N_{a}$ and $N_{t}$ are large. If a black-box neural network is adopted to model these optimization parameters, the training of the giant and unstructured neural network will be very difficult. In addition, for FDD massive MIMO CSI feedback, the compression ratio needs to be adjusted according to the dynamic communication environment. A reconstruction algorithm that enjoys good generalization ability is thus greatly in need. By taking these two aspects into consideration, we design an element-wise L2O mechanism. In contrast to traditional deep unfolding methods that directly set optimization parameters $\mathbf{p}^{[t]}$, $\mathbf{a}^{[t]}$, $\mathbf{b}^{[t]}$, $\mathbf{b}_{1}^{[t]}$, and $\mathbf{b}_{2}^{[t]}$ trainable, we model them as the output of an element-wise LSTM that greatly improves the generalization ability to different problem scale. The element-wise LSTM is parameterized by learnable parameters $\phi_{\text{LSTM}}$ and takes the current estimate $\mathbf{x}^{[t]}$ and the gradient $\nabla f(\mathbf{x}^{[t]})$ as the input: