Hybrid Precoder Design for Angle-of-Departure Estimation with Limited-Resolution Phase Shifters

Hui** Huang, Member, IEEE, Musa Furkan Keskin, Member, IEEE, Henk Wymeersch, Fellow, IEEE,
Xuesong Cai, Senior Member, IEEE, Linlong Wu, Member, IEEE, Johan Thunberg,
Fredrik Tufvesson, Fellow, IEEE This paper is supported by the Vinnova B5GPOS Project under Grant 2022-01640.H. Huang, M. F. Keskin, and H. Wymeersch are with Department of Electrical Engineering, Chalmers University of Technology, 41296 Gothenburg, Sweden (e-mail: {hui**; furkan; henkw}@chalmers.se).X. Cai, J. Thunberg, and F. Tufvesson are with Department of Electrical and Information Technology, Lund University, 22100 Lund, Sweden (e-mail: {xuesong.cai; johan.thunberg; fredrik.tufvesson}@eit.lth.se).L. Wu is with the Interdisciplinary Centre for Security, Reliability and Trust (SnT), University of Luxembourg, 1855 Luxembourg, Luxembourg (e-mail: [email protected]).

Abstract

Hybrid analog-digital beamforming stands out as a key enabler for future communication systems with a massive number of antennas. In this paper, we investigate the hybrid precoder design problem for angle-of-departure (AoD) estimation, where we take into account the practical constraint on the limited resolution of phase shifters. Our goal is to design a radio-frequency (RF) precoder and a base-band (BB) precoder to estimate AoD of the user with a high accuracy. To this end, we propose a two-step strategy where we first obtain the fully digital precoder that minimizes the angle error bound, and then the resulting digital precoder is decomposed into an RF precoder and a BB precoder, based on the alternating optimization and the alternating direction method of multipliers. Besides, we derive the quantization error upper bound and analyse the convergence behavior of the proposed algorithm. Numerical results demonstrate the superior performance of the proposed method over state-of-the-art baselines.

Index Terms:

Hybrid beamforming, hybrid precoder, phase shifter, angle-of-departure estimation, alternating optimization, alternating direction method of multipliers.

I Introduction

Millimeter wave (mmWave) and terahertz (THz) band have been proven to play an important role in future wireless systems, because they can provide ultra-high data rates [1, 2, 3, 4, 5, 6, 7, 8]. However, high carrier frequencies result in severe path loss. Large-scale antenna systems, which are equipped with hundreds or even thousands of antennas, have emerged as a crucial technology for addressing this problem [9, 10, 11, 12, 13].

It is not feasible for large-scale antenna systems to employ fully digital beamforming at mmWave/THz, since fully digital beamforming requires as many radio-frequency (RF) chains (including digital-to-analog converters, mixers, etc.) as the antennas, leading to prohibitive hardware costs and power consumption [14]. On the contrary, hybrid beamforming where only a small number of RF chains are needed is a promising solution to handle this problem [15]. The RF chains are connected to antennas via phase shifters with a finite number of quantized phases [16, 17, 18, 19, 20, 21, 22].

Numerous works have been devoted to hybrid beamformer (precoder and/or combiner) design with practical constraints [23, 24, 25, 18, 26, 27, 28, 29, 30, 31, 32, 19, 33, 20, 21, 22, 34, 35]. Among them, the following four methods attract much attention. (i) The authors in [18] proposed a hybrid beamforming algorithm with 1-bit resolution phase shifter, which is based on alternating optimization framework and the Babai algorithm [36] (termed as “Alt-Babai”). (ii) An iterative hybrid transceiver design approach using alternating optimization and coordinate descent method (CDM) was developed in [19] (termed as “Alt-CDM”). (iii) [24] exploited the spatial structure of mmWave channels and proposed a method for optimal unconstrained precoders and combiners, which employs sparse representation and orthogonal matching pursuit (termed as “Spa-OMP”). (iv) Another hybrid precoding method was presented in [29], which is on the basis of the manifold optimization [37, 38, 39] (termed as “ManiOpt”). All the above-mentioned hybrid beamforming design methods are from the communications perspective. In contrast, much less work has focused on hybrid beamforming design for channel parameters (such as angles, delays, Dopplers, etc.) estimation and positioning. Although optimal beamforming design for positioning has been investigated in e.g., [40, 41, 42, 43], these works investigated fully digital beamforming rather than hybrid beamforming. Note that existing hybrid beamformer design methods in [23, 24, 25, 18, 26, 27, 28, 29, 30, 31, 32, 19, 33, 20, 21, 22, 34, 35] can be applied for positioning. However, these methods do not guarantee a good performance in positioning (since they are proposed for the purpose of communications). Therefore, there is lack of dedicated hybrid beamforming design for the purpose of channel parameters estimation and positioning.

To fill the research gap, in this paper we delve into the intricate problem of hybrid precoder design for angle-of-departure (AoD) estimation, accounting for practical limitation on the finite resolution of phase shifters. Our objective is to derive a solution comprising an RF precoder and a base-band (BB) precoder that not only adheres to the practical constraint but also facilitates precise user AoD estimation. To achieve this goal, we present a two-step approach. We first find a fully digital precoder that minimizes the angle error bound, which is the theoretical lower bound on AoD estimation. Then, we decompose the resulting digital precoder into an RF precoder and a BB precoder, by using alternating optimization framework and the alternating direction method of multipliers (ADMM). The numerical results show that the proposed method outperforms existing state-of-the-art approaches while incurring less complexity. The main contributions of this work are listed as follows:

•

The problem of hybrid beamformer design under practical constraints has not yet been considered for positioning (specifically AoD estimation). We consider such a problem and develop an efficient algorithm to obtain the RF precoder and BB precoder.
•

In the existing literature of hybrid beamforming with limited-resolution phase shifters, e.g., [23, 24, 25, 18, 26, 27, 28, 29, 30, 31, 32, 19, 33, 20, 21, 22, 34, 35], no theoretical results are available regarding the quantization error bound. In this paper, we derive such an error bound.
•

We provide convergence analyses of the proposed algorithm. Our analyses differ from the related works in [44, 45, 46, 47, 48, 49] since our algorithm involves a quantization operation, which is not the case in the related works.
•

The convergence analyses presented in this paper go beyond our previous works in [47, 48, 49], as the former additionally reveal that the point sequence produced by the proposed algorithm is a Cauchy sequence and it converges to a fixed point after a finite number of iterations.

The remainder of this paper is organized as follows. The system model is described in Section II. Section III presents the proposed method for hybrid precoder design for AoD estimation. Section IV analyzes the quantization error bound and convergence behavior of the proposed algorithm. Various numerical examples are provided in Section V to demonstrate the effectiveness of the proposed approach, followed by conclusions in Section VI.

II System Model

We consider a mmWave downlink positioning scenario as in [50], shown in Fig. 1, where the base station (BS) consists of a BB precoder, an RF precoder, and a uniform linear array (ULA) of $N_{\text{Tx}}$ antennas; while the user equipment (UE) consists of a single antenna. The RF precoder is implemented by limited-resolution phase shifters.

The BS transmits $M$ pilot symbols sequentially with identical power, denoted as $s_{m},m=1,2,\cdots,M$ . Employing a two-timescale hybrid precoding approach [51, 52, 53], we adopt a transmission model in which the analog RF precoder is optimized at a slower time scale compared to the digital BB precoder. This prevents high hardware costs (attributed to rapid adaptation of the analog precoder) and reduces computational complexity, along with minimizing signaling overhead [51]. In particular, each symbol is first precoded by a dedicated BB precoder vector, ${\bf f}_{\text{BB},m}\in\mathbb{C}^{N_{\text{RF}}}$ , and then precoded by an RF precoder constant for all symbols, ${\bf F}_{\text{RF}}\in\mathbb{C}^{N_{\text{Tx}}\times N_{\text{RF}}}$ , where $N_{\text{RF}}\leq N_{\text{Tx}}$ denotes the number of RF chains. Considering highly directional mmWave transmissions, we assume a line-of-sight (LOS)-only channel¹¹1The LOS path is resolvable from the non-line-of-sight paths due to channel sparsity, large number of antennas, and large bandwidth in the mmWave/THz wireless communication systems [5].. Thus, corresponding to the transmitted signal $s_{m}$ , the received signal at the single-antenna UE can be modeled as

\displaystyle y_{m}=\beta{\bf a}^{\textrm{T}}(\theta){\bf F}_{\text{RF}}{\bf f% }_{\text{BB},m}s_{m}+n_{m},~{}m=1,2,\cdots,M,

(1)

where $\beta\in\mathbb{C}$ is the complex amplitude of the path, $\theta$ is the AoD, $n_{m}$ is the complex additive white Gaussian noise with zero mean and variance $\sigma_{\text{n}}^{2}$ , and the steering vector is

\displaystyle{\bf a}(\theta)=\left[1,e^{-\jmath\frac{2\pi d}{\lambda}\sin{% \theta}},\cdots,e^{-\jmath\frac{2\pi d}{\lambda}(N_{\text{Tx}}-1)\sin{\theta}}% \right]^{\textrm{T}},

(2)

with $\jmath=\sqrt{-1}$ , $d$ being the element spacing of the ULA, and $\lambda$ denoting the transmit signal wavelength.

The signal model (1) can be written in vector form as

\displaystyle{\bf y}=\beta{\bf S}({\bf F}_{\text{RF}}{\bf F}_{\text{BB}})^{% \textrm{T}}{\bf a}(\theta)+{\bf n},

(3)

where ${\bf y}\!=\![{y}_{1},{y}_{2},\cdots,{y}_{M}]^{\textrm{T}}\in\mathbb{C}^{M}$ , ${\bf n}\!=\![{n}_{1},{n}_{2},\cdots,{n}_{M}]^{\textrm{T}}\in\mathbb{C}^{M}$ , ${\bf F}_{\text{BB}}=[{\bf f}_{\text{BB},1},{\bf f}_{\text{BB},2},\cdots,{\bf f% }_{\text{BB},M}]\in\mathbb{C}^{N_{\text{RF}}\times M}$ , and ${\bf S}=\textrm{diag}\{[s_{1},s_{2},\cdots,s_{M}]\}$ with $\textrm{diag}\{\cdot\}$ representing the diagonal matrix operator. In addition, ${\bf n}\sim\mathcal{CN}({\bf 0},\sigma_{\text{n}}^{2}{\bf I})$ with known noise power $\sigma_{\text{n}}^{2}$ . Our goal is to design an RF precoder and a BB precoder such that the accuracy of estimation of AoD is maximized, under the BS transmit power constraint and the hardware constraint on the limited resolution of phase shifters.

Refer to caption — Figure 1: Illustration of a mmWave downlink positioning scenario, where the single-antenna UE receives the pilot signals transmitted from the BS (consisting of a BB precoder, an RF precoder, and a ULA of multiple antennas).

III Proposed Method

III-A CRB-Based Performance Metric

Define ${\bf{\tilde{y}}}\triangleq\beta{\bf S}({\bf F}_{\text{RF}}{\bf F}_{\text{BB}})% ^{\textrm{T}}{\bf a}(\theta)$ . Then, the Fisher information matrix (FIM) ${\bf J}({\bf F}_{\text{RF}},{\bf F}_{\text{BB}};{\bf x})\in\mathbb{R}^{3\times 3}$ can be computed by using the Slepian-Bangs formula [54] as

\displaystyle[{\bf J}]_{ij}=\frac{2}{\sigma_{\text{n}}^{2}}\Re\left\{\left(% \frac{\partial{\bf{\tilde{y}}}}{\partial[{\bf x}]_{i}}\right)^{\textrm{H}}% \left(\frac{\partial{\bf{\tilde{y}}}}{\partial[{\bf x}]_{j}}\right)\right\},

(4)

where ${\bf x}=[\theta,\beta_{\text{R}},\beta_{\text{I}}]^{\textrm{T}}$ contains all the unknown parameters, $[{\bf J}]_{ij}$ is the entry of ${\bf J}$ in the $i$ -th row and $j$ -th column, and $[{\bf x}]_{i}$ is the $i$ -th entry of ${\bf x}$ . In addition, $\beta_{\text{R}}$ and $\beta_{\text{I}}$ denote the real and imaginary parts of $\beta$ , respectively. The derivative of ${\bf{\tilde{y}}}$ with respect to (w.r.t.) $[{\bf x}]_{i}$ is calculated as in Appendix A. The corresponding Cramér-Rao bound (CRB) matrix is defined as

\displaystyle{\bf C}={\bf J}^{-1}.

(5)

To quantify the AoD estimation accuracy, we adopt the angle error bound (AEB) as our performance metric, computed as (6) displayed at the top of the next page, where $\sigma_{\text{s}}$ is the signal power, ${\bf D}\triangleq\text{diag}\{0,1,\cdots,N_{\text{Tx}-1}\}$ , ${\bf F}={\bf F}_{\text{RF}}{\bf F}_{\text{BB}}$ , and we have employed the block matrix inversion lemma [55] as detailed in Appendix B.

\displaystyle\text{AEB}({\bf F}_{\text{RF}},{\bf F}_{\text{BB}};{\bf x})=\sqrt% {[{\bf C}]_{11}}=\frac{\sigma_{\text{n}}}{\sigma_{\text{s}}}\frac{\lambda}{2% \sqrt{2}\pi d}\sqrt{\frac{{\bf a}^{\textrm{H}}(\theta){\bf F}^{*}{\bf F}^{% \textrm{T}}{\bf a}(\theta)}{{\bf a}^{\textrm{H}}(\theta){\bf F}^{*}{\bf F}^{% \textrm{T}}\big{[}|\beta|^{2}{\bf a}(\theta){\bf a}^{\textrm{H}}(\theta){\bf D% }{\bf F}^{*}{\bf F}^{\textrm{T}}{\bf D}-{\bf D}{\bf a}(\theta){\bf a}^{\textrm% {H}}(\theta){\bf D}{\bf F}^{*}{\bf F}^{\textrm{T}}\big{]}{\bf a}(\theta)}}

(6)

III-B Problem Formulation for Optimal Precoder Design

The AEB depends on the unknown parameters in ${\bf x}$ . We assume that ${\bf x}$ belongs to an uncertainty set $\mathcal{X}$ that can be, e.g., determined via some tracking algorithms [56, 40, 41]. For any ${\bf x}\in\mathcal{X}$ , the AEB is only a function of ${\bf F}_{\text{RF}}$ and ${\bf F}_{\text{BB}}$ . The optimal precoder design problem can be formulated as

$\displaystyle\min_{{\bf F}_{\text{RF}},{\bf F}_{\text{BB}}}$	$\displaystyle~{}\text{AEB}({\bf F}_{\text{RF}},{\bf F}_{\text{BB}};{\bf x})$	(7a)
s.t.	$\displaystyle~{}\\|{\bf F}_{\text{RF}}{\bf F}_{\text{BB}}\\|_{\text{F}}^{2}=P,$	(7b)
	$\displaystyle~{}\left[{\bf F}_{\text{RF}}\right]_{ij}\in\mathcal{F},~{}~{}1% \leq i\leq N_{\text{Tx}}~{}\text{and}~{}1\leq j\leq N_{\text{RF}},$	(7c)

where $P$ stands for the total transmit power of the BS antennas, and $\mathcal{F}$ denotes the set for limited resolution of the phase shifters, which is defined as:

\displaystyle\mathcal{F}\triangleq\left\{\frac{1}{\sqrt{N_{\text{Tx}}}}e^{% \jmath 2\pi b/2^{B}}\bigg{|}b=0,1,\cdots,2^{B}\!-\!1\right\},

(8)

with $B$ representing the total number of quantization bits of the phase shifters.

III-C Two-Step Strategy for Solving Problem (III-B)

It is difficult to directly solve Problem (III-B) w.r.t. ${\bf F}_{\text{RF}}$ and ${\bf F}_{\text{BB}}$ , due to the complicated structure²²2The denominator of $\text{AEB}({\bf F}_{\text{RF}},{\bf F}_{\text{BB}};{\bf x})$ contains quartic terms w.r.t. ${\bf F}={\bf F}_{\text{RF}}{\bf F}_{\text{BB}}$ . of $\text{AEB}({\bf F}_{\text{RF}},{\bf F}_{\text{BB}};{\bf x})$ and the discrete-phase nature of the entries of ${\bf F}_{\text{RF}}$ . We provide a strategy for solving Problem (III-B) via the following two steps:

•

Step 1: Finding the optimal fully digital precoder ${\bf F}_{\text{opt}}$ as a solution to Problem (III-B).
•

Step 2: Finding a decomposition of ${\bf F}_{\text{opt}}$ to obtain the best approximation ${\bf F}_{\text{opt}}\approx{\bf F}_{\text{RF}}{\bf F}_{\text{BB}}$ in the least-squares (LS) sense.

We now elaborate on these two steps.

Step 1: Based on the fact that the unknown variables ${\bf F}_{\text{RF}}$ and ${\bf F}_{\text{BB}}$ appear as a product (i.e., ${\bf F}_{\text{RF}}{\bf F}_{\text{BB}}$ ) in both the objective function (7a) and the constraint (7b), for any ${\bf x}\in\mathcal{X}$ , we consider the following optimization problem:

\displaystyle\min_{{\bf F}}~{}\text{AEB}({\bf F};{\bf x})\quad\text{s.t.}~{}\|% {\bf F}\|_{\text{F}}^{2}=P,

(9)

where ${\bf F}={\bf F}_{\text{RF}}{\bf F}_{\text{BB}}\in\mathbb{C}^{N_{\text{Tx}}% \times M}$ and we drop the constraint (7c) temporarily. This corresponds to a fully digital precoder optimization [41]. We define ${\bf Z}\triangleq{\bf F}{\bf F}^{\textrm{H}}$ , and relax Problem (9) by removing the constraint $\text{rank}({\bf Z})=M$ , as

\displaystyle\min_{{\bf Z},u}~{}\!u~{}~{}\text{s.t.}\left[\begin{array}[]{cc}{% \bf J}({\bf Z};{\bf x})&{\bf e}_{1}\\ {\bf e}_{1}^{\textrm{T}}&u\end{array}\right]\succeq{\bf 0},~{}\text{tr}({\bf Z% })=P,~{}{\bf Z}\succeq{\bf 0},

(12)

where ${\bf e}_{1}=[1,0,0]^{\text{T}}$ , $\text{tr}(\cdot)$ is the trace of a matrix, and ${\bf Z}\succeq{\bf 0}$ means that ${\bf Z}$ is positive semidefinite. Taking into account the uncertainty of ${\bf x}$ , i.e., ${\bf x}\in\mathcal{X}$ , and by discretizing ${\mathcal{X}}$ into a uniform grid of $G$ points $\{{\bf x}_{g}\}_{g=1}^{G}$ , a robust design for the above problem can be given as

$\displaystyle\min_{{\bf Z},\{u_{g}\}}$	$\displaystyle~{}\!\max_{{\bf x}\in\mathcal{X}}~{}\!u_{g}$	(13a)
s.t.	$\displaystyle\left[\begin{array}[]{cc}{\bf J}({\bf Z};{\bf x})&{\bf e}_{1}\\ {\bf e}_{1}^{\textrm{T}}&u_{g}\end{array}\right]\succeq{\bf 0},~{}\!g=1,2,% \cdots,G,$	(13d)
	$\displaystyle~{}\!\text{tr}({\bf Z})=P,~{}{\bf Z}\succeq{\bf 0}.$	(13e)

Problem (III-C) can be further formulated as

$\displaystyle\min_{{\bf Z},\{u_{g}\},t}$	$\displaystyle~{}\!t$	(14a)
s.t.	$\displaystyle\left[\begin{array}[]{cc}{\bf J}({\bf Z};{\bf x})&{\bf e}_{1}\\ {\bf e}_{1}^{\textrm{T}}&u_{g}\end{array}\right]\succeq{\bf 0},~{}\!g=1,2,% \cdots,G,$	(14d)
	$\displaystyle~{}\!u_{g}\leq t,~{}\!g=1,2,\cdots,G,$	(14e)
	$\displaystyle~{}\!\text{tr}({\bf Z})=P,~{}{\bf Z}\succeq{\bf 0}.$	(14f)

It is shown in [41] that a codebook-based approach can be applied to decrease the complexity while achieving a optimal design. Specifically, a predefined codebook consists of directional and derivative beams [41], that is, ${\bf F}^{(\text{pre})}=[{\bf F}^{(\text{direc})},{\bf F}^{(\text{deriv})}]$ , where ${\bf F}^{(\text{direc})}=[{\bf a}(\theta_{1}),{\bf a}(\theta_{2}),\cdots,{\bf a% }(\theta_{G})]$ and ${\bf F}^{(\text{deriv})}=[{\bf{\dot{a}}}(\theta_{1}),{\bf{\dot{a}}}(\theta_{2}% ),\cdots,{\bf{\dot{a}}}(\theta_{G})]$ , with ${\bf{\dot{a}}}(\theta)=\frac{\partial{\bf a}(\theta)}{\partial\theta}$ and $G=M/2$ . With the predefine codebook, we consider the optimal beam power allocation problem in ${\bf q}=[q_{1},q_{2},\cdots,q_{M}]^{\textrm{T}}$ [41]:

$\displaystyle\min_{{\bf q},\{u_{g}\},t}$	$\displaystyle~{}\!t$	(15a)
s.t.	$\displaystyle\left[\begin{array}[]{cc}{\bf J}({\bf Z};{\bf x})&{\bf e}_{1}\\ {\bf e}_{1}^{\textrm{T}}&u_{g}\end{array}\right]\succeq{\bf 0},~{}\!g=1,2,% \cdots,G,$	(15d)
	$\displaystyle~{}\!u_{g}\leq t,~{}\!g=1,2,\cdots,G,$	(15e)
	$\displaystyle~{}\!{\bf Z}={\bf F}^{(\text{pre})}\text{diag}({\bf q})({\bf F}^{% (\text{pre})})^{\textrm{H}},$	(15f)
	$\displaystyle~{}\!\text{tr}({\bf Z})=P,~{}{\bf Z}\succeq{\bf 0},$	(15g)

which yields the optimal fully digital precoder as

\displaystyle{\bf F}_{\text{opt}}={\bf F}^{(\text{pre})}{\text{diag}}([\sqrt{q% _{1}},\sqrt{q_{2}},\cdots,\sqrt{q}_{M}]).

(16)

Step 2: We decompose ${\bf F}_{\text{opt}}$ into two matrices, i.e., ${\bf F}_{\text{RF}}$ and ${\bf F}_{\text{BB}}$ , by taking into account the constraints (7b) and (7c):

\displaystyle\min_{{\bf F}_{\text{RF}},{\bf F}_{\text{BB}}}~{}\frac{1}{2}\|{% \bf F}_{\text{opt}}-{\bf F}_{\text{RF}}{\bf F}_{\text{BB}}\|_{\text{F}}^{2}% \quad\text{s.t.}~{}\text{(\ref{constraint1_OptPrecoder_problem})}~{}\text{and}% ~{}\text{(\ref{constraint2_OptPrecoder_problem})}.

(17)

In what follows, we propose an alternating optimization approach for solving Problem (17). To be specific, we first solve ${\bf F}_{\text{BB}}$ with a fixed ${\bf F}_{\text{RF}}$ , as

\displaystyle\min_{{\bf F}_{\text{BB}}}~{}\frac{1}{2}\|{\bf F}_{\text{opt}}-{% \bf F}_{\text{RF}}{\bf F}_{\text{BB}}\|_{\text{F}}^{2}\quad\text{s.t.}~{}\text% {(\ref{constraint1_OptPrecoder_problem})}.

(18)

It has a LS closed-form solution as

\displaystyle{\bf F}_{\text{BB}}=\frac{\sqrt{P}}{\|{\bf F}_{\text{RF}}{\bf F}_% {\text{RF}}^{\dagger}{\bf F}_{\text{opt}}\|_{\text{F}}}{\bf F}_{\text{RF}}^{% \dagger}{\bf F}_{\text{opt}},

(19)

where ${\bf F}_{\text{RF}}^{\dagger}=({\bf F}_{\text{RF}}^{\textrm{H}}{\bf F}_{\text{% RF}})^{-1}{\bf F}_{\text{RF}}^{\textrm{H}}$ . Then, we solve ${\bf F}_{\text{RF}}$ with the obtained ${\bf F}_{\text{BB}}$ in (19), as

\displaystyle\min_{{\bf F}_{\text{RF}}}~{}\frac{1}{2}\|{\bf F}_{\text{opt}}-{% \bf F}_{\text{RF}}{\bf F}_{\text{BB}}\|_{\text{F}}^{2}\quad\text{s.t.}~{}\text% {(\ref{constraint2_OptPrecoder_problem})}.

(20)

We develop an algorithm based on the ADMM [45] to solve the above problem. To this end, we introduce an auxiliary variable ${\bf{\tilde{F}}}_{\text{RF}}\in\mathbb{C}^{N_{\text{Tx}}\times N_{\text{RF}}}$ , and Problem (20) can be equivalently expressed as

\displaystyle\min_{{\bf F}_{\text{RF}},{\bf{\tilde{F}}}_{\text{RF}}}~{}\!\!% \frac{1}{2}\|{\bf F}_{\text{opt}}-{\bf{\tilde{F}}}_{\text{RF}}{\bf F}_{\text{% BB}}\|_{\text{F}}^{2}\quad\text{s.t.}~{}\text{(\ref{constraint2_OptPrecoder_% problem})}~{}\text{and}~{}{\bf{\tilde{F}}}_{\text{RF}}={\bf F}_{\text{RF}}.

(21)

The corresponding scaled-form augmented Lagrangian function is given as [45]

	$\displaystyle\mathcal{L}\Big{(}{\bf{\tilde{F}}}_{\text{RF}},{\bf F}_{\text{RF}},$	$\displaystyle~{}\!{\bf U}\Big{)}=\frac{1}{2}\\|{\bf F}_{\text{opt}}-{\bf{\tilde% {F}}}_{\text{RF}}{\bf F}_{\text{BB}}\\|_{\text{F}}^{2}$
		$\displaystyle+\frac{\rho}{2}\left(\\|{\bf{\tilde{F}}}_{\text{RF}}-{\bf F}_{% \text{RF}}+{\bf U}\\|_{\text{F}}^{2}-\\|{\bf U}\\|_{\text{F}}^{2}\right),$		(22)

where ${\bf U}\in\mathbb{C}^{N_{\text{Tx}}\times N_{\text{RF}}}$ is the scaled dual variable and $\rho>0$ is the augmented Lagrangian parameter. Parameter $\rho$ can be set based on the proposed convergence analyses in Section IV-B. The primal, auxiliary, and dual variables are updated as:

$\displaystyle{\bf F}_{\text{RF}}^{(\!k\!+\!1\!)}$	$\displaystyle=\operatorname*{arg\,min}_{\left[{\bf F}_{\text{RF}}\right]_{i,j}% \in\mathcal{F}}~{}\mathcal{L}({\bf{\tilde{F}}}_{\text{RF}}^{(\!k\!)},{\bf F}_{% \text{RF}},{\bf U}^{(\!k\!)})$
	$\displaystyle=\frac{1}{\sqrt{N_{\text{Tx}}}}e^{\jmath\mathcal{Q}\left(\angle({% \bf{\tilde{F}}}_{\text{RF}}^{(\!k\!)}+{\bf U}^{(\!k\!)})\right)},$	(23a)
$\displaystyle{\bf{\tilde{F}}}_{\text{RF}}^{(\!k\!+\!1\!)}$	$\displaystyle=\operatorname*{arg\,min}_{{\bf{\tilde{F}}}_{\text{RF}}}~{}% \mathcal{L}({\bf{\tilde{F}}}_{\text{RF}},{\bf F}_{\text{RF}}^{(\!k\!+\!1\!)},{% \bf U}^{(\!k\!)})$
	$\displaystyle=[{\bf F}_{\text{opt}}{\bf F}_{\text{BB}}^{\textrm{H}}\!+\!\rho({% \bf F}_{\text{RF}}^{(\!k\!+\!1\!)}\!\!-\!{\bf U}^{(\!k\!)}\!)]({\bf F}_{\text{% BB}}{\bf F}_{\text{BB}}^{\textrm{H}}\!+\!\rho{\bf I})^{-1},$	(23b)
$\displaystyle{\bf U}^{(\!k\!+\!1\!)}$	$\displaystyle={\bf U}^{(\!k\!)}+{\bf{\tilde{F}}}_{\text{RF}}^{(\!k\!+\!1\!)}-{% \bf F}_{\text{RF}}^{(\!k\!+\!1\!)}.$	(23c)

In (III-C), $\angle\cdot$ denotes the angle of its argument in an element-wise manner, and ${\mathcal{Q}}(\cdot)$ stands for the quantization function rounding its argument to the available phases of the phase shifters $\bigl{(}$ i.e., $\frac{2\pi}{2^{B}}\!\times\!\left\{0,1,\cdots,2^{B}\!-\!1\right\}$ $\bigr{)}$ .

The proposed algorithm for solving Problem (17) is referred to as AltOpt-LS-ADMM, and summarized in Algorithm 1, where superscript $\cdot^{(\!i\!)}$ denotes the corresponding variable at the $i$ -th outer iteration, superscript $\cdot^{(\!k\!)}$ denotes the corresponding variable at the $k$ -th inner (i.e., ADMM) iteration, and $I_{\text{max}}$ and $k_{\text{max}}$ are the maximal numbers of the outer and the inner loops, respectively. Besides, ${\bf F}_{\text{RF}}^{(\text{init})}$ and ${\bf{\tilde{F}}}_{\text{RF}}^{(\text{init})}$ are obtained by randomly selecting from the feasible set (8), the update of $\rho$ in Line 3 comes from (26) in Section IV-B, and ${\bf O}$ in Line 4 is an all-zeros matrix.

Algorithm 1 AltOpt-LS-ADMM for solving Problem (17)

Input : ${\bf F}_{\text{opt}}\in\mathbb{C}^{N_{\text{Tx}}\times M}$ , $I_{\text{max}}$ , $k_{\text{max}}$
Output : ${\bf F}_{\text{RF}}\in\mathbb{C}^{N_{\text{Tx}}\times N_{\text{RF}}}$ , ${\bf F}_{\text{BB}}\in\mathbb{C}^{N_{\text{RF}}\times M}$
Initialize: ${\bf F}_{\text{RF}}^{(\!0\!)}={\bf F}_{\text{RF}}^{(\text{init})}$ , $i=0$

1:while

i<I_{\text{max}}

{\bf F}_{\text{BB}}^{(\!i\!+\!1\!)}=\frac{\sqrt{P}}{\|{\bf F}_{\text{RF}}^{(\!% i\!)}{\bf F}_{\text{RF}}^{(\!i\!)\dagger}{\bf F}_{\text{opt}}\|_{\text{F}}}{% \bf F}_{\text{RF}}^{(\!i\!)\dagger}{\bf F}_{\text{opt}}

\rho=\mathrm{max}\left\{\sqrt{2}\|{\bf F}_{\text{{BB}}}^{(\!i+1\!)}({\bf F}_{% \text{{BB}}}^{(\!i+1\!)})^{\textrm{{H}}}\|_{\text{{F}}}~{}\!,\|{\bf F}_{\text{% {BB}}}^{(\!i+1\!)}\|_{\text{{F}}}^{2}\right\}

{\bf{\tilde{F}}}_{\text{RF}}^{(\!0\!)}\!=\!{\bf{\tilde{F}}}_{\text{RF}}^{(% \text{init})}

{\bf U}^{(\!0\!)}\!=\!{\bf O}

k\!=\!0

\triangleright

ADMM initialization

5: repeat

{\bf F}_{\text{RF}}^{(\!k+1\!)}\leftarrow

(III-C)

{\bf{\tilde{F}}}_{\text{RF}}^{(\!k+1\!)}\leftarrow

(III-C)

{\bf U}^{(\!k+1\!)}\leftarrow

(23c)

k=k+1

10: until

k=k_{\text{max}}

11:

{\bf F}_{\text{RF}}^{(\!i+1\!)}=\frac{1}{\sqrt{N}_{\text{Tx}}}e^{\jmath% \mathcal{Q}(\angle{\bf{\tilde{F}}}_{\text{RF}}^{(\!k\!)})}

12:

i=i+1

13:end while

14:

{\bf F}_{\text{RF}}={\bf F}_{\text{RF}}^{(\!i\!)}

and

{\bf F}_{\text{BB}}={\bf F}_{\text{BB}}^{(\!i\!)}

Remark.

It is worth mentioning that the proposed two-step strategy can find approximate (but not exact) solutions to the original optimal precoder design problem, i.e., Problem (III-B).

III-D Computational Complexity Analysis

The computational cost of the proposed AltOpt-LS-ADMM algorithm mainly comes from the pseudo-inverse operation and the multiplication operation in Line 2 and the inverse operation and the multiplication operation in Line 7, which incur the complexities $\mathcal{O}(N_{\text{RF}}^{2}N_{\text{Tx}})$ , $\mathcal{O}(N_{\text{RF}}N_{\text{Tx}}M)$ , $\mathcal{O}(N_{\text{RF}}^{3})$ , and $\mathcal{O}(N_{\text{RF}}N_{\text{Tx}}M)$ , respectively. Since we can compute the inverse operation and the multiplication operation in Line 7 outside the ADMM iteration and then use their results for all inner iterations, the total computational cost of the proposed AltOpt-LS-ADMM algorithm is $\mathcal{O}(k_{\text{max}}(N_{\text{RF}}^{2}N_{\text{Tx}}+2N_{\text{RF}}N_{% \text{Tx}}M))$ .

IV Analysis of Error Bounds and Convergence

IV-A Analysis of Quantization Error Bound

In this subsection, we analyse the quantization error bound in the proposed ADMM algorithm (i.e., inner iteration of Algorithm 1) resulting from the quantization operation in Line 6 of Algorithm 1. We first denote ${\bf{\widehat{F}}}_{\text{RF}}$ and ${\bf F}_{\text{RF}}^{\star}$ as the RF precoder with (i.e., $B<\infty$ ) and without (i.e., $B=\infty$ ) quantization, respectively. Then, the relation of these two matrices is given as

\displaystyle{\bf{\widehat{F}}}_{\text{RF}}={\bf\Phi}\odot{\bf F}_{\text{RF}}^% {\star},

(24)

where $\odot$ denotes the element-wise product and ${\bf\Phi}\in\mathbb{C}^{N_{\text{Tx}}\times N_{\text{RF}}}$ is the quantization error matrix. Moreover, the elements of ${\bf\Phi}$ can be formulated as $[{\bf\Phi}]_{ij}\!=\!e^{\jmath\phi_{ij}}$ , where $0\!\leq\!|\phi_{ij}|\!\leq\!\pi/2^{B}$ for all $1\leq i\leq N_{\text{Tx}}$ and $1\leq j\leq N_{\text{RF}}$ . Therefore, the quantization error can be calculated as

	$\displaystyle\\|{\bf F}_{\text{opt}}-{\bf{\widehat{F}}}_{\text{RF}}{\bf F}_{% \text{BB}}\\|_{\text{F}}-\\|{\bf F}_{\text{opt}}-{\bf F}_{\text{RF}}^{\star}{\bf F% }_{\text{BB}}\\|_{\text{F}}$
$\displaystyle=$	$\displaystyle~{}\!\\|{\bf F}_{\text{opt}}-({\bf\Phi}\odot{\bf F}_{\text{RF}}^{% \star}){\bf F}_{\text{BB}}\\|_{\text{F}}-\\|{\bf F}_{\text{opt}}-{\bf F}_{\text{% RF}}^{\star}{\bf F}_{\text{BB}}\\|_{\text{F}}$
$\displaystyle\leq$	$\displaystyle~{}\!\\|({\bf F}_{\text{RF}}^{\star}-({\bf\Phi}\odot{\bf F}_{\text% {RF}}^{\star})){\bf F}_{\text{BB}}\\|_{\text{F}}$	(25a)
$\displaystyle=$	$\displaystyle~{}\!\\|[({\bf 1}-{\bf\Phi})\odot{\bf F}_{\text{RF}}^{\star}]{\bf F% }_{\text{BB}}\\|_{\text{F}}$
$\displaystyle\leq$	$\displaystyle~{}\!\\|({\bf 1}-{\bf\Phi})\odot{\bf F}_{\text{RF}}^{\star}\\|_{% \text{F}}\\|{\bf F}_{\text{BB}}\\|_{\text{F}}$	(25b)
$\displaystyle\leq$	$\displaystyle~{}\!\\|{\bf 1}-{\bf\Phi}\\|_{\text{F}}\\|{\bf F}_{\text{RF}}^{\star% }\\|_{\text{F}}\\|{\bf F}_{\text{BB}}\\|_{\text{F}}$	(25c)
$\displaystyle\leq$	$\displaystyle~{}\!\left\|1-e^{\jmath{\pi}/{2^{B}}}\right\|\sqrt{N_{\text{Tx}}N_{% \text{RF}}}\\|{\bf F}_{\text{RF}}^{\star}\\|_{\text{F}}\\|{\bf F}_{\text{BB}}\\|_{% \text{F}},$	(25d)

where ${\bf 1}$ is the all-ones matrix of appropriate size, in (25a) we used the triangle inequality, in (25b) we employed the fact that $\|{\bf M}{\bf N}\|_{\text{F}}\leq\|{\bf M}\|_{\text{F}}\|{\bf N}\|_{\text{F}}$ holds for any matrices ${\bf M}$ and ${\bf N}$ of appropriate sizes, in (25c) we utilized the Cauchy-Schwarz inequality, and in (25d) we used the following inequality:

	$\displaystyle\\|{\bf 1}-{\bf\Phi}\\|_{\text{F}}=$	$\displaystyle~{}\!\sqrt{\sum_{i=1}^{N_{\text{Tx}}}\sum_{j=1}^{N_{\text{RF}}}% \left\|1-e^{\jmath\phi_{ij}}\right\|^{2}}$
	$\displaystyle\leq$	$\displaystyle~{}\!\sqrt{\sum_{i=1}^{N_{\text{Tx}}}\sum_{j=1}^{N_{\text{RF}}}% \left\|1-e^{\jmath\pi/2^{B}}\right\|^{2}}$
	$\displaystyle=$	$\displaystyle~{}\!\left\|1-e^{\jmath{\pi}/{2^{B}}}\right\|\sqrt{N_{\text{Tx}}N_{% \text{RF}}}~{}\!.$

Note that $\sqrt{N_{\text{Tx}}N_{\text{RF}}}\|{\bf F}_{\text{RF}}^{\star}\|_{\text{F}}\|{% \bf F}_{\text{BB}}\|_{\text{F}}$ in (25d) is a constant w.r.t. the number of quantization bits of phase shifters. For notational simplicity, we define $C\triangleq\sqrt{N_{\text{Tx}}N_{\text{RF}}}\|{\bf F}_{\text{RF}}^{\star}\|_{% \text{F}}\|{\bf F}_{\text{BB}}\|_{\text{F}}$ , and rewrite the quantization error as

\displaystyle\|{\bf F}_{\text{opt}}-{\bf{\widehat{F}}}_{\text{RF}}{\bf F}_{% \text{BB}}\|_{\text{F}}-\|{\bf F}_{\text{opt}}-{\bf F}_{\text{RF}}^{\star}{\bf F% }_{\text{BB}}\|_{\text{F}}\leq C\left|1-e^{\jmath{\pi}/{2^{B}}}\right|.

The values of $\left|1-e^{\jmath{\pi}/{2^{B}}}\right|$ with different numbers of quantization bits are presented in Table I. It is seen from Table I that when $B\geq 5$ , the quantization upper bound decreases by more than 10 times compared to $B=1$ , suggesting that $B=5$ can be sufficient to approach the performance of infinite-resolution phase shifters. In order to illustrate the impact of quantization bits $B$ on the decomposition error upper bound (DecpUB), we define

\displaystyle{\text{DecpUB}}\triangleq\|{\bf F}_{\text{opt}}-{\bf F}_{\text{RF% }}^{\star}{\bf F}_{\text{BB}}\|_{\text{F}}+C\left|1-e^{\jmath{\pi}/{2^{B}}}% \right|,

and then plot it w.r.t. the number of quantization bits in Fig. 2, where the simulation parameters are $N_{\text{Tx}}=16$ , $M=20$ , and $P=10$ dBm. It can be observed from Fig. 2 that when $B\leq 3$ the DecpUB decrease sharply, and when $B\geq 5$ the slopes of the DecpUB are approximately equal to 0. This leads to the same conclusion as the one drawn from Table I, that is, $B=5$ is sufficient to approach the performance of infinite-resolution phase shifters. This will be further verified through simulations in Section V.

IV-B Convergence Analysis

In this subsection, we analyse the convergence behaviors of the proposed ADMM algorithm (i.e., the inner iteration of Algorithm 1) and AltOpt-LS-ADMM algorithm (i.e., the outer iteration of Algorithm 1), which are stated in the following two theorems, respectively.

Theorem 1.

The augmented Lagrangian function value sequence $\left\{\mathcal{L}\left({\bf{\tilde{F}}}_{\text{\emph{RF}}}^{(k)},{\bf F}_{% \text{\emph{RF}}}^{(k)},{\bf U}^{(k)}\right)\Big{|}k=0,1,2,\cdots\right\}$ produced by the proposed ADMM algorithm converges if

\displaystyle\rho\geq\mathrm{max}\left\{\sqrt{2}\|{\bf F}_{\text{\emph{BB}}}{% \bf F}_{\text{\emph{BB}}}^{\textrm{\emph{H}}}\|_{\text{\emph{F}}},\|{\bf F}_{% \text{\emph{BB}}}\|_{\text{\emph{F}}}^{2}\right\}.

(26)

Furthermore, as $k\to\infty$ , we have ${\bf F}_{\text{\emph{RF}}}^{(k+1)}={\bf F}_{\text{\emph{RF}}}^{(k)}$ , ${\bf{\tilde{F}}}_{\text{\emph{RF}}}^{(k+1)}={\bf{\tilde{F}}}_{\text{\emph{RF}}% }^{(k)}$ , ${\bf U}^{(k+1)}={\bf U}^{(k)}$ , and ${\bf F}_{\text{\emph{RF}}}^{(k)}={\bf{\tilde{F}}}_{\text{\emph{RF}}}^{(k)}$ ; and the point sequence $\left\{\left({\bf{\tilde{F}}}_{\text{\emph{RF}}}^{(k)},{\bf F}_{\text{\emph{RF% }}}^{(k)},{\bf U}^{(k)}\right)\right\}$ is a Cauchy sequence and it converges to a fixed point after a finite number of iterations.

Proof.

See Appendix C. ∎

Theorem 2.

If (26) holds, the sequence $\left\{\|{\bf F}_{\text{\emph{opt}}}-{\bf F}_{\text{\emph{RF}}}^{(i)}{\bf F}_{% \text{\emph{BB}}}^{(i)}\|_{\text{\emph{F}}}\right\}$ generated by the proposed AltOpt-LS-ADMM algorithm converges.

Proof.

See Appendix D. ∎

Theorem 1 asserts that as long as the augmented Lagrangian parameter $\rho$ is large enough (see (26)), the proposed ADMM algorithm is convergent. Additionally, Theorem 2 establishes that the proposed AltOpt-LS-ADMM can generate a convergent sequence of cost function values, as long as (26) holds.

TABLE I: Quantization error bound with different

B

$B$	1	2	3	4	5	6
$\left\|1-e^{\jmath{\pi}/{2^{B}}}\right\|$	1.4142	0.7654	0.3902	0.1960	0.0981	0.0491

V Numerical Results

V-A Scenario, Performance Metric, and Benchmark

In this section, we conduct simulations to verify the performance of the proposed AltOpt-LS-ADMM algorithm. Two scenarios are considered as follows:

•

Scenario I: We randomly generate a digital precoder ${\bf F}_{\text{opt}}$ , and our performance metric is the decomposition error (DecpErr), defined as: ${\|{\bf F}_{\text{opt}}-{\bf F}_{\text{RF}}{\bf F}_{\text{BB}}\|_{\text{F}}}/{% \|{\bf F}_{\text{opt}}\|_{\text{F}}}$ . The simulation parameters are summarized in Table II.
•

Scenario II: We obtain an optimal digital precoder ${\bf F}_{\text{opt}}$ as introduced in Step 1 in Section III-C, and our performance metric is the AEB in (6). The simulation parameters are summarized in Table III.

TABLE II: Simulation Parameters in Scenario I

Parameter	Scenario I
$I_{\text{max}}$	$10$
$k_{\text{max}}$	$50$
BS Transmit Power, $P$	$10$ dBm
Pilot Length, $M$	$20$
$N_{\text{Tx}}$	$16$
$N_{\text{RF}}$	$1,2,\cdots,16$ (Figs. 3 and 4);
	$1,2,4,7,10,13,16$ (Fig. 5);
	$8$ (Fig. 6)
$B$	$1,2,3,4,5,\infty$ (Fig. 3);
	2 (Fig. 4);
	$1,2,\cdots,16$ (Figs. 5 and 6)

TABLE III: Simulation Parameters in Scenario II

Parameter	Scenario II
$I_{\text{max}}$	$10$
$k_{\text{max}}$	$50$
BS Transmit Power, $P$	$10$ dBm
Signal-to-Noise Ratio (SNR)	$10$ dB
Pilot Length, $M$	$20$
$N_{\text{Tx}}$	$4,6,\cdots,30$ (Figs. 7 and 11);
$N_{\text{Tx}}$	$20$ (Figs. 8, 9, and 10)
$N_{\text{RF}}$	$1,2,\cdots,20$ (Figs. 8 and 11);
$N_{\text{RF}}$	$4$ (Figs. 7, 9, and 10)
$B$	$1,2,\cdots,16$ (Fig. 9);
$B$	$\infty$ (Figs. 7, 8, and 10)
AoD	$-80^{\circ},-75^{\circ},\cdots,80^{\circ}$ (Fig. 10);
AoD	$0^{\circ}$ (Figs. 7, 8, and 9)

We compare the proposed method with the following methods:

•

Alt-Babai [18]: alternating optimization + the Babai algorithm
•

Alt-CDM [19]: alternating optimization + coordinate descent method
•

Spa-OMP [24]: spatially sparse representation + orthogonal matching pursuit
•

ManiOpt [29]: manifold optimization (where we utilize the Manopt function [57] for implementation)

Note that ManiOpt in [29] utilizes infinite-resolution (i.e., $B=\infty$ ) phase shifters, which is adopted as a benchmark in this work. Also note that the ManiOpt is initialized with random value or with the output of the proposed AltOpt-LS-ADMM method, labelled as “ManiOpt (random initialization)” and “ManiOpt (proposed initialization)”, respectively.

V-B Results and Discussion of Scenario I

V-B1 DecpErr as a Function of $N_{\text{\emph{RF}}}$

We randomly generate an ${\bf F}_{\text{opt}}$ and decompose it into ${\bf F}_{\text{RF}}$ and ${\bf F}_{\text{BB}}$ by using the proposed AltOpt-LS-ADMM algorithm. The DecpErrs are averaged over 500 Monte-Carlo trials, and the results w.r.t. the number of RF chains, $N_{\text{RF}}$ , are plotted in Fig. 3. It can be seen that: (i) when $N_{\text{RF}}$ or $B$ increases, the DecpErr decreases; (ii) when $B=5$ , its performance approaches the one with $B=\infty$ (i.e., the infinite resolution phase shifter); (iii) when $N_{\text{RF}}=N_{\text{Tx}}=16$ , the DecpErrs are always 0. This is because when $N_{\text{RF}}=N_{\text{Tx}}$ , ${\bf F}_{\text{RF}}$ is a square matrix and invertible, and thus there always exists a matrix ${\bf F}_{\text{BB}}={\bf F}_{\text{RF}}^{-1}{\bf F}_{\text{opt}}$ such that ${\bf F}_{\text{opt}}={\bf F}_{\text{RF}}{\bf F}_{\text{BB}}$ .

Next, the DecpErrs of different methods are displayed in Fig. 4, with $B=2$ . We see that ManiOpt with random initialization has the worst performance and ManiOpt with the proposed method as initialization achieves the highest performance. Besides, when $N_{\text{RF}}\geq 7$ , the decomposition error of the proposed method is larger than that of ManiOpt with proposed initilization, and smaller than those of Alt-Babai, Alt-CDM, Spa-OMP, and ManiOpt with random initialization. Note that ManiOpt with proposed initialization attains the best decomposition performance at a cost of higher computational complexity, which will be verified in Fig. 11.

V-B2 DecpErr as a Function of $B$

The DecpErr of the proposed algorithm w.r.t. the number of quantization bits, $B$ , is shown in Fig. 5. We observe that when $N_{\text{RF}}$ increases, the decomposition error decreases, as expected. For $N_{\text{RF}}<16$ , when $B$ increases from 1 until 5, the decomposition error decreases; when $B\geq 5$ , the decomposition error remains nearly unchanged. Therefore, taking into account the outcomes presented in Fig. 3, we can infer that $B=5$ bits prove to be sufficient in achieving near-optimal hybrid precoding performance (i.e., reaching a performance level very close to that obtained by digital precoding). Besides, it is seen from Fig. 5 that, the decomposition error is 0 when $N_{\text{RF}}=N_{\text{Tx}}=16$ , which has been explained in the first example.

Next, the DecpErrs of different algorithms are depicted in Fig. 6, with $N_{\text{RF}}=8$ , which verifies the better performance of the proposed method against Alt-Babai, Alt-CDM, Spa-OMP, and ManiOpt with random initialization, especially when number of bits $B\geq 2$ . Note that the ManiOpt with random initialization has a horizontal line because it uses $B=\infty$ quantization bits; while ManiOpt with proposed initialization does not has a horizontal line because it is sensitive to its initialization (its initialization, i.e., the proposed method, has better performance as $B$ increases from 1 until 5).

V-C Results and Discussion of Scenario II

V-C1 AEB as a Function of $N_{\text{\emph{Tx}}}$

We now evaluate the AEB performance of the proposed algorithm and the benchmark methods. The optimal digital precoder ${\bf F}_{\text{opt}}$ can be obtained by the method³³3Since our main focus is the decomposition step, i.e., Step 2, in Section III-C, in the following simulations, ${\bf F}_{\text{opt}}$ is obtained heuristically by assigning $0.1$ of power to derivative beams and the rest to directional beams. as introduced in Section III-C. Then we decompose this ${\bf F}_{\text{opt}}$ into ${\bf F}_{\text{RF}}$ and ${\bf F}_{\text{BB}}$ by using different algorithms. The AEBs w.r.t. the number of Tx antennas, $N_{\text{Tx}}$ , achieved by the different methods are shown in Fig. 7, where the curve labelled as “Optimal (fully digital)” is the result by using ${\bf F}_{\text{opt}}$ directly without decomposition. We can see that the proposed algorithm attains lower AEB than those of Alt-Babai, Alt-CDM, Spa-OMP, and ManiOpt with random initialization. The ManiOpt with proposed initialization outperforms the others and is much closer to the “Optimal (fully digital)” curve.

V-C2 AEB as a Function of $N_{\text{\emph{RF}}}$

The results of AEB w.r.t. the number of RF chains, $N_{\text{RF}}$ , are presented in Fig. 8. It can be observed from Fig. 8 that the AEB of the proposed algorithm is smaller than those of Alt-Babai, Alt-CDM, Spa-OMP, and ManiOpt with random initilization, while ManiOpt with proposed initialization has the best performance.

V-C3 AEB as a Function of $B$

We compare AEB versus different quantization bits of phase shifters, and the results are displayed in Fig. 9. We have similar findings as in Fig. 6.

V-C4 AEB as a Function of AoD

The predefined codebook for ${\bf F}_{\text{opt}}$ is set around $0^{\circ}$ , while the true AoD changes from $-80^{\circ}$ to $80^{\circ}$ . The results of AEB versus AoD are plotted in Fig. 10. We see that when AoD is $0^{\circ}$ (matching with our predefined codebook), all the curves reach their lowest AEB. Besides, the ManiOpt with proposed initialization outperforms others, followed by the proposed AltOpt-LS-ADMM algorithm. In addition, the proposed algorithm for hybrid precoder design, both when used independently and as an initialization for ManiOpt, exhibits AoD estimation performance very close to that achieved via a fully digital array, which underscores its effectiveness over a broad range of AoD values.

V-C5 CPU Runtime as a Function of $N_{\text{\emph{Tx}}}$ or $N_{\text{\emph{RF}}}$

We compare the computational complexity of different algorithms. The central processing unit (CPU) runtime versus number of transmit antennas is drawn in Fig. 11 (left). On the other hand, the CPU runtime versus number of RF chains is drawn in Fig. 11 (right). From Fig. 11, it can be seen that ManiOpt with random initialization and ManiOpt with proposed initialization have almost the same CPU runtime. The Optimal method has the least CPU runtime since it does not need to perform the decomposition operation on ${\bf F}_{\text{opt}}$ . Besides, the proposed algorithm consumes less CPU runtime than Alt-Babai, Alt-CDM, Spa-OMP, and ManiOpt.

VI Conclusion

In this paper, we have investigated the hybrid precoder design problem for angle-of-departure (AoD) estimation, where we took into account practical limitation on the finite resolution of phase shifters. Our aim was to devise a radio-frequency (RF) precoder and a base-band (BB) precoder that could simultaneously adhere to the practical constraint and achieve highly precise AoD estimation. To accomplish this goal, we developed a two-step approach. Firstly, we derived a fully digital precoder that minimizes the angle error bound by using a predefined codebook. Then, we decomposed this digital precoder into an RF precoder and a BB precoder, employing the alternating optimization framework and alternating direction method of multipliers. We also analysed the quantization error bound, and provided convergence analyses of the proposed algorithm. Numerical results demonstrated the exceptional performance of the proposed method with low complexity, leading to the following key conclusions:

•

Number of Bits Sufficient for AoD Estimation: 5 bits are sufficient to achieve almost the same decomposition and AoD estimation performance as the case with infinite-resolution phase shifters.
•

Number of RF Chains Sufficient for AoD Estimation: For a 20-element transmit array, 4 RF chains are sufficient to attain the same AoD estimation performance as the fully-digital architecture.
•

High-Quality Initialization: The proposed algorithm can provide high-quality initialization that boosts the performance of manifold optimization compared to random initialization.
•

Covering Wide Range of AoDs: The proposed algorithm attains near-optimal (in the sense of achieving the fully-digital performance) AoD performance over a broad range of AoD values ranging from -80 to 80 degrees.

Appendix A Calculation of the FIM

The derivatives of ${\bf{\tilde{y}}}=\beta{\bf S}({\bf F}_{\text{RF}}{\bf F}_{\text{BB}})^{\textrm% {T}}{\bf a}(\theta)$ w.r.t. $[{\bf x}]_{i}$ , $i=1,2,3$ , are given as follows

	$\displaystyle\frac{\partial{\bf{\tilde{y}}}}{\partial[{\bf x}]_{1}}$	$\displaystyle=\frac{\partial{\bf{\tilde{y}}}}{\partial\sin{\theta}}=-\jmath% \frac{2\beta\pi d}{\lambda}{\bf S}({\bf F}_{\text{RF}}{\bf F}_{\text{BB}})^{% \textrm{T}}{\bf D}{\bf a}(\theta),$
	$\displaystyle\frac{\partial{\bf{\tilde{y}}}}{\partial[{\bf x}]_{2}}$	$\displaystyle=\frac{\partial{\bf{\tilde{y}}}}{\partial\beta_{\text{R}}}={\bf S% }({\bf F}_{\text{RF}}{\bf F}_{\text{BB}})^{\textrm{T}}{\bf a}(\theta),$
	$\displaystyle\frac{\partial{\bf{\tilde{y}}}}{\partial[{\bf x}]_{3}}$	$\displaystyle=\frac{\partial{\bf{\tilde{y}}}}{\partial\beta_{\text{I}}}=\jmath% {\bf S}({\bf F}_{\text{RF}}{\bf F}_{\text{BB}})^{\textrm{T}}{\bf a}(\theta),$

where ${\bf D}\triangleq\text{diag}\{0,1,\cdots,N_{\text{Tx}-1}\}$ . According to (4), we have

	$\displaystyle[{\bf J}]_{11}$	$\displaystyle=\frac{8\|\beta\|^{2}\pi^{2}d^{2}}{\sigma_{\text{n}}^{2}\lambda^{2}% }{\bf a}^{\textrm{H}}(\theta){\bf D}{\bf F}^{*}{\bf S}^{\textrm{H}}{\bf S}{\bf F% }^{\textrm{T}}{\bf D}{\bf a}(\theta),$
	$\displaystyle[{\bf J}]_{12}$	$\displaystyle=[{\bf J}]_{21}=\frac{4\pi d}{\sigma_{\text{n}}^{2}\lambda}\times% \Re\left\{\jmath\beta^{}{\bf a}^{\textrm{H}}(\theta){\bf D}{\bf F}^{}{\bf S}% ^{\textrm{H}}{\bf S}{\bf F}^{\textrm{T}}{\bf a}(\theta)\right\},$
	$\displaystyle[{\bf J}]_{13}$	$\displaystyle=[{\bf J}]_{31}=-\frac{4\pi d}{\sigma_{\text{n}}^{2}\lambda}% \times\Re\left\{\beta^{}{\bf a}^{\textrm{H}}(\theta){\bf D}{\bf F}^{}{\bf S}% ^{\textrm{H}}{\bf S}{\bf F}^{\textrm{T}}{\bf a}(\theta)\right\},$
	$\displaystyle[{\bf J}]_{22}$	$\displaystyle=[{\bf J}]_{33}=\frac{2}{\sigma_{\text{n}}^{2}}{\bf a}^{\textrm{H% }}(\theta){\bf F}^{*}{\bf S}^{\textrm{H}}{\bf S}{\bf F}^{\textrm{T}}{\bf a}(% \theta),$
	$\displaystyle[{\bf J}]_{23}$	$\displaystyle=[{\bf J}]_{32}=0.$

Appendix B Calculation of the AEB

Based on the block matrix inversion lemma [55], we have

		$\displaystyle\text{AEB}({\bf F}_{\text{RF}},{\bf F}_{\text{BB}};{\bf x})$
	$\displaystyle=$	$\displaystyle~{}\!\left(\![{\bf J}]_{11}-\bigg{[}[{\bf J}]_{12},[{\bf J}]_{13}% \bigg{]}\!\!\left[\!\!\!\begin{array}[]{cc}[{\bf J}]_{22}&0\\ 0&[{\bf J}]_{33}\end{array}\!\!\!\right]^{-1}\!\left[\!\!\begin{array}[]{c}% \left[{\bf J}\right]_{21}\\ \left[{\bf J}\right]_{31}\end{array}\!\!\right]\!\right)^{-\frac{1}{2}}$
	$\displaystyle=$	$\displaystyle~{}\!\left([{\bf J}]_{11}-\frac{(\left[{\bf J}\right]_{12})^{2}+(% [{\bf J}]_{13})^{2}}{[{\bf J}]_{22}}\right)^{-\frac{1}{2}}.$

Substituting the results in Appendix A and ${\bf S}^{\textrm{H}}{\bf S}=\sigma_{\text{s}}^{2}{\bf I}_{M}$ into the above equation yields (6).

Appendix C Proof of Theorem 1

To show the convergence of $\left\{\mathcal{L}\!\left({\bf{\tilde{F}}}_{\text{RF}}^{(k)},{\bf F}_{\text{RF% }}^{(k)},{\bf U}^{(k)}\right)\right\}$ , we first provide the following two lemmas:

Lemma 1.

The proposed AltOpt-LS-ADMM algorithm, i.e., Algorithm 1, produces a monotonically decreasing sequence $\{\mathcal{L}^{(k)}|k=0,1,2,\cdots\}$ , where $\mathcal{L}^{(k)}\triangleq\mathcal{L}\left({\bf{\tilde{F}}}_{\text{\emph{RF}}% }^{(k)},{\bf F}_{\text{\emph{RF}}}^{(k)},{\bf U}^{(k)}\right)$ , provided that the augmented Lagrangian parameter $\rho$ satisfies

\displaystyle\rho\geq\sqrt{2}\|{\bf F}_{\text{\emph{BB}}}{\bf F}_{\text{\emph{% BB}}}^{\textrm{\emph{H}}}\|_{\text{\emph{F}}}.

(30)

Lemma 2.

The function $\mathcal{L}\left({\bf{\tilde{F}}}_{\text{\emph{RF}}},{\bf F}_{\text{\emph{RF}}% },{\bf U}\right)$ defined in (III-C) is bounded from below by 0 during the iteration process (III-C), provided that the augmented Lagrangian parameter $\rho$ satisfies

\displaystyle\rho\geq\|{\bf F}_{\text{\emph{BB}}}\|_{\text{\emph{F}}}^{2}.

(31)

The proofs of Lemmas 1 and 2 are relegated to Appendix E and Appendix F, respectively. These two lemmas straightforwardly implies that the sequence $\left\{\mathcal{L}\!\left({\bf{\tilde{F}}}_{\text{RF}}^{(k)},{\bf F}_{\text{RF% }}^{(k)},{\bf U}^{(k)}\right)\right\}$ is convergent. Therefore, when the augmented Lagrangian parameter $\rho$ satisfies (26), we have

\displaystyle\mathcal{L}^{(k+1)}-\mathcal{L}^{(k)}=0

(32)

as $k\to\infty$ . On the other hand, it is showed that if $\rho\geq\|{\bf F}_{\text{BB}}^{\textrm{H}}\|_{\text{F}}^{2}$ ,

\displaystyle\mathcal{L}^{(k+1)}-\mathcal{L}^{(k)}\leq(\text{i})\times\left\|{% \bf{\tilde{F}}}_{\text{RF}}^{(k+1)}-{\bf{\tilde{F}}}_{\text{RF}}^{(k)}\right\|% _{\text{F}}^{2}\leq 0,

(33)

where the term $(\text{i})$ is defined in Appendix E. Combining (32) and (33) leads to

\displaystyle{\bf{\tilde{F}}}_{\text{RF}}^{(k+1)}={\bf{\tilde{F}}}_{\text{RF}}% ^{(k)}.

(34)

The above equation together with

\displaystyle{\bf U}=\frac{1}{\rho}\left({\bf F}_{\text{opt}}-{\bf{\tilde{F}}}% _{\text{RF}}{\bf F}_{\text{BB}}\right){\bf F}_{\text{BB}}^{\textrm{H}}

(35)

(which is the result of combining (III-C) and (23c)) yields

\displaystyle{\bf U}^{(k+1)}={\bf U}^{(k)}.

(36)

Since ${\bf F}_{\text{RF}}$ is calculated based on ${\bf{\tilde{F}}}_{\text{RF}}$ and ${\bf U}$ (see Line 6 in Algorithm 1), (34) and (36) yields

\displaystyle{\bf F}_{\text{RF}}^{(k+1)}={\bf F}_{\text{RF}}^{(k)}.

(37)

Further, according to Line 8 in Algorithm 1, we have

\displaystyle{\bf F}_{\text{RF}}^{(k)}={\bf{\tilde{F}}}_{\text{RF}}^{(k)}.

(38)

On the other hand, from (34) we have $\left\|{\bf{\tilde{F}}}_{\text{RF}}^{(k+1)}\!-\!{\bf{\tilde{F}}}_{\text{RF}}^{% (k)}\!\right\|_{\text{F}}\to 0$ as $k\to\infty$ . This leads to the fact that: for any positive $\epsilon$ , there always exists an integer $T$ (large enough), such that

		$\displaystyle\left\\|{\bf{\tilde{F}}}_{\text{RF}}^{(k_{1})}\!-\!{\bf{\tilde{F}}% }_{\text{RF}}^{(k_{2})}\!\right\\|_{\text{F}}$
	$\displaystyle=$	$\displaystyle\left\\|{\bf{\tilde{F}}}_{\text{RF}}^{(k_{1})}\!\!-\!{\bf{\tilde{F% }}}_{\text{RF}}^{(k_{1}+1)}\!+\!{\bf{\tilde{F}}}_{\text{RF}}^{(k_{1}+1)}\!\!-% \!{\bf{\tilde{F}}}_{\text{RF}}^{(k_{1}+2)}\!+\!\cdots\!+\!{\bf{\tilde{F}}}_{% \text{RF}}^{(k_{2}-1)}\!\!-\!{\bf{\tilde{F}}}_{\text{RF}}^{(k_{2})}\!\right\\|_% {\text{F}}$
	$\displaystyle\leq$	$\displaystyle\left\\|{\bf{\tilde{F}}}_{\text{RF}}^{(k_{1})}\!\!-\!{\bf{\tilde{F% }}}_{\text{RF}}^{(k_{1}+1)}\right\\|_{\text{F}}+\left\\|{\bf{\tilde{F}}}_{\text{% RF}}^{(k_{1}+1)}\!\!-\!{\bf{\tilde{F}}}_{\text{RF}}^{(k_{1}+2)}\right\\|_{\text% {F}}+\cdots$
		$\displaystyle\cdots+\left\\|{\bf{\tilde{F}}}_{\text{RF}}^{(k_{2}-1)}\!\!-\!{\bf% {\tilde{F}}}_{\text{RF}}^{(k_{2})}\right\\|_{\text{F}}$
	$\displaystyle\leq$	$\displaystyle~{}\!\epsilon$

holds for all $k_{1}$ , $k_{2}$ $\geq T$ (without loss of generality we assume $k_{2}>k_{1}$ in the above inequalities). This indicates that sequence $\left\{\!{\bf{\tilde{F}}}_{\text{RF}}^{(k)}\!\right\}$ is a Cauchy sequence, and thus it converges to a fixed point after a finite number (i.e., $T$ ) of iterations [58]. Similarly, both sequences $\left\{\!{\bf{U}}^{(k)}\!\right\}$ and $\left\{\!{\bf{F}}_{\text{RF}}^{(k)}\!\right\}$ are Cauchy sequences and they converge to fixed points after $T$ iterations, thanks to (35) and Line 6 in Algorithm 1. This completes the proof of Theorem 1.

Appendix D Proof of Theorem 2

Since the proposed AltOpt-LS-ADMM algorithm, i.e., Algorithm 1, has unique optimal solutions for both ${\bf F}_{\text{BB}}$ (see (19)) and ${\bf F}_{\text{RF}}$ (see Theorem 1) at each iteration, we have

	$\displaystyle\\|{\bf F}_{\text{opt}}-{\bf F}_{\text{RF}}^{(i+1)}{\bf F}_{\text{% BB}}^{(i+1)}\\|_{\text{F}}\leq$	$\displaystyle~{}\!\\|{\bf F}_{\text{opt}}-{\bf F}_{\text{RF}}^{(i)}{\bf F}_{% \text{BB}}^{(i+1)}\\|_{\text{F}}$
	$\displaystyle\leq$	$\displaystyle~{}\!\\|{\bf F}_{\text{opt}}-{\bf F}_{\text{RF}}^{(i)}{\bf F}_{% \text{BB}}^{(i)}\\|_{\text{F}},$

which shows that sequence $\left\{\|{\bf F}_{\text{opt}}-{\bf F}_{\text{RF}}^{(i)}{\bf F}_{\text{BB}}^{(i% )}\|_{\text{F}}\right\}$ is monotonically decreasing. On the other hand, it is straightforward to see that $\|{\bf F}_{\text{opt}}-{\bf F}_{\text{RF}}^{(i)}{\bf F}_{\text{BB}}^{(i)}\|_{% \text{F}}$ is bounded from below by 0. This indicates that sequence $\left\{\|{\bf F}_{\text{opt}}-{\bf F}_{\text{RF}}^{(i)}{\bf F}_{\text{BB}}^{(i% )}\|_{\text{F}}\right\}$ generated by the proposed algorithm converges. This completes the proof of Theorem 2.

Appendix E Proof of Lemma 1

The difference between the augmented Lagrangian function values at two successive iterations is calculated as

	$\displaystyle\mathcal{L}\!\left({\bf{\tilde{F}}}_{\text{RF}}^{(k+1)},{\bf F}_{% \text{RF}}^{(k+1)},{\bf U}^{(k+1)}\right)-\mathcal{L}\!\left({\bf{\tilde{F}}}_% {\text{RF}}^{(k)},{\bf F}_{\text{RF}}^{(k)},{\bf U}^{(k)}\right)$
$\displaystyle=$	$\displaystyle\left[\mathcal{L}\!\left({\bf{\tilde{F}}}_{\text{RF}}^{(k+1)},{% \bf F}_{\text{RF}}^{(k+1)},{\bf U}^{(k+1)}\right)\!-\!\mathcal{L}\!\left({\bf{% \tilde{F}}}_{\text{RF}}^{(k+1)},{\bf F}_{\text{RF}}^{(k+1)},{\bf U}^{(k)}% \right)\right]$
	$\displaystyle+\left[\mathcal{L}\!\left({\bf{\tilde{F}}}_{\text{RF}}^{(k+1)},{% \bf F}_{\text{RF}}^{(k+1)},{\bf U}^{(k)}\right)\!-\!\mathcal{L}\!\left({\bf{% \tilde{F}}}_{\text{RF}}^{(k)},{\bf F}_{\text{RF}}^{(k+1)},{\bf U}^{(k)}\right)\right]$
	$\displaystyle+\left[\mathcal{L}\!\left({\bf{\tilde{F}}}_{\text{RF}}^{(k)},{\bf F% }_{\text{RF}}^{(k+1)},{\bf U}^{(k)}\right)\!-\!\mathcal{L}\!\left({\bf{\tilde{% F}}}_{\text{RF}}^{(k)},{\bf F}_{\text{RF}}^{(k)},{\bf U}^{(k)}\right)\right].$	(39)

The three terms in the above three square brackets are respectively calculated as follows. The first term is bounded as

	$\displaystyle\mathcal{L}\!\left({\bf{\tilde{F}}}_{\text{RF}}^{(k+1)},{\bf F}_{% \text{RF}}^{(k+1)},{\bf U}^{(k+1)}\right)-\mathcal{L}\!\left({\bf{\tilde{F}}}_% {\text{RF}}^{(k+1)},{\bf F}_{\text{RF}}^{(k+1)},{\bf U}^{(k)}\right)$
$\displaystyle=$	$\displaystyle~{}\!\frac{\rho}{2}\left(\left\\|{\bf{\tilde{F}}}_{\text{RF}}^{(k+% 1)}-{\bf F}_{\text{RF}}^{(k+1)}+{\bf U}^{(k+1)}\right\\|_{\text{F}}^{2}-\left\\|% {\bf U}^{(k+1)}\right\\|_{\text{F}}^{2}\right)$
	$\displaystyle-\frac{\rho}{2}\left(\left\\|{\bf{\tilde{F}}}_{\text{RF}}^{(k+1)}-% {\bf F}_{\text{RF}}^{(k+1)}+{\bf U}^{(k)}\right\\|_{\text{F}}^{2}-\left\\|{\bf U% }^{(k)}\right\\|_{\text{F}}^{2}\right)$	(40a)
$\displaystyle=$	$\displaystyle~{}\!\frac{\rho}{2}\left(\left\\|2{\bf U}^{(k+1)}\!-\!{\bf U}^{(k)% }\right\\|_{\text{F}}^{2}\!-\!2\left\\|{\bf U}^{(k+1)}\right\\|_{\text{F}}^{2}\!+% \!\left\\|{\bf U}^{(k)}\right\\|_{\text{F}}^{2}\right)$	(40b)
$\displaystyle=$	$\displaystyle~{}\!\rho\left\\|{\bf U}^{(k+1)}-{\bf U}^{(k)}\right\\|_{\text{F}}^% {2}$
$\displaystyle=$	$\displaystyle~{}\!\frac{1}{\rho}\left\\|\!\left(\!{\bf F}_{\text{opt}}\!-\!{\bf% {\tilde{F}}}_{\text{RF}}^{(k+1)}{\bf F}_{\text{BB}}\!\!\right)\!\!{\bf F}_{% \text{BB}}^{\textrm{H}}\!-\!\left(\!{\bf F}_{\text{opt}}\!-\!{\bf{\tilde{F}}}_% {\text{RF}}^{(k)}{\bf F}_{\text{BB}}\!\!\right)\!\!{\bf F}_{\text{BB}}^{% \textrm{H}}\right\\|_{\text{F}}^{2}$	(40c)
$\displaystyle=$	$\displaystyle~{}\!\frac{1}{\rho}\left\\|\left({\bf{\tilde{F}}}_{\text{RF}}^{(k)% }-{\bf{\tilde{F}}}_{\text{RF}}^{(k+1)}\right){\bf F}_{\text{BB}}{\bf F}_{\text% {BB}}^{\textrm{H}}\right\\|_{\text{F}}^{2}$
$\displaystyle\leq$	$\displaystyle~{}\!\frac{1}{\rho}\left\\|{\bf F}_{\text{BB}}{\bf F}_{\text{BB}}^% {\textrm{H}}\right\\|_{\text{F}}^{2}\left\\|{\bf{\tilde{F}}}_{\text{RF}}^{(k+1)}% -{\bf{\tilde{F}}}_{\text{RF}}^{(k)}\right\\|_{\text{F}}^{2},$	(40d)

where in (40a) we used the definition of $\mathcal{L}\Big{(}{\bf{\tilde{F}}}_{\text{RF}},{\bf F}_{\text{RF}},{\bf U}\Big% {)}$ ; in (40b) we employed ${\bf{\tilde{F}}}_{\text{RF}}^{(k+1)}-{\bf F}_{\text{RF}}^{(k+1)}={\bf U}^{(k+1% )}-{\bf U}^{(k)}$ (due to (23c)); in (40c) we utilized (35); in (40d) we used the fact that $\|{\bf M}{\bf N}\|_{\text{F}}\leq\|{\bf M}\|_{\text{F}}\|{\bf N}\|_{\text{F}}$ holds for any matrices ${\bf M}$ and ${\bf N}$ of appropriate sizes. The second term is bounded as

	$\displaystyle\mathcal{L}\!\left({\bf{\tilde{F}}}_{\text{RF}}^{(k+1)},{\bf F}_{% \text{RF}}^{(k+1)},{\bf U}^{(k)}\right)-\mathcal{L}\!\left({\bf{\tilde{F}}}_{% \text{RF}}^{(k)},{\bf F}_{\text{RF}}^{(k+1)},{\bf U}^{(k)}\right)$
$\displaystyle\leq$	$\displaystyle~{}\!\Re\!\left\{\!\Big{\langle}\nabla_{{\bf{\tilde{F}}}_{\text{% RF}}}\mathcal{L}\!\left({\bf{\tilde{F}}}_{\text{RF}}^{(k+1)},{\bf F}_{\text{RF% }}^{(k+1)},{\bf U}^{(k)}\right),{\bf{\tilde{F}}}_{\text{RF}}^{(k+1)}\!-\!{\bf{% \tilde{F}}}_{\text{RF}}^{(k)}\Big{\rangle}\!\right\}$
	$\displaystyle-\frac{\gamma}{2}\left\\|{\bf{\tilde{F}}}_{\text{RF}}^{(k+1)}-{\bf% {\tilde{F}}}_{\text{RF}}^{(k)}\right\\|_{\text{F}}^{2}$	(41a)
$\displaystyle=$	$\displaystyle~{}\!-\frac{\lambda_{\textrm{min}}({\bf F}_{\text{BB}}{\bf F}_{% \text{BB}}^{\textrm{H}})+\rho}{2}\left\\|{\bf{\tilde{F}}}_{\text{RF}}^{(k+1)}-{% \bf{\tilde{F}}}_{\text{RF}}^{(k)}\right\\|_{\text{F}}^{2},$	(41b)

where in (41a) we utilized the strongly convexity of the Lagrangian function $\mathcal{L}\Big{(}{\bf{\tilde{F}}}_{\text{RF}},{\bf F}_{\text{RF}},{\bf U}\Big% {)}$ w.r.t. ${\bf{\tilde{F}}}_{\text{RF}}$ with parameter $\gamma>0$ [59]; in (41b) we adopted the optimality condition of (III-C) and $\gamma=\lambda_{\textrm{min}}({\bf F}_{\text{BB}}{\bf F}_{\text{BB}}^{\textrm{% H}})+\rho$ with $\lambda_{\textrm{min}}(\cdot)$ being the minimal eigenvalue of its argument (which is due to the facts that $\mathcal{L}\Big{(}{\bf{\tilde{F}}}_{\text{RF}},{\bf F}_{\text{RF}},{\bf U}\Big% {)}$ is twice continuously differentiable w.r.t. ${\bf{\tilde{F}}}_{\text{RF}}$ , and its strong convexity parameter $\gamma$ satisfies $\nabla_{{\bf{\tilde{F}}}_{\text{RF}}}^{2}\mathcal{L}={\bf F}_{\text{BB}}{\bf F% }_{\text{BB}}^{\textrm{H}}+\rho{\bf I}\succeq\gamma{\bf I}$ for all ${\bf{\tilde{F}}}_{\text{RF}}$ [59]). Finally, the third term is bounded as

\displaystyle\mathcal{L}\!\left(\!{\bf{\tilde{F}}}_{\text{RF}}^{(k)},{\bf F}_{% \text{RF}}^{(k+1)},{\bf U}^{(k)}\!\right)\!-\!\mathcal{L}\!\left(\!{\bf{\tilde% {F}}}_{\text{RF}}^{(k)},{\bf F}_{\text{RF}}^{(k)},{\bf U}^{(k)}\!\right)\leq 0,

(42)

where we employed the fact that ${\bf F}_{\text{RF}}^{(k+1)}$ is the minimum of $\mathcal{L}\Big{(}{\bf{\tilde{F}}}_{\text{RF}}^{(k)},{\bf F}_{\text{RF}},{\bf U% }^{(k)}\Big{)}$ according to (III-C).

Substituting the results of (E), (E), and (42) in (39) yields

		$\displaystyle\mathcal{L}\!\left({\bf{\tilde{F}}}_{\text{RF}}^{(k+1)},{\bf F}_{% \text{RF}}^{(k+1)},{\bf U}^{(k+1)}\right)-\mathcal{L}\!\left({\bf{\tilde{F}}}_% {\text{RF}}^{(k)},{\bf F}_{\text{RF}}^{(k)},{\bf U}^{(k)}\right)$
	$\displaystyle\leq$	$\displaystyle~{}\!\underbrace{\left(\frac{1}{\rho}\left\\|{\bf F}_{\text{BB}}{% \bf F}_{\text{BB}}^{\textrm{H}}\right\\|_{\text{F}}^{2}\!-\!\frac{\lambda_{% \textrm{min}}({\bf F}_{\text{BB}}{\bf F}_{\text{BB}}^{\textrm{H}})\!+\!\rho}{2% }\right)}_{\text{(i)}}\left\\|{\bf{\tilde{F}}}_{\text{RF}}^{(k+1)}\!-\!{\bf{% \tilde{F}}}_{\text{RF}}^{(k)}\right\\|_{\text{F}}^{2}.$

If $\rho\geq\sqrt{2}\|{\bf F}_{\text{BB}}{\bf F}_{\text{BB}}^{\textrm{H}}\|_{\text% {F}}$ , the term (i) satisfies: $(\text{i})\leq 0$ , and thus

\displaystyle\mathcal{L}\!\left({\bf{\tilde{F}}}_{\text{RF}}^{(k+1)},{\bf F}_{% \text{RF}}^{(k+1)},{\bf U}^{(k+1)}\right)-\mathcal{L}\!\left({\bf{\tilde{F}}}_% {\text{RF}}^{(k)},{\bf F}_{\text{RF}}^{(k)},{\bf U}^{(k)}\right)\leq 0.

This completes the proof of Lemma 1.

Appendix F Proof of Lemma 2

By using (35), we have

		$\displaystyle\mathcal{L}\Big{(}{\bf{\tilde{F}}}_{\text{RF}},{\bf F}_{\text{RF}% },{\bf U}\Big{)}$
	$\displaystyle=$	$\displaystyle~{}\!\frac{1}{2}\\|{\bf F}_{\text{opt}}\!-\!{\bf{\tilde{F}}}_{% \text{RF}}{\bf F}_{\text{BB}}\\|_{\text{F}}^{2}\!+\!\frac{\rho}{2}\\|{\bf{\tilde% {F}}}_{\text{RF}}\!-\!{\bf F}_{\text{RF}}\!+\!{\bf U}\\|_{\text{F}}^{2}$
		$\displaystyle-\frac{\rho}{2}\left\\|\frac{1}{\rho}\left({\bf F}_{\text{opt}}-{% \bf{\tilde{F}}}_{\text{RF}}{\bf F}_{\text{BB}}\right){\bf F}_{\text{BB}}^{% \textrm{H}}\right\\|_{\text{F}}^{2}$
	$\displaystyle\geq$	$\displaystyle~{}\!\frac{1}{2}\\|{\bf F}_{\text{opt}}\!-\!{\bf{\tilde{F}}}_{% \text{RF}}{\bf F}_{\text{BB}}\\|_{\text{F}}^{2}\!+\!\frac{\rho}{2}\\|{\bf{\tilde% {F}}}_{\text{RF}}\!-\!{\bf F}_{\text{RF}}\!+\!{\bf U}\\|_{\text{F}}^{2}$
		$\displaystyle-\frac{1}{2\rho}\left\\|{\bf F}_{\text{opt}}-{\bf{\tilde{F}}}_{% \text{RF}}{\bf F}_{\text{BB}}\right\\|_{\text{F}}^{2}\left\\|{\bf F}_{\text{BB}}% \right\\|_{\text{F}}^{2}$
	$\displaystyle=$	$\displaystyle~{}\!\frac{1}{2}\left(1-\frac{1}{\rho}\left\\|{\bf F}_{\text{BB}}% \right\\|_{\text{F}}^{2}\right)\\|{\bf F}_{\text{opt}}\!-\!{\bf{\tilde{F}}}_{% \text{RF}}{\bf F}_{\text{BB}}\\|_{\text{F}}^{2}$
		$\displaystyle~{}\!+\frac{\rho}{2}\\|{\bf{\tilde{F}}}_{\text{RF}}-{\bf F}_{\text% {RF}}+{\bf U}\\|_{\text{F}}^{2}.$

If $\rho\geq\|{\bf F}_{\text{BB}}\|_{\text{F}}^{2}$ , then $\mathcal{L}\Big{(}{\bf{\tilde{F}}}_{\text{RF}},{\bf F}_{\text{RF}},{\bf U}\Big% {)}\geq 0$ , which completes the proof of Lemma 2.

References

[1] R. W. Heath, N. González-Prelcic, S. Rangan, W. Roh, and A. M. Sayeed, “An overview of signal processing techniques for millimeter wave MIMO systems,” IEEE J. Sel. Top. Signal Process., vol. 10, no. 3, pp. 436–453, April 2016.
[2] Y. Ge, O. Kaltiokallio, H. Kim, J. Talvitie, S. Kim, L. Svensson, M. Valkama, and H. Wymeersch, “MmWave map** and SLAM for 5G and beyond,” in Integrated Sensing and Communications, F. Liu, C. Masouros, and Y. C. Eldar, Eds. Singapore: Springer Nature Singapore, 2023, pp. 445–475.
[3] S. Weng, F. Jiang, and H. Wymeersch, “Wideband mmWave massive MIMO channel estimation and localization,” IEEE Wireless Commun. Lett., vol. 12, no. 8, pp. 1314–1318, August 2023.
[4] M. A. Nazari, G. Seco-Granados, P. Johannisson, and H. Wymeersch, “mmWave 6D radio localization with a snapshot observation from a single BS,” IEEE Trans. Veh. Technol., vol. 72, no. 7, pp. 8914–8928, July 2023.
[5] H. Chen, H. Sarieddeen, T. Ballal, H. Wymeersch, M.-S. Alouini, and T. Y. Al-Naffouri, “A tutorial on terahertz-band localization for 6G communication systems,” IEEE Commun. Surv. Tutor., vol. 24, no. 3, pp. 1780–1815, May 2022.
[6] Z. Chen, X. Ma, B. Zhang, Y. Zhang, Z. Niu, N. Kuang, W. Chen, L. Li, and S. Li, “A survey on terahertz communications,” China Commun., vol. 16, no. 2, pp. 1–35, February 2019.
[7] V. Petrov, T. Kurner, and I. Hosako, “IEEE 802.15.3d: First standardization efforts for sub-terahertz band communications toward 6G,” IEEE Commun. Mag., vol. 58, no. 11, pp. 28–33, November 2020.
[8] X. Cai, X. Cheng, and F. Tufvesson, “Toward 6G with terahertz communications: Understanding the propagation channels,” arXiv, February 2023. [Online]. Available: https://arxiv.longhoe.net/abs/2209.07864
[9] C. Studer and G. Durisi, “Quantized massive MU-MIMO-OFDM uplink,” IEEE Trans. Commun., vol. 64, no. 6, pp. 2387–2399, June 2016.
[10] S. Jacobsson, G. Durisi, M. Coldrey, T. Goldstein, and C. Studer, “Quantized precoding for massive MU-MIMO,” IEEE Trans. Commun., vol. 65, no. 11, pp. 4670–4684, November 2017.
[11] E. Björnson, L. Sanguinetti, H. Wymeersch, J. Hoydis, and T. L. Marzetta, “Massive MIMO is a reality—What is next?: Five promising research directions for antenna arrays,” Digit. Signal Process., vol. 94, pp. 3–20, November 2019.
[12] M. F. Keskin, C. Marcus, O. Eriksson, A. Alvarado, J. Widmer, and H. Wymeersch, “Integrated sensing and communications with MIMO-OTFS,” 2023. [Online]. Available: https://arxiv.longhoe.net/abs/2306.06361
[13] T. Wei, Z. Cheng, and B. Liao, “Transmit beampattern synthesis for MIMO radar with one-bit digital-to-analog converters,” Signal Process., vol. 188, p. 108228, November 2021.
[14] Z. Gao, L. Dai, D. Mi, Z. Wang, M. A. Imran, and M. Z. Shakir, “MmWave massive-MIMO-based wireless backhaul for the 5G ultra-dense network,” IEEE Wirel. Commun., vol. 22, no. 5, pp. 13–21, October 2015.
[15] A. F. Molisch, V. V. Ratnam, S. Han, Z. Li, S. L. H. Nguyen, L. Li, and K. Haneda, “Hybrid beamforming for massive MIMO: A survey,” IEEE Commun. Mag., vol. 55, no. 9, pp. 134–141, December 2017.
[16] R. Rajamäki, S. P. Chepuri, and V. Koivunen, “Hybrid beamforming for active sensing using sparse arrays,” IEEE Trans. Signal Process., vol. 68, pp. 6402–6417, October 2020.
[17] H. Li, M. Li, and Q. Liu, “Hybrid beamforming with dynamic subarrays and low-resolution PSs for mmWave MU-MISO systems,” IEEE Trans. Commun., vol. 68, no. 1, pp. 602–614, January 2020.
[18] S. Lyu, Z. Wang, Z. Gao, H. He, and L. Hanzo, “Lattice-based mmWave hybrid beamforming,” IEEE Trans. Commun., vol. 69, no. 7, pp. 4907–4920, July 2021.
[19] J.-C. Chen, “Hybrid beamforming with discrete phase shifters for millimeter-wave massive MIMO systems,” IEEE Trans. Veh. Technol., vol. 66, no. 8, pp. 7604–7608, August 2017.
[20] Z. Wang, M. Li, Q. Liu, and A. L. Swindlehurst, “Hybrid precoder and combiner design with low-resolution phase shifters in mmWave MIMO systems,” IEEE J. Sel. Top. Signal Process., vol. 12, no. 2, pp. 256–269, May 2018.
[21] A. N. Uwaechia, N. M. Mahyuddin, M. F. Ain, N. M. Abdul Latiff, and N. F. Za’bah, “On the spectral-efficiency of low-complexity and reso-lution hybrid precoding and combining transceivers for mmWave MIMO systems,” IEEE Access, vol. 7, pp. 109 259–109 277, August 2019.
[22] H. Li, M. Li, Q. Liu, and A. L. Swindlehurst, “Dynamic hybrid beamforming with low-resolution PSs for wideband mmWave MIMO-OFDM systems,” IEEE J. Sel. Areas Commun., vol. 38, no. 9, pp. 2168–2181, September 2020.
[23] X. Zhang, A. Molisch, and S.-Y. Kung, “Variable-phase-shift-based RF-baseband codesign for MIMO antenna selection,” IEEE Trans. Signal Process., vol. 53, no. 11, pp. 4091–4103, November 2005.
[24] O. E. Ayach, S. Rajagopal, S. Abu-Surra, Z. Pi, and R. W. Heath, “Spatially sparse precoding in millimeter wave MIMO systems,” IEEE Trans. Wirel. Commun., vol. 13, no. 3, pp. 1499–1513, March 2014.
[25] C.-J. Wang, C.-K. Wen, S. **, and S.-H. Tsai, “Finite-alphabet precoding for massive MU-MIMO with low-resolution DACs,” IEEE Trans. Wirel. Commun., vol. 17, no. 7, pp. 4706–4720, July 2018.
[26] F. Dong, W. Wang, and Z. Wei, “Low-complexity hybrid precoding for multi-user mmWave systems with low-resolution phase shifters,” IEEE Trans. Veh. Technol., vol. 68, no. 10, pp. 9774–9784, October 2019.
[27] R. Rajamäki, S. P. Chepuri, and V. Koivunen, “Hybrid beamforming for active sensing using sparse arrays,” IEEE Trans. Signal Process., vol. 68, pp. 6402–6417, October 2020.
[28] E. Zhang and C. Huang, “On achieving optimal rate of digital precoder by RF-baseband codesign for MIMO systems,” in Proc. IEEE Veh. Technol. Conf., Vancouver, Canada, September 2014, pp. 1–5.
[29] X. Yu, J.-C. Shen, J. Zhang, and K. B. Letaief, “Alternating minimization algorithms for hybrid precoding in millimeter wave MIMO systems,” IEEE J. Sel. Top. Signal Process., vol. 10, no. 3, pp. 485–500, April 2016.
[30] X. Gao, L. Dai, S. Han, C.-L. I, and R. W. Heath, “Energy-efficient hybrid analog and digital precoding for mmWave MIMO systems with large antenna arrays,” IEEE J. Sel. Areas Commun., vol. 34, no. 4, pp. 998–1009, April 2016.
[31] F. Sohrabi and W. Yu, “Hybrid beamforming with finite-resolution phase shifters for large-scale MIMO systems,” in Proc. IEEE Workshop Signal Process. Adv. Wirel. Commun., Stockholm, Sweden, June 2015, pp. 136–140.
[32] F. Sohrabi and W. Yu, “Hybrid digital and analog beamforming design for large-scale antenna arrays,” IEEE J. Sel. Top. Signal Process., vol. 10, no. 3, pp. 501–513, April 2016.
[33] Y.-P. Lin, “On the quantization of phase shifters for hybrid precoding systems,” IEEE Trans. Signal Process., vol. 65, no. 9, pp. 2237–2246, May 2017.
[34] Z. Cheng, L. Wu, B. Wang, M. R. B. Shankar, and B. Ottersten, “Double-phase-shifter based hybrid beamforming for mmWave DFRC in the presence of extended target and clutters,” IEEE Trans. Wirel. Commun., vol. 22, no. 6, pp. 3671–3686, June 2023.
[35] Z. Cheng, L. Wu, B. Wang, J. Xie, and H. Li, “Relative entropy-based constant-envelope beamforming for target detection in large-scale MIMO radar with low-resoultion ADCs,” IEEE Trans. Veh. Technol., vol. 72, no. 8, pp. 10 090–10 106, August 2023.
[36] L. Babai, “On lovász’ lattice reduction and the nearest lattice point problem,” Combinatorica, vol. 6, pp. 1–13, March 1986.
[37] N. Boumal, An Introduction to Optimization on Smooth Manifolds. Cambridge University Press, 2023.
[38] P.-A. Absil, R. Mahony, and R. Sepulchre, Optimization Algorithms on Matrix Manifolds. USA: Princeton University Press, 2008.
[39] J. Hu, X. Liu, Z.-W. Wen, and Y.-X. Yuan, “A brief introduction to manifold optimization,” J. Oper. Res. Soc. China, vol. 8, pp. 199–248, April 2020.
[40] N. Garcia, H. Wymeersch, and D. T. M. Slock, “Optimal precoders for tracking the AoD and AoA of a mmWave path,” IEEE Trans. Signal Process., vol. 66, no. 21, pp. 5718–5729, November 2018.
[41] M. F. Keskin, F. Jiang, F. Munier, G. Seco-Granados, and H. Wymeersch, “Optimal spatial signal design for mmWave positioning under imperfect synchronization,” IEEE Trans. Veh. Technol., vol. 71, no. 5, pp. 5558–5563, May 2022.
[42] J. Li, L. Xu, P. Stoica, K. W. Forsythe, and D. W. Bliss, “Range compression and waveform optimization for MIMO radar: A Cramér–Rao bound based study,” IEEE Trans. Signal Process., vol. 56, no. 1, pp. 218–232, January 2008.
[43] M. Deng, Z. Cheng, L. Wu, B. Shankar, and Z. He, “One-bit ADCs/DACs based MIMO radar: Performance analysis and joint design,” IEEE Trans. Signal Process., vol. 70, pp. 2609–2624, May 2022.
[44] M. Hong, Z.-Q. Luo, and M. Razaviyayn, “Convergence analysis of alternating direction method of multipliers for a family of nonconvex problems,” SIAM J. Optim., vol. 26, no. 1, pp. 337–364, January 2016.
[45] S. Boyd, N. Parikh, E. Chu, B. Peleato, and J. Eckstein, “Distributed optimization and statistical learning via the alternating direction method of multipliers,” Found. Trends Mach. Learn., vol. 3, no. 1, pp. 1–122, January 2011.
[46] K. Huang and N. D. Sidiropoulos, “Consensus-ADMM for general quadratically constrained quadratic programming,” IEEE Trans. Signal Process., vol. 64, no. 20, pp. 5297–5310, October 2016.
[47] H. Huang, H. C. So, and A. M. Zoubir, “Sparse array beamformer design via ADMM,” IEEE Trans. Signal Process., vol. 71, pp. 3357–3372, September 2023.
[48] H. Huang, H. C. So, and A. M. Zoubir, “Convergence analysis of consensus-ADMM for general QCQP,” Signal Process., vol. 208, p. 108991, July 2023.
[49] L. Wu, X. Cheng, H. Huang, D. Ciuonzo, B. Shankar, and B. Ottersten, “Constant-modulus waveform design with polarization-adaptive power allocation in polarimetric radar,” IEEE Trans. Signal Process., vol. 71, pp. 2146–2161, June 2023.
[50] A. Fascista, A. Coluccia, H. Wymeersch, and G. Seco-Granados, “Millimeter-wave downlink positioning with a single-antenna receiver,” IEEE Trans. Wirel. Commun., vol. 18, no. 9, pp. 4479–4490, September 2019.
[51] Q. Hu, Y. Cai, K. Kang, G. Yu, J. Hoydis, and Y. C. Eldar, “Two-timescale end-to-end learning for channel acquisition and hybrid precoding,” IEEE J. Sel. Areas Commun., vol. 40, no. 1, pp. 163–181, January 2022.
[52] A. Liu, V. K. N. Lau, and M.-J. Zhao, “Stochastic successive convex optimization for two-timescale hybrid precoding in massive MIMO,” IEEE J. Sel. Top. Signal Process., vol. 12, no. 3, pp. 432–444, June 2018.
[53] A. Alkhateeb, O. El Ayach, G. Leus, and R. W. Heath, “Channel estimation and hybrid precoding for millimeter wave cellular systems,” IEEE J. Sel. Top. Signal Process., vol. 8, no. 5, pp. 831–846, October 2014.
[54] S. Kay, Fundamentals of Statistical Signal Processing, Volume 3, ser. Fundamentals of statistical signal processing. Prentice Hall PTR, 2017.
[55] H. Anton, Elementary Linear Algebra. Wiley, 1994.
[56] N. Garcia, H. Wymeersch, E. G. Ström, and D. Slock, “Location-aided mm-wave channel estimation for vehicular communication,” in Proc. IEEE Workshop Signal Process. Adv. Wirel. Commun., Edinburgh, UK, July 2016, pp. 1–5.
[57] N. Boumal, B. Mishra, P.-A. Absil, and R. Sepulchre, “Manopt, a Matlab toolbox for optimization on manifolds,” J. Mach. Learn. Res., vol. 15, no. 42, pp. 1455–1459, 2014.
[58] W. Rudin, “Chapter 3 - Numerical sequences and series,” in Principles of mathematical analysis. McGraw - Hill Book C., 1986, pp. 47–78.
[59] E. K. Ryu and S. P. Boyd, “A primer on monotone operator methods,” Appl. Comput. Math., vol. 15, no. 1, pp. 3–43, 2016.

	$\displaystyle\\|{\bf 1}-{\bf\Phi}\\|_{\text{F}}=$	$\displaystyle~{}\!\sqrt{\sum_{i=1}^{N_{\text{Tx}}}\sum_{j=1}^{N_{\text{RF}}}% \left\|1-e^{\jmath\phi_{ij}}\right\|^{2}}$
	$\displaystyle\leq$	$\displaystyle~{}\!\sqrt{\sum_{i=1}^{N_{\text{Tx}}}\sum_{j=1}^{N_{\text{RF}}}% \left\|1-e^{\jmath\pi/2^{B}}\right\|^{2}}$
	$\displaystyle=$	$\displaystyle~{}\!\left\|1-e^{\jmath{\pi}/{2^{B}}}\right\|\sqrt{N_{\text{Tx}}N_{% \text{RF}}}~{}\!.$

	$\displaystyle\\|{\bf F}_{\text{opt}}-{\bf F}_{\text{RF}}^{(i+1)}{\bf F}_{\text{% BB}}^{(i+1)}\\|_{\text{F}}\leq$	$\displaystyle~{}\!\\|{\bf F}_{\text{opt}}-{\bf F}_{\text{RF}}^{(i)}{\bf F}_{% \text{BB}}^{(i+1)}\\|_{\text{F}}$
	$\displaystyle\leq$	$\displaystyle~{}\!\\|{\bf F}_{\text{opt}}-{\bf F}_{\text{RF}}^{(i)}{\bf F}_{% \text{BB}}^{(i)}\\|_{\text{F}},$

		$\displaystyle\mathcal{L}\Big{(}{\bf{\tilde{F}}}_{\text{RF}},{\bf F}_{\text{RF}% },{\bf U}\Big{)}$
	$\displaystyle=$	$\displaystyle~{}\!\frac{1}{2}\\|{\bf F}_{\text{opt}}\!-\!{\bf{\tilde{F}}}_{% \text{RF}}{\bf F}_{\text{BB}}\\|_{\text{F}}^{2}\!+\!\frac{\rho}{2}\\|{\bf{\tilde% {F}}}_{\text{RF}}\!-\!{\bf F}_{\text{RF}}\!+\!{\bf U}\\|_{\text{F}}^{2}$
		$\displaystyle-\frac{\rho}{2}\left\\|\frac{1}{\rho}\left({\bf F}_{\text{opt}}-{% \bf{\tilde{F}}}_{\text{RF}}{\bf F}_{\text{BB}}\right){\bf F}_{\text{BB}}^{% \textrm{H}}\right\\|_{\text{F}}^{2}$
	$\displaystyle\geq$	$\displaystyle~{}\!\frac{1}{2}\\|{\bf F}_{\text{opt}}\!-\!{\bf{\tilde{F}}}_{% \text{RF}}{\bf F}_{\text{BB}}\\|_{\text{F}}^{2}\!+\!\frac{\rho}{2}\\|{\bf{\tilde% {F}}}_{\text{RF}}\!-\!{\bf F}_{\text{RF}}\!+\!{\bf U}\\|_{\text{F}}^{2}$
		$\displaystyle-\frac{1}{2\rho}\left\\|{\bf F}_{\text{opt}}-{\bf{\tilde{F}}}_{% \text{RF}}{\bf F}_{\text{BB}}\right\\|_{\text{F}}^{2}\left\\|{\bf F}_{\text{BB}}% \right\\|_{\text{F}}^{2}$
	$\displaystyle=$	$\displaystyle~{}\!\frac{1}{2}\left(1-\frac{1}{\rho}\left\\|{\bf F}_{\text{BB}}% \right\\|_{\text{F}}^{2}\right)\\|{\bf F}_{\text{opt}}\!-\!{\bf{\tilde{F}}}_{% \text{RF}}{\bf F}_{\text{BB}}\\|_{\text{F}}^{2}$
		$\displaystyle~{}\!+\frac{\rho}{2}\\|{\bf{\tilde{F}}}_{\text{RF}}-{\bf F}_{\text% {RF}}+{\bf U}\\|_{\text{F}}^{2}.$

Hybrid Precoder Design for Angle-of-Departure Estimation with Limited-Resolution Phase Shifters

Abstract

Index Terms:

I Introduction

II System Model

III Proposed Method

III-A CRB-Based Performance Metric

III-B Problem Formulation for Optimal Precoder Design

III-C Two-Step Strategy for Solving Problem (III-B)

Remark.

III-D Computational Complexity Analysis

IV Analysis of Error Bounds and Convergence

IV-A Analysis of Quantization Error Bound

IV-B Convergence Analysis

Theorem 1.

Proof.

Theorem 2.

Proof.

V Numerical Results

V-A Scenario, Performance Metric, and Benchmark

V-B Results and Discussion of Scenario I

V-B1 DecpErr as a Function of NRFsubscript𝑁RFN_{\text{\emph{RF}}}italic_N start_POSTSUBSCRIPT RF end_POSTSUBSCRIPT

V-B2 DecpErr as a Function of B𝐵Bitalic_B

V-C Results and Discussion of Scenario II

V-C1 AEB as a Function of NTxsubscript𝑁TxN_{\text{\emph{Tx}}}italic_N start_POSTSUBSCRIPT Tx end_POSTSUBSCRIPT

V-C2 AEB as a Function of NRFsubscript𝑁RFN_{\text{\emph{RF}}}italic_N start_POSTSUBSCRIPT RF end_POSTSUBSCRIPT

V-C3 AEB as a Function of B𝐵Bitalic_B

V-C4 AEB as a Function of AoD

V-C5 CPU Runtime as a Function of NTxsubscript𝑁TxN_{\text{\emph{Tx}}}italic_N start_POSTSUBSCRIPT Tx end_POSTSUBSCRIPT or NRFsubscript𝑁RFN_{\text{\emph{RF}}}italic_N start_POSTSUBSCRIPT RF end_POSTSUBSCRIPT

VI Conclusion

Appendix A Calculation of the FIM

Appendix B Calculation of the AEB

Appendix C Proof of Theorem 1

Lemma 1.

Lemma 2.

Appendix D Proof of Theorem 2

Appendix E Proof of Lemma 1

Appendix F Proof of Lemma 2

References

V-B1 DecpErr as a Function of $N_{\text{\emph{RF}}}$

V-B2 DecpErr as a Function of $B$

V-C1 AEB as a Function of $N_{\text{\emph{Tx}}}$

V-C2 AEB as a Function of $N_{\text{\emph{RF}}}$

V-C3 AEB as a Function of $B$

V-C5 CPU Runtime as a Function of $N_{\text{\emph{Tx}}}$ or $N_{\text{\emph{RF}}}$