SUBSAMPLING FOR BIG DATA LINEAR MODELS
WITH MEASUREMENT ERRORS
Jiangshan Ju, Mingqiu Wang and Shengli Zhao
School of Statistics and Data Science, Qufu Normal University
1 Introduction
To address the ever-increasing volume of data brought about by technological advancements, it is imperative to adopt refined techniques such as divide and conquer, online updating of streaming data, and subsampling-based methods. These techniques offer effective solutions to computational challenges posed by large datasets. However, existing literature often assumes that direct and accurate observation of covariates, which may not always be feasible in practical data collection scenarios. Consequently, statistical models that do not account for measurement errors can lead to biased estimated results. Therefore, it is essential to investigate subsampling algorithms for linear models that consider measurement errors in covariates.
For the subsampling algorithm of linear models, Ma, Mahoney and Yu (2015 ) proposed a leverage sampling algorithm based on leverage scores and their linear transformations.
A deterministic subsampling method named information-based optimal subdata selection (IBOSS) was proposed by Wang, Yang and Stufken (2019 ) , and extended by Wang (2019b ) , aiming to find subsamples with the maximum information matrix under the D-optimality criterion, which performs well in finding corners.
Cheng, Wang and Yang (2020 ) and Yu, Liu and Wang (2023 ) extended the IBOSS algorithm to logistic and nonlinear models, respectively.
Wang et al. (2021 ) proposed an orthogonal subsampling approach for big data linear regression.
Yi and Zhou (2023 ) and Zhang et al. (2024 ) explored using space-filling or uniform designs to obtain the subsample so that a wide range of models could be considered.
Wang, Zhu and Ma (2018 ) proposed the optimal subsampling method based on the A-optimality criterion.
This method was further developed by Wang (2019a ) , Ai et al. (2021a ) , Ai et al. (2021b ) , Wang and Ma (2021 ) , and Yu et al. (2022 ) .
In addition to the widely used method based on inverse probability weighting, Wang and Kim (2022 ) introduced the maximum sample conditional likelihood estimation, and enhanced the estimator for selected subsamples. This approach overcomes the limitations of inverse probability weighting and makes more efficient use of sample information.
Yao and ** (2024 ) introduced a perturbation subsampling method that employs repeated random weighting of known distributions to address the limitations of inverse probability weighting. This method has been successfully applied to linear models, longitudinal data, and high-dimensional data with promising performance.
For the measurement error model, Fuller (1987 ) systematically introduced a comprehensive statistical inference of linear regression models with measurement errors.
Nakamura (1990 ) proposed the corrected score method for the generalized linear model with measurement errors.
Carroll et al. (2006 ) systematically studied the theory of nonlinear regression models with measurement errors.
Liang, Hardle and Carroll (1999 ) offered the parameter estimation of a semi-parametric partially linear model with measurement errors.
Liang and Li (2009 ) examined variable selection in partially linear models with measurement errors, and proposed that when the variance of the measurement error is unknown, it can be estimated through repeated observations.
Lee, Wang and Schifano (2020 ) introduced an online update method for correcting measurement errors in big data streams.
This study focuses on the subsampling problem in linear models with measurement errors. The presence of measurement errors in covariates can introduce inaccuracies of parameter estimation, thereby diminishing the statistical power. We employ the corrected likelihood approach proposed by Nakamura (1990 ) to estimate the parameters with subsamples. We introduce an optimal subsampling method based on the corrected likelihood approach, and the optimal subsampling probabilities are determined by minimizing the trace of the variance. The consistency and asymptotic normality of estimators obtained by this approach are established. Furthermore, we propose a perturbation subsampling method based on the corrected likelihood approach that approximates the objective function of the full data using a perturbation with independently generated stochastic weights. The effectiveness of our method is also confirmed through numerical analysis. By accounting for measurement errors in covariates, more precise and reliable results can be obtained in the analysis of massive datasets.
Our approaches not only alleviate the computational burden associated with parameter estimation in big data but also enhance computational efficiency and improve prediction accuracy.
The rest of the paper is outlined as follows.
Section 2 offers a comprehensive introduction to the model setup and parameter estimation of the measurement error model.
Sections 3 and 4 introduce the linear model subsampling algorithm with measurement errors and establish the correspondingly theoretical properties.
Section 5 comprises numerical simulations. Section 6 presents case studies aimed at validating effectiveness of the algorithm.
Finally, Section 7 summarizes this paper. The detailed proofs are provided in the supplementary materials
5 Simulation studies
We generate the full data from model (2.1) with n = 10000 𝑛 10000 n=10000 italic_n = 10000 , 𝜷 = ( 1 , 1 , 1 , 1 , 1 ) T 𝜷 superscript 1 1 1 1 1 𝑇 \boldsymbol{\beta}=(1,1,1,1,1)^{T} bold_italic_β = ( 1 , 1 , 1 , 1 , 1 ) start_POSTSUPERSCRIPT italic_T end_POSTSUPERSCRIPT and ϵ i ∼ N ( 0 , σ ϵ 2 ) similar-to subscript italic-ϵ 𝑖 𝑁 0 superscript subscript 𝜎 italic-ϵ 2 \epsilon_{i}\sim N(0,\sigma_{\epsilon}^{2}) italic_ϵ start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT ∼ italic_N ( 0 , italic_σ start_POSTSUBSCRIPT italic_ϵ end_POSTSUBSCRIPT start_POSTSUPERSCRIPT 2 end_POSTSUPERSCRIPT ) .
Let 𝐗 i ∼ N 5 ( 𝟎 , 𝚺 ) similar-to subscript 𝐗 𝑖 subscript 𝑁 5 0 𝚺 \mathbf{X}_{i}\sim N_{5}(\mathbf{0},\boldsymbol{\Sigma}) bold_X start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT ∼ italic_N start_POSTSUBSCRIPT 5 end_POSTSUBSCRIPT ( bold_0 , bold_Σ ) , where 𝚺 j , k = 0.5 | j − k | subscript 𝚺 𝑗 𝑘
superscript 0.5 𝑗 𝑘 \boldsymbol{\Sigma}_{j,k}=0.5^{|j-k|} bold_Σ start_POSTSUBSCRIPT italic_j , italic_k end_POSTSUBSCRIPT = 0.5 start_POSTSUPERSCRIPT | italic_j - italic_k | end_POSTSUPERSCRIPT , for j , k = 1 , 2 , … , 5 formulae-sequence 𝑗 𝑘
1 2 … 5
j,k=1,2,\ldots,5 italic_j , italic_k = 1 , 2 , … , 5 , and
𝐔 i ∼ N 5 ( 𝟎 , σ u 2 I ) similar-to subscript 𝐔 𝑖 subscript 𝑁 5 0 superscript subscript 𝜎 𝑢 2 𝐼 \mathbf{U}_{i}\sim N_{5}(\mathbf{0},\sigma_{u}^{2}I) bold_U start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT ∼ italic_N start_POSTSUBSCRIPT 5 end_POSTSUBSCRIPT ( bold_0 , italic_σ start_POSTSUBSCRIPT italic_u end_POSTSUBSCRIPT start_POSTSUPERSCRIPT 2 end_POSTSUPERSCRIPT italic_I ) , then 𝐖 i = 𝐗 i + 𝐔 i subscript 𝐖 𝑖 subscript 𝐗 𝑖 subscript 𝐔 𝑖 \mathbf{W}_{i}=\mathbf{X}_{i}+\mathbf{U}_{i} bold_W start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT = bold_X start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT + bold_U start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT .
We consider the following three values for σ u 2 superscript subscript 𝜎 𝑢 2 \sigma_{u}^{2} italic_σ start_POSTSUBSCRIPT italic_u end_POSTSUBSCRIPT start_POSTSUPERSCRIPT 2 end_POSTSUPERSCRIPT and σ ϵ 2 superscript subscript 𝜎 italic-ϵ 2 \sigma_{\epsilon}^{2} italic_σ start_POSTSUBSCRIPT italic_ϵ end_POSTSUBSCRIPT start_POSTSUPERSCRIPT 2 end_POSTSUPERSCRIPT , respectively:
σ u 2 = 0.6 , 0.4 , 0.2 superscript subscript 𝜎 𝑢 2 0.6 0.4 0.2
\sigma_{u}^{2}=0.6,0.4,0.2 italic_σ start_POSTSUBSCRIPT italic_u end_POSTSUBSCRIPT start_POSTSUPERSCRIPT 2 end_POSTSUPERSCRIPT = 0.6 , 0.4 , 0.2 ;
σ ϵ 2 = 1.44 , 1 , 0.64 superscript subscript 𝜎 italic-ϵ 2 1.44 1 0.64
\sigma_{\epsilon}^{2}=1.44,1,0.64 italic_σ start_POSTSUBSCRIPT italic_ϵ end_POSTSUBSCRIPT start_POSTSUPERSCRIPT 2 end_POSTSUPERSCRIPT = 1.44 , 1 , 0.64 .
In Algorithm 3 , let m = 10 𝑚 10 m=10 italic_m = 10 , and assume that the known distribution with random weighting follows an exponential distribution with mean 1 / q n 1 subscript 𝑞 𝑛 1/q_{n} 1 / italic_q start_POSTSUBSCRIPT italic_n end_POSTSUBSCRIPT , i.e., ν i ∼ Exp ( q n ) similar-to subscript 𝜈 𝑖 Exp subscript 𝑞 𝑛 \nu_{i}\sim\mathrm{Exp}(q_{n}) italic_ν start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT ∼ roman_Exp ( italic_q start_POSTSUBSCRIPT italic_n end_POSTSUBSCRIPT ) . Correspondingly, b n 2 = 1 / q n 2 superscript subscript 𝑏 𝑛 2 1 superscript subscript 𝑞 𝑛 2 b_{n}^{2}=1/q_{n}^{2} italic_b start_POSTSUBSCRIPT italic_n end_POSTSUBSCRIPT start_POSTSUPERSCRIPT 2 end_POSTSUPERSCRIPT = 1 / italic_q start_POSTSUBSCRIPT italic_n end_POSTSUBSCRIPT start_POSTSUPERSCRIPT 2 end_POSTSUPERSCRIPT and a n = 1 − q n + b n 2 q n 2 = 2 − q n subscript 𝑎 𝑛 1 subscript 𝑞 𝑛 superscript subscript 𝑏 𝑛 2 superscript subscript 𝑞 𝑛 2 2 subscript 𝑞 𝑛 a_{n}=1-q_{n}+b_{n}^{2}q_{n}^{2}=2-q_{n} italic_a start_POSTSUBSCRIPT italic_n end_POSTSUBSCRIPT = 1 - italic_q start_POSTSUBSCRIPT italic_n end_POSTSUBSCRIPT + italic_b start_POSTSUBSCRIPT italic_n end_POSTSUBSCRIPT start_POSTSUPERSCRIPT 2 end_POSTSUPERSCRIPT italic_q start_POSTSUBSCRIPT italic_n end_POSTSUBSCRIPT start_POSTSUPERSCRIPT 2 end_POSTSUPERSCRIPT = 2 - italic_q start_POSTSUBSCRIPT italic_n end_POSTSUBSCRIPT .
We choose r 0 = 100 subscript 𝑟 0 100 r_{0}=100 italic_r start_POSTSUBSCRIPT 0 end_POSTSUBSCRIPT = 100 and r = 200 , 400 , 600 , 800 , 1000 𝑟 200 400 600 800 1000
r=200,400,600,800,1000 italic_r = 200 , 400 , 600 , 800 , 1000 . For each value of r 𝑟 r italic_r , we perform N = 1000 𝑁 1000 N=1000 italic_N = 1000 repetitions to calculate the mean squared error (MSE): 1 N ∑ i = 1 N ‖ 𝜷 ^ i − 𝜷 ‖ 2 1 𝑁 superscript subscript 𝑖 1 𝑁 superscript norm subscript ^ 𝜷 𝑖 𝜷 2 \frac{1}{N}\sum_{i=1}^{N}\|\hat{\boldsymbol{\beta}}_{i}-\boldsymbol{\beta}\|^{2} divide start_ARG 1 end_ARG start_ARG italic_N end_ARG ∑ start_POSTSUBSCRIPT italic_i = 1 end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_N end_POSTSUPERSCRIPT ∥ over^ start_ARG bold_italic_β end_ARG start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT - bold_italic_β ∥ start_POSTSUPERSCRIPT 2 end_POSTSUPERSCRIPT . For comparison, we consider six subsampling methods including the perturbation subsampling based on corrected likelihood in Algorithm 3 (CLEPS), A (or L)-optimal subsampling based on corrected likelihood in Algorithm 2 (A-opt and L-opt), uniform subsampling (UNIF), leverage subsampling (BLEV), and D-optimal subsampling (IBOSS). Measurement errors are not considered for UNIF, BLEV and IBOSS. To ensure equity, all methods except for A-opt and L-opt use r 0 + r subscript 𝑟 0 𝑟 r_{0}+r italic_r start_POSTSUBSCRIPT 0 end_POSTSUBSCRIPT + italic_r subsamples for parameter estimation.
Figure 2: The MSEs based on different σ u 2 superscript subscript 𝜎 𝑢 2 \sigma_{u}^{2} italic_σ start_POSTSUBSCRIPT italic_u end_POSTSUBSCRIPT start_POSTSUPERSCRIPT 2 end_POSTSUPERSCRIPT and σ ϵ 2 superscript subscript 𝜎 italic-ϵ 2 \sigma_{\epsilon}^{2} italic_σ start_POSTSUBSCRIPT italic_ϵ end_POSTSUBSCRIPT start_POSTSUPERSCRIPT 2 end_POSTSUPERSCRIPT for different r 𝑟 r italic_r .
The results presented in Figure 2 demonstrate the superior performance of CLEPS compared to other methods, closely followed by the A-opt and L-opt. Furthermore, as the subsample size increases, the MSEs of our proposed methods approach to zero, while other approaches that do not account for measurement errors show relatively stable MSEs. These results confirm the inconsistency of ordinary least squares estimator for linear models with measurement errors. Additionally, decreasing σ u 2 superscript subscript 𝜎 𝑢 2 \sigma_{u}^{2} italic_σ start_POSTSUBSCRIPT italic_u end_POSTSUBSCRIPT start_POSTSUPERSCRIPT 2 end_POSTSUPERSCRIPT and σ ϵ 2 superscript subscript 𝜎 italic-ϵ 2 \sigma_{\epsilon}^{2} italic_σ start_POSTSUBSCRIPT italic_ϵ end_POSTSUBSCRIPT start_POSTSUPERSCRIPT 2 end_POSTSUPERSCRIPT lead to a reductions for MSEs obtained by all six methods. Notably, even for big variance of the random error, our methods maintain good performance.
To further investigate the impact of other parameters on the sampling method, Figure 3 offers the variations in MSE across distinct values of r 0 , n , p , subscript 𝑟 0 𝑛 𝑝
r_{0},n,p, italic_r start_POSTSUBSCRIPT 0 end_POSTSUBSCRIPT , italic_n , italic_p , and m 𝑚 m italic_m while kee** σ ϵ 2 = 1 superscript subscript 𝜎 italic-ϵ 2 1 \sigma_{\epsilon}^{2}=1 italic_σ start_POSTSUBSCRIPT italic_ϵ end_POSTSUBSCRIPT start_POSTSUPERSCRIPT 2 end_POSTSUPERSCRIPT = 1 and σ u 2 = 0.4 superscript subscript 𝜎 𝑢 2 0.4 \sigma_{u}^{2}=0.4 italic_σ start_POSTSUBSCRIPT italic_u end_POSTSUBSCRIPT start_POSTSUPERSCRIPT 2 end_POSTSUPERSCRIPT = 0.4 . Additionally, Table 1 presents the results about different m 𝑚 m italic_m for CLEPS.
Figure 3: The plots except the bottom right present the MSEs for different r 0 , n , p subscript 𝑟 0 𝑛 𝑝
r_{0},n,p italic_r start_POSTSUBSCRIPT 0 end_POSTSUBSCRIPT , italic_n , italic_p , respectively. The bottom right plot presents the MSEs for different r 𝑟 r italic_r with m = 1 𝑚 1 m=1 italic_m = 1 .
In Figure 3 , the top left plot shows that as r 0 subscript 𝑟 0 r_{0} italic_r start_POSTSUBSCRIPT 0 end_POSTSUBSCRIPT ascends, the performance of A-opt and L-opt initially enhances and subsequently declines for fixed r 0 + r = 1000 subscript 𝑟 0 𝑟 1000 r_{0}+r=1000 italic_r start_POSTSUBSCRIPT 0 end_POSTSUBSCRIPT + italic_r = 1000 . This reason is that the inaccurate estimation is obtained in the first step when r 0 subscript 𝑟 0 r_{0} italic_r start_POSTSUBSCRIPT 0 end_POSTSUBSCRIPT is too small.
The top right plot indicates that as the sample size increases, the CLEPS performs more efficiently, while other methods remain almost unchanged.
The bottom left plot displays that as the dimension p 𝑝 p italic_p increases, the MSEs of various methods rises. When p > 15 𝑝 15 p>15 italic_p > 15 , A-opt and L-opt underperform compared to other methods, whereas the CLEPS consistently demonstrates the best performance.
The bottom right plot shows that when m = 1 𝑚 1 m=1 italic_m = 1 , the MSE of CLEPS is slightly bigger than the A-opt and L-opt.
Table 1: The MSEs for different m 𝑚 m italic_m .
In Table 1 , it is observed that as m 𝑚 m italic_m increases, the MSE decreases. However, the rate of reduction in MSE progressively diminishes. This suggests that m 𝑚 m italic_m should be significantly smaller than r 𝑟 r italic_r in order to achieve effective inference. It is advisable to set m < r / 10 𝑚 𝑟 10 m<r/10 italic_m < italic_r / 10 , in accordance with the findings in Shang and Cheng (2017 ) , Wang (2019b ) , and Wang and Ma (2021 ) , which suggest that the number of partitions should be significantly smaller than the sample size within each data partition.
S2 Proof of Theorem 1
Note that
𝜷 ~ − 𝜷 ^ ~ 𝜷 ^ 𝜷 \displaystyle\tilde{\boldsymbol{\beta}}-\hat{\boldsymbol{\beta}} over~ start_ARG bold_italic_β end_ARG - over^ start_ARG bold_italic_β end_ARG
= ( 1 n ∑ i = 1 r 1 r π i * 𝐖 i * 𝐖 i * T − 𝚺 u u ) − 1 ⋅ [ 1 n ∑ i = 1 r 1 r π i * 𝐖 i * ( y i * − 𝐖 i * T 𝜷 ^ ) + 𝚺 u u 𝜷 ^ ] absent ⋅ superscript 1 𝑛 superscript subscript 𝑖 1 𝑟 1 𝑟 superscript subscript 𝜋 𝑖 superscript subscript 𝐖 𝑖 superscript subscript 𝐖 𝑖 absent 𝑇 subscript 𝚺 𝑢 𝑢 1 delimited-[] 1 𝑛 superscript subscript 𝑖 1 𝑟 1 𝑟 superscript subscript 𝜋 𝑖 superscript subscript 𝐖 𝑖 superscript subscript 𝑦 𝑖 superscript subscript 𝐖 𝑖 absent 𝑇 ^ 𝜷 subscript 𝚺 𝑢 𝑢 ^ 𝜷 \displaystyle=(\frac{1}{n}\sum_{i=1}^{r}\frac{1}{r\pi_{i}^{*}}\mathbf{W}_{i}^{%
*}\mathbf{W}_{i}^{*T}-\boldsymbol{\Sigma}_{uu})^{-1}\cdot[\frac{1}{n}\sum_{i=1%
}^{r}\frac{1}{r\pi_{i}^{*}}\mathbf{W}_{i}^{*}(y_{i}^{*}-\mathbf{W}_{i}^{*T}%
\hat{\boldsymbol{\beta}})+\boldsymbol{\Sigma}_{uu}\hat{\boldsymbol{\beta}}] = ( divide start_ARG 1 end_ARG start_ARG italic_n end_ARG ∑ start_POSTSUBSCRIPT italic_i = 1 end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_r end_POSTSUPERSCRIPT divide start_ARG 1 end_ARG start_ARG italic_r italic_π start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT start_POSTSUPERSCRIPT * end_POSTSUPERSCRIPT end_ARG bold_W start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT start_POSTSUPERSCRIPT * end_POSTSUPERSCRIPT bold_W start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT start_POSTSUPERSCRIPT * italic_T end_POSTSUPERSCRIPT - bold_Σ start_POSTSUBSCRIPT italic_u italic_u end_POSTSUBSCRIPT ) start_POSTSUPERSCRIPT - 1 end_POSTSUPERSCRIPT ⋅ [ divide start_ARG 1 end_ARG start_ARG italic_n end_ARG ∑ start_POSTSUBSCRIPT italic_i = 1 end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_r end_POSTSUPERSCRIPT divide start_ARG 1 end_ARG start_ARG italic_r italic_π start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT start_POSTSUPERSCRIPT * end_POSTSUPERSCRIPT end_ARG bold_W start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT start_POSTSUPERSCRIPT * end_POSTSUPERSCRIPT ( italic_y start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT start_POSTSUPERSCRIPT * end_POSTSUPERSCRIPT - bold_W start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT start_POSTSUPERSCRIPT * italic_T end_POSTSUPERSCRIPT over^ start_ARG bold_italic_β end_ARG ) + bold_Σ start_POSTSUBSCRIPT italic_u italic_u end_POSTSUBSCRIPT over^ start_ARG bold_italic_β end_ARG ]
(S2.1)
= − ( ℋ ~ W ) − 1 ℓ ˙ * ( 𝜷 ^ ) , absent superscript subscript ~ ℋ 𝑊 1 superscript ˙ ℓ ^ 𝜷 \displaystyle=-(\tilde{\mathcal{H}}_{W})^{-1}\dot{\ell}^{*}(\hat{\boldsymbol{%
\beta}}), = - ( over~ start_ARG caligraphic_H end_ARG start_POSTSUBSCRIPT italic_W end_POSTSUBSCRIPT ) start_POSTSUPERSCRIPT - 1 end_POSTSUPERSCRIPT over˙ start_ARG roman_ℓ end_ARG start_POSTSUPERSCRIPT * end_POSTSUPERSCRIPT ( over^ start_ARG bold_italic_β end_ARG ) ,
where ℋ ~ W = 1 r ∑ i = 1 r [ 1 n π i * 𝐖 i * 𝐖 i * T − 𝚺 u u ] , ℓ ˙ * ( 𝜷 ) = 1 n ∑ i = 1 r 1 r π i * [ − 𝐖 i * ( y i * − 𝐖 i * T 𝜷 ) ] − 𝚺 u u 𝜷 . formulae-sequence subscript ~ ℋ 𝑊 1 𝑟 superscript subscript 𝑖 1 𝑟 delimited-[] 1 𝑛 superscript subscript 𝜋 𝑖 superscript subscript 𝐖 𝑖 superscript subscript 𝐖 𝑖 absent 𝑇 subscript 𝚺 𝑢 𝑢 superscript ˙ ℓ 𝜷 1 𝑛 superscript subscript 𝑖 1 𝑟 1 𝑟 superscript subscript 𝜋 𝑖 delimited-[] superscript subscript 𝐖 𝑖 superscript subscript 𝑦 𝑖 superscript subscript 𝐖 𝑖 absent 𝑇 𝜷 subscript 𝚺 𝑢 𝑢 𝜷 \tilde{\mathcal{H}}_{W}=\frac{1}{r}\sum_{i=1}^{r}[\frac{1}{n\pi_{i}^{*}}%
\mathbf{W}_{i}^{*}\mathbf{W}_{i}^{*T}-\boldsymbol{\Sigma}_{uu}],\dot{\ell}^{*}%
(\boldsymbol{\beta})=\frac{1}{n}\sum_{i=1}^{r}\frac{1}{r\pi_{i}^{*}}[-\mathbf{%
W}_{i}^{*}(y_{i}^{*}-\mathbf{W}_{i}^{*T}\boldsymbol{\beta})]-\boldsymbol{%
\Sigma}_{uu}\boldsymbol{\beta}. over~ start_ARG caligraphic_H end_ARG start_POSTSUBSCRIPT italic_W end_POSTSUBSCRIPT = divide start_ARG 1 end_ARG start_ARG italic_r end_ARG ∑ start_POSTSUBSCRIPT italic_i = 1 end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_r end_POSTSUPERSCRIPT [ divide start_ARG 1 end_ARG start_ARG italic_n italic_π start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT start_POSTSUPERSCRIPT * end_POSTSUPERSCRIPT end_ARG bold_W start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT start_POSTSUPERSCRIPT * end_POSTSUPERSCRIPT bold_W start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT start_POSTSUPERSCRIPT * italic_T end_POSTSUPERSCRIPT - bold_Σ start_POSTSUBSCRIPT italic_u italic_u end_POSTSUBSCRIPT ] , over˙ start_ARG roman_ℓ end_ARG start_POSTSUPERSCRIPT * end_POSTSUPERSCRIPT ( bold_italic_β ) = divide start_ARG 1 end_ARG start_ARG italic_n end_ARG ∑ start_POSTSUBSCRIPT italic_i = 1 end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_r end_POSTSUPERSCRIPT divide start_ARG 1 end_ARG start_ARG italic_r italic_π start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT start_POSTSUPERSCRIPT * end_POSTSUPERSCRIPT end_ARG [ - bold_W start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT start_POSTSUPERSCRIPT * end_POSTSUPERSCRIPT ( italic_y start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT start_POSTSUPERSCRIPT * end_POSTSUPERSCRIPT - bold_W start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT start_POSTSUPERSCRIPT * italic_T end_POSTSUPERSCRIPT bold_italic_β ) ] - bold_Σ start_POSTSUBSCRIPT italic_u italic_u end_POSTSUBSCRIPT bold_italic_β .
Therefore, we only need to prove
ℓ ˙ * ( 𝜷 ^ ) = O P | ℱ n ( r − 1 / 2 ) , superscript ˙ ℓ ^ 𝜷 subscript 𝑂 conditional 𝑃 subscript ℱ 𝑛 superscript 𝑟 1 2 \dot{\ell}^{*}(\hat{\boldsymbol{\beta}})=O_{P|\mathcal{F}_{n}}(r^{-1/2}), over˙ start_ARG roman_ℓ end_ARG start_POSTSUPERSCRIPT * end_POSTSUPERSCRIPT ( over^ start_ARG bold_italic_β end_ARG ) = italic_O start_POSTSUBSCRIPT italic_P | caligraphic_F start_POSTSUBSCRIPT italic_n end_POSTSUBSCRIPT end_POSTSUBSCRIPT ( italic_r start_POSTSUPERSCRIPT - 1 / 2 end_POSTSUPERSCRIPT ) ,
(S2.2)
and
ℋ ~ W − ℋ W = O P | ℱ n ( r − 1 / 2 ) , subscript ~ ℋ 𝑊 subscript ℋ 𝑊 subscript 𝑂 conditional 𝑃 subscript ℱ 𝑛 superscript 𝑟 1 2 \tilde{\mathcal{H}}_{W}-\mathcal{H}_{W}=O_{P|\mathcal{F}_{n}}(r^{-1/2}), over~ start_ARG caligraphic_H end_ARG start_POSTSUBSCRIPT italic_W end_POSTSUBSCRIPT - caligraphic_H start_POSTSUBSCRIPT italic_W end_POSTSUBSCRIPT = italic_O start_POSTSUBSCRIPT italic_P | caligraphic_F start_POSTSUBSCRIPT italic_n end_POSTSUBSCRIPT end_POSTSUBSCRIPT ( italic_r start_POSTSUPERSCRIPT - 1 / 2 end_POSTSUPERSCRIPT ) ,
(S2.3)
where ℋ W = 1 n ∑ i = 1 n 𝐖 i 𝐖 i T − 𝚺 u u . subscript ℋ 𝑊 1 𝑛 superscript subscript 𝑖 1 𝑛 subscript 𝐖 𝑖 superscript subscript 𝐖 𝑖 𝑇 subscript 𝚺 𝑢 𝑢 \mathcal{H}_{W}=\frac{1}{n}\sum_{i=1}^{n}\mathbf{W}_{i}\mathbf{W}_{i}^{T}-%
\boldsymbol{\Sigma}_{uu}. caligraphic_H start_POSTSUBSCRIPT italic_W end_POSTSUBSCRIPT = divide start_ARG 1 end_ARG start_ARG italic_n end_ARG ∑ start_POSTSUBSCRIPT italic_i = 1 end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_n end_POSTSUPERSCRIPT bold_W start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT bold_W start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_T end_POSTSUPERSCRIPT - bold_Σ start_POSTSUBSCRIPT italic_u italic_u end_POSTSUBSCRIPT .
To prove (S2.2 ), we directly calculate
E ( ℓ ˙ * ( 𝜷 ^ ) | ℱ n ) 𝐸 conditional superscript ˙ ℓ ^ 𝜷 subscript ℱ 𝑛 \displaystyle E(\dot{\ell}^{*}(\hat{\boldsymbol{\beta}})|\mathcal{F}_{n}) italic_E ( over˙ start_ARG roman_ℓ end_ARG start_POSTSUPERSCRIPT * end_POSTSUPERSCRIPT ( over^ start_ARG bold_italic_β end_ARG ) | caligraphic_F start_POSTSUBSCRIPT italic_n end_POSTSUBSCRIPT )
= E { 1 n ∑ i = 1 r 1 r π i * [ − 𝐖 i * ( y i * − 𝐖 i * T 𝜷 ^ ) ] − 𝚺 u u 𝜷 ^ | ℱ n } absent 𝐸 conditional-set 1 𝑛 superscript subscript 𝑖 1 𝑟 1 𝑟 superscript subscript 𝜋 𝑖 delimited-[] superscript subscript 𝐖 𝑖 superscript subscript 𝑦 𝑖 superscript subscript 𝐖 𝑖 absent 𝑇 ^ 𝜷 subscript 𝚺 𝑢 𝑢 ^ 𝜷 subscript ℱ 𝑛 \displaystyle=E\left\{\frac{1}{n}\sum_{i=1}^{r}\frac{1}{r\pi_{i}^{*}}[-\mathbf%
{W}_{i}^{*}(y_{i}^{*}-\mathbf{W}_{i}^{*T}\hat{\boldsymbol{\beta}})]-%
\boldsymbol{\Sigma}_{uu}\hat{\boldsymbol{\beta}}\bigg{|}\mathcal{F}_{n}\right\} = italic_E { divide start_ARG 1 end_ARG start_ARG italic_n end_ARG ∑ start_POSTSUBSCRIPT italic_i = 1 end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_r end_POSTSUPERSCRIPT divide start_ARG 1 end_ARG start_ARG italic_r italic_π start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT start_POSTSUPERSCRIPT * end_POSTSUPERSCRIPT end_ARG [ - bold_W start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT start_POSTSUPERSCRIPT * end_POSTSUPERSCRIPT ( italic_y start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT start_POSTSUPERSCRIPT * end_POSTSUPERSCRIPT - bold_W start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT start_POSTSUPERSCRIPT * italic_T end_POSTSUPERSCRIPT over^ start_ARG bold_italic_β end_ARG ) ] - bold_Σ start_POSTSUBSCRIPT italic_u italic_u end_POSTSUBSCRIPT over^ start_ARG bold_italic_β end_ARG | caligraphic_F start_POSTSUBSCRIPT italic_n end_POSTSUBSCRIPT }
= 1 n r ∑ i = 1 r E { 1 π i * [ − 𝐖 i * ( y i * − 𝐖 i * T 𝜷 ^ ) ] | ℱ n } − 𝚺 u u 𝜷 ^ absent 1 𝑛 𝑟 superscript subscript 𝑖 1 𝑟 𝐸 conditional-set 1 superscript subscript 𝜋 𝑖 delimited-[] superscript subscript 𝐖 𝑖 superscript subscript 𝑦 𝑖 superscript subscript 𝐖 𝑖 absent 𝑇 ^ 𝜷 subscript ℱ 𝑛 subscript 𝚺 𝑢 𝑢 ^ 𝜷 \displaystyle=\frac{1}{nr}\sum_{i=1}^{r}E\left\{\frac{1}{\pi_{i}^{*}}[-\mathbf%
{W}_{i}^{*}(y_{i}^{*}-\mathbf{W}_{i}^{*T}\hat{\boldsymbol{\beta}})]\Big{|}%
\mathcal{F}_{n}\right\}-\boldsymbol{\Sigma}_{uu}\hat{\boldsymbol{\beta}} = divide start_ARG 1 end_ARG start_ARG italic_n italic_r end_ARG ∑ start_POSTSUBSCRIPT italic_i = 1 end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_r end_POSTSUPERSCRIPT italic_E { divide start_ARG 1 end_ARG start_ARG italic_π start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT start_POSTSUPERSCRIPT * end_POSTSUPERSCRIPT end_ARG [ - bold_W start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT start_POSTSUPERSCRIPT * end_POSTSUPERSCRIPT ( italic_y start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT start_POSTSUPERSCRIPT * end_POSTSUPERSCRIPT - bold_W start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT start_POSTSUPERSCRIPT * italic_T end_POSTSUPERSCRIPT over^ start_ARG bold_italic_β end_ARG ) ] | caligraphic_F start_POSTSUBSCRIPT italic_n end_POSTSUBSCRIPT } - bold_Σ start_POSTSUBSCRIPT italic_u italic_u end_POSTSUBSCRIPT over^ start_ARG bold_italic_β end_ARG
= 1 n ∑ i = 1 n π i ⋅ 1 π i [ − 𝐖 i ( y i − 𝐖 i T 𝜷 ^ ) ] − 𝚺 u u 𝜷 ^ absent 1 𝑛 superscript subscript 𝑖 1 𝑛 ⋅ subscript 𝜋 𝑖 1 subscript 𝜋 𝑖 delimited-[] subscript 𝐖 𝑖 subscript 𝑦 𝑖 superscript subscript 𝐖 𝑖 𝑇 ^ 𝜷 subscript 𝚺 𝑢 𝑢 ^ 𝜷 \displaystyle=\frac{1}{n}\sum_{i=1}^{n}\pi_{i}\cdot\frac{1}{\pi_{i}}[-\mathbf{%
W}_{i}(y_{i}-\mathbf{W}_{i}^{T}\hat{\boldsymbol{\beta}})]-\boldsymbol{\Sigma}_%
{uu}\hat{\boldsymbol{\beta}} = divide start_ARG 1 end_ARG start_ARG italic_n end_ARG ∑ start_POSTSUBSCRIPT italic_i = 1 end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_n end_POSTSUPERSCRIPT italic_π start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT ⋅ divide start_ARG 1 end_ARG start_ARG italic_π start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT end_ARG [ - bold_W start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT ( italic_y start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT - bold_W start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_T end_POSTSUPERSCRIPT over^ start_ARG bold_italic_β end_ARG ) ] - bold_Σ start_POSTSUBSCRIPT italic_u italic_u end_POSTSUBSCRIPT over^ start_ARG bold_italic_β end_ARG
= 1 n ∑ i = 1 n [ − 𝐖 i ( y i − 𝐖 i T 𝜷 ^ ) ] − 𝚺 u u 𝜷 ^ absent 1 𝑛 superscript subscript 𝑖 1 𝑛 delimited-[] subscript 𝐖 𝑖 subscript 𝑦 𝑖 superscript subscript 𝐖 𝑖 𝑇 ^ 𝜷 subscript 𝚺 𝑢 𝑢 ^ 𝜷 \displaystyle=\frac{1}{n}\sum_{i=1}^{n}[-\mathbf{W}_{i}(y_{i}-\mathbf{W}_{i}^{%
T}\hat{\boldsymbol{\beta}})]-\boldsymbol{\Sigma}_{uu}\hat{\boldsymbol{\beta}} = divide start_ARG 1 end_ARG start_ARG italic_n end_ARG ∑ start_POSTSUBSCRIPT italic_i = 1 end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_n end_POSTSUPERSCRIPT [ - bold_W start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT ( italic_y start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT - bold_W start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_T end_POSTSUPERSCRIPT over^ start_ARG bold_italic_β end_ARG ) ] - bold_Σ start_POSTSUBSCRIPT italic_u italic_u end_POSTSUBSCRIPT over^ start_ARG bold_italic_β end_ARG
= 𝟎 . absent 0 \displaystyle=\mathbf{0}. = bold_0 .
For the j 𝑗 j italic_j -th element ℓ ˙ j * ( 𝜷 ^ ) superscript subscript ˙ ℓ 𝑗 ^ 𝜷 \dot{\ell}_{j}^{*}(\hat{\boldsymbol{\beta}}) over˙ start_ARG roman_ℓ end_ARG start_POSTSUBSCRIPT italic_j end_POSTSUBSCRIPT start_POSTSUPERSCRIPT * end_POSTSUPERSCRIPT ( over^ start_ARG bold_italic_β end_ARG ) of ℓ ˙ * ( 𝜷 ^ ) superscript ˙ ℓ ^ 𝜷 \dot{\ell}^{*}(\hat{\boldsymbol{\beta}}) over˙ start_ARG roman_ℓ end_ARG start_POSTSUPERSCRIPT * end_POSTSUPERSCRIPT ( over^ start_ARG bold_italic_β end_ARG ) where 1 ≤ j ≤ p 1 𝑗 𝑝 1\leq j\leq p 1 ≤ italic_j ≤ italic_p ,
V a r ( ℓ ˙ j * ( 𝜷 ^ ) | ℱ n ) = 𝑉 𝑎 𝑟 conditional superscript subscript ˙ ℓ 𝑗 ^ 𝜷 subscript ℱ 𝑛 absent \displaystyle Var(\dot{\ell}_{j}^{*}(\hat{\boldsymbol{\beta}})|\mathcal{F}_{n})= italic_V italic_a italic_r ( over˙ start_ARG roman_ℓ end_ARG start_POSTSUBSCRIPT italic_j end_POSTSUBSCRIPT start_POSTSUPERSCRIPT * end_POSTSUPERSCRIPT ( over^ start_ARG bold_italic_β end_ARG ) | caligraphic_F start_POSTSUBSCRIPT italic_n end_POSTSUBSCRIPT ) =
E { 1 n ∑ i = 1 r 1 r π i * [ − w i j * ( y i * − 𝐖 i * T 𝜷 ^ ) ] − ( 𝚺 u u 𝜷 ^ ) j | ℱ n } 2 𝐸 superscript conditional-set 1 𝑛 superscript subscript 𝑖 1 𝑟 1 𝑟 superscript subscript 𝜋 𝑖 delimited-[] superscript subscript 𝑤 𝑖 𝑗 superscript subscript 𝑦 𝑖 superscript subscript 𝐖 𝑖 absent 𝑇 ^ 𝜷 subscript subscript 𝚺 𝑢 𝑢 ^ 𝜷 𝑗 subscript ℱ 𝑛 2 \displaystyle E\left\{\frac{1}{n}\sum_{i=1}^{r}\frac{1}{r\pi_{i}^{*}}[-w_{ij}^%
{*}(y_{i}^{*}-\mathbf{W}_{i}^{*T}\hat{\boldsymbol{\beta}})]-(\boldsymbol{%
\Sigma}_{uu}\hat{\boldsymbol{\beta}})_{j}\bigg{|}\mathcal{F}_{n}\right\}^{2} italic_E { divide start_ARG 1 end_ARG start_ARG italic_n end_ARG ∑ start_POSTSUBSCRIPT italic_i = 1 end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_r end_POSTSUPERSCRIPT divide start_ARG 1 end_ARG start_ARG italic_r italic_π start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT start_POSTSUPERSCRIPT * end_POSTSUPERSCRIPT end_ARG [ - italic_w start_POSTSUBSCRIPT italic_i italic_j end_POSTSUBSCRIPT start_POSTSUPERSCRIPT * end_POSTSUPERSCRIPT ( italic_y start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT start_POSTSUPERSCRIPT * end_POSTSUPERSCRIPT - bold_W start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT start_POSTSUPERSCRIPT * italic_T end_POSTSUPERSCRIPT over^ start_ARG bold_italic_β end_ARG ) ] - ( bold_Σ start_POSTSUBSCRIPT italic_u italic_u end_POSTSUBSCRIPT over^ start_ARG bold_italic_β end_ARG ) start_POSTSUBSCRIPT italic_j end_POSTSUBSCRIPT | caligraphic_F start_POSTSUBSCRIPT italic_n end_POSTSUBSCRIPT } start_POSTSUPERSCRIPT 2 end_POSTSUPERSCRIPT
= \displaystyle= =
E { 1 r ∑ i = 1 r { 1 n π i * [ − w i j * ( y i * − 𝐖 i * T 𝜷 ^ ) ] − ( 𝚺 u u 𝜷 ^ ) j } | ℱ n } 2 𝐸 superscript conditional-set 1 𝑟 superscript subscript 𝑖 1 𝑟 1 𝑛 superscript subscript 𝜋 𝑖 delimited-[] superscript subscript 𝑤 𝑖 𝑗 superscript subscript 𝑦 𝑖 superscript subscript 𝐖 𝑖 absent 𝑇 ^ 𝜷 subscript subscript 𝚺 𝑢 𝑢 ^ 𝜷 𝑗 subscript ℱ 𝑛 2 \displaystyle E\left\{\frac{1}{r}\sum_{i=1}^{r}\left\{\frac{1}{n\pi_{i}^{*}}[-%
w_{ij}^{*}(y_{i}^{*}-\mathbf{W}_{i}^{*T}\hat{\boldsymbol{\beta}})]-(%
\boldsymbol{\Sigma}_{uu}\hat{\boldsymbol{\beta}})_{j}\right\}\bigg{|}\mathcal{%
F}_{n}\right\}^{2} italic_E { divide start_ARG 1 end_ARG start_ARG italic_r end_ARG ∑ start_POSTSUBSCRIPT italic_i = 1 end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_r end_POSTSUPERSCRIPT { divide start_ARG 1 end_ARG start_ARG italic_n italic_π start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT start_POSTSUPERSCRIPT * end_POSTSUPERSCRIPT end_ARG [ - italic_w start_POSTSUBSCRIPT italic_i italic_j end_POSTSUBSCRIPT start_POSTSUPERSCRIPT * end_POSTSUPERSCRIPT ( italic_y start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT start_POSTSUPERSCRIPT * end_POSTSUPERSCRIPT - bold_W start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT start_POSTSUPERSCRIPT * italic_T end_POSTSUPERSCRIPT over^ start_ARG bold_italic_β end_ARG ) ] - ( bold_Σ start_POSTSUBSCRIPT italic_u italic_u end_POSTSUBSCRIPT over^ start_ARG bold_italic_β end_ARG ) start_POSTSUBSCRIPT italic_j end_POSTSUBSCRIPT } | caligraphic_F start_POSTSUBSCRIPT italic_n end_POSTSUBSCRIPT } start_POSTSUPERSCRIPT 2 end_POSTSUPERSCRIPT
= \displaystyle= =
1 r 2 ∑ i = 1 r E { 1 n π i * [ − w i j * ( y i * − 𝐖 i * T 𝜷 ^ ) ] − ( 𝚺 u u 𝜷 ^ ) j | ℱ n } 2 1 superscript 𝑟 2 superscript subscript 𝑖 1 𝑟 𝐸 superscript conditional-set 1 𝑛 superscript subscript 𝜋 𝑖 delimited-[] superscript subscript 𝑤 𝑖 𝑗 superscript subscript 𝑦 𝑖 superscript subscript 𝐖 𝑖 absent 𝑇 ^ 𝜷 subscript subscript 𝚺 𝑢 𝑢 ^ 𝜷 𝑗 subscript ℱ 𝑛 2 \displaystyle\frac{1}{r^{2}}\sum_{i=1}^{r}E\left\{\frac{1}{n\pi_{i}^{*}}[-w_{%
ij}^{*}(y_{i}^{*}-\mathbf{W}_{i}^{*T}\hat{\boldsymbol{\beta}})]-(\boldsymbol{%
\Sigma}_{uu}\hat{\boldsymbol{\beta}})_{j}\Big{|}\mathcal{F}_{n}\right\}^{2} divide start_ARG 1 end_ARG start_ARG italic_r start_POSTSUPERSCRIPT 2 end_POSTSUPERSCRIPT end_ARG ∑ start_POSTSUBSCRIPT italic_i = 1 end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_r end_POSTSUPERSCRIPT italic_E { divide start_ARG 1 end_ARG start_ARG italic_n italic_π start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT start_POSTSUPERSCRIPT * end_POSTSUPERSCRIPT end_ARG [ - italic_w start_POSTSUBSCRIPT italic_i italic_j end_POSTSUBSCRIPT start_POSTSUPERSCRIPT * end_POSTSUPERSCRIPT ( italic_y start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT start_POSTSUPERSCRIPT * end_POSTSUPERSCRIPT - bold_W start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT start_POSTSUPERSCRIPT * italic_T end_POSTSUPERSCRIPT over^ start_ARG bold_italic_β end_ARG ) ] - ( bold_Σ start_POSTSUBSCRIPT italic_u italic_u end_POSTSUBSCRIPT over^ start_ARG bold_italic_β end_ARG ) start_POSTSUBSCRIPT italic_j end_POSTSUBSCRIPT | caligraphic_F start_POSTSUBSCRIPT italic_n end_POSTSUBSCRIPT } start_POSTSUPERSCRIPT 2 end_POSTSUPERSCRIPT
= \displaystyle= =
1 r { E { 1 n π i * [ − w i j * ( y i * − 𝐖 i * T 𝜷 ^ ) ] | ℱ n } 2 + E [ ( 𝚺 u u 𝜷 ^ ) j | ℱ n ] 2 \displaystyle\frac{1}{r}\Bigg{\{}E\left\{\frac{1}{n\pi_{i}^{*}}[-w_{ij}^{*}(y_%
{i}^{*}-\mathbf{W}_{i}^{*T}\hat{\boldsymbol{\beta}})]\Big{|}\mathcal{F}_{n}%
\right\}^{2}+E[(\boldsymbol{\Sigma}_{uu}\hat{\boldsymbol{\beta}})_{j}|\mathcal%
{F}_{n}]^{2} divide start_ARG 1 end_ARG start_ARG italic_r end_ARG { italic_E { divide start_ARG 1 end_ARG start_ARG italic_n italic_π start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT start_POSTSUPERSCRIPT * end_POSTSUPERSCRIPT end_ARG [ - italic_w start_POSTSUBSCRIPT italic_i italic_j end_POSTSUBSCRIPT start_POSTSUPERSCRIPT * end_POSTSUPERSCRIPT ( italic_y start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT start_POSTSUPERSCRIPT * end_POSTSUPERSCRIPT - bold_W start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT start_POSTSUPERSCRIPT * italic_T end_POSTSUPERSCRIPT over^ start_ARG bold_italic_β end_ARG ) ] | caligraphic_F start_POSTSUBSCRIPT italic_n end_POSTSUBSCRIPT } start_POSTSUPERSCRIPT 2 end_POSTSUPERSCRIPT + italic_E [ ( bold_Σ start_POSTSUBSCRIPT italic_u italic_u end_POSTSUBSCRIPT over^ start_ARG bold_italic_β end_ARG ) start_POSTSUBSCRIPT italic_j end_POSTSUBSCRIPT | caligraphic_F start_POSTSUBSCRIPT italic_n end_POSTSUBSCRIPT ] start_POSTSUPERSCRIPT 2 end_POSTSUPERSCRIPT
− 2 E [ 1 n π i * [ − w i j * ( y i * − 𝐖 i * T 𝜷 ^ ) ] ( 𝚺 u u 𝜷 ^ ) j | ℱ n ] } \displaystyle-2E\left[\frac{1}{n\pi_{i}^{*}}[-w_{ij}^{*}(y_{i}^{*}-\mathbf{W}_%
{i}^{*T}\hat{\boldsymbol{\beta}})](\boldsymbol{\Sigma}_{uu}\hat{\boldsymbol{%
\beta}})_{j}\Big{|}\mathcal{F}_{n}\right]\Bigg{\}} - 2 italic_E [ divide start_ARG 1 end_ARG start_ARG italic_n italic_π start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT start_POSTSUPERSCRIPT * end_POSTSUPERSCRIPT end_ARG [ - italic_w start_POSTSUBSCRIPT italic_i italic_j end_POSTSUBSCRIPT start_POSTSUPERSCRIPT * end_POSTSUPERSCRIPT ( italic_y start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT start_POSTSUPERSCRIPT * end_POSTSUPERSCRIPT - bold_W start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT start_POSTSUPERSCRIPT * italic_T end_POSTSUPERSCRIPT over^ start_ARG bold_italic_β end_ARG ) ] ( bold_Σ start_POSTSUBSCRIPT italic_u italic_u end_POSTSUBSCRIPT over^ start_ARG bold_italic_β end_ARG ) start_POSTSUBSCRIPT italic_j end_POSTSUBSCRIPT | caligraphic_F start_POSTSUBSCRIPT italic_n end_POSTSUBSCRIPT ] }
= \displaystyle= =
1 n 2 r ∑ i = 1 n π i ⋅ { 1 π i [ − w i j ( y i − 𝐖 i T 𝜷 ^ ) ] } 2 − 1 r ( 𝚺 u u 𝜷 ^ ) j 2 1 superscript 𝑛 2 𝑟 superscript subscript 𝑖 1 𝑛 ⋅ subscript 𝜋 𝑖 superscript 1 subscript 𝜋 𝑖 delimited-[] subscript 𝑤 𝑖 𝑗 subscript 𝑦 𝑖 superscript subscript 𝐖 𝑖 𝑇 ^ 𝜷 2 1 𝑟 superscript subscript subscript 𝚺 𝑢 𝑢 ^ 𝜷 𝑗 2 \displaystyle\frac{1}{n^{2}r}\sum_{i=1}^{n}\pi_{i}\cdot\left\{\frac{1}{\pi_{i}%
}[-w_{ij}(y_{i}-\mathbf{W}_{i}^{T}\hat{\boldsymbol{\beta}})]\right\}^{2}-\frac%
{1}{r}(\boldsymbol{\Sigma}_{uu}\hat{\boldsymbol{\beta}})_{j}^{2} divide start_ARG 1 end_ARG start_ARG italic_n start_POSTSUPERSCRIPT 2 end_POSTSUPERSCRIPT italic_r end_ARG ∑ start_POSTSUBSCRIPT italic_i = 1 end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_n end_POSTSUPERSCRIPT italic_π start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT ⋅ { divide start_ARG 1 end_ARG start_ARG italic_π start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT end_ARG [ - italic_w start_POSTSUBSCRIPT italic_i italic_j end_POSTSUBSCRIPT ( italic_y start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT - bold_W start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_T end_POSTSUPERSCRIPT over^ start_ARG bold_italic_β end_ARG ) ] } start_POSTSUPERSCRIPT 2 end_POSTSUPERSCRIPT - divide start_ARG 1 end_ARG start_ARG italic_r end_ARG ( bold_Σ start_POSTSUBSCRIPT italic_u italic_u end_POSTSUBSCRIPT over^ start_ARG bold_italic_β end_ARG ) start_POSTSUBSCRIPT italic_j end_POSTSUBSCRIPT start_POSTSUPERSCRIPT 2 end_POSTSUPERSCRIPT
= \displaystyle= =
1 n 2 r ∑ i = 1 n w i j 2 ( y i − 𝐖 i T 𝜷 ^ ) 2 π i − 1 r ( 𝚺 u u 𝜷 ^ ) j 2 1 superscript 𝑛 2 𝑟 superscript subscript 𝑖 1 𝑛 superscript subscript 𝑤 𝑖 𝑗 2 superscript subscript 𝑦 𝑖 superscript subscript 𝐖 𝑖 𝑇 ^ 𝜷 2 subscript 𝜋 𝑖 1 𝑟 superscript subscript subscript 𝚺 𝑢 𝑢 ^ 𝜷 𝑗 2 \displaystyle\frac{1}{n^{2}r}\sum_{i=1}^{n}\frac{w_{ij}^{2}(y_{i}-\mathbf{W}_{%
i}^{T}\hat{\boldsymbol{\beta}})^{2}}{\pi_{i}}-\frac{1}{r}(\boldsymbol{\Sigma}_%
{uu}\hat{\boldsymbol{\beta}})_{j}^{2} divide start_ARG 1 end_ARG start_ARG italic_n start_POSTSUPERSCRIPT 2 end_POSTSUPERSCRIPT italic_r end_ARG ∑ start_POSTSUBSCRIPT italic_i = 1 end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_n end_POSTSUPERSCRIPT divide start_ARG italic_w start_POSTSUBSCRIPT italic_i italic_j end_POSTSUBSCRIPT start_POSTSUPERSCRIPT 2 end_POSTSUPERSCRIPT ( italic_y start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT - bold_W start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_T end_POSTSUPERSCRIPT over^ start_ARG bold_italic_β end_ARG ) start_POSTSUPERSCRIPT 2 end_POSTSUPERSCRIPT end_ARG start_ARG italic_π start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT end_ARG - divide start_ARG 1 end_ARG start_ARG italic_r end_ARG ( bold_Σ start_POSTSUBSCRIPT italic_u italic_u end_POSTSUBSCRIPT over^ start_ARG bold_italic_β end_ARG ) start_POSTSUBSCRIPT italic_j end_POSTSUBSCRIPT start_POSTSUPERSCRIPT 2 end_POSTSUPERSCRIPT
≤ \displaystyle\leq ≤
1 r max i = 1 , … , n ( n π i ) − 1 ∑ i = 1 n ‖ 𝐖 i ‖ 2 ( y i − 𝐖 i T 𝜷 ^ ) 2 n − 1 r ( 𝚺 u u 𝜷 ^ ) j 2 . \displaystyle\frac{1}{r}\max_{i=1,\ldots,n}(n\pi_{i})^{-1}\sum_{i=1}^{n}\frac{%
\|\mathbf{W}_{i}\|^{2}(y_{i}-\mathbf{W}_{i}^{T}\hat{\boldsymbol{\beta}})^{2}}{%
n}-\frac{1}{r}(\boldsymbol{\Sigma}_{uu}\hat{\boldsymbol{\beta}})_{j}^{2}. divide start_ARG 1 end_ARG start_ARG italic_r end_ARG roman_max start_POSTSUBSCRIPT italic_i = 1 , … , italic_n end_POSTSUBSCRIPT ( italic_n italic_π start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT ) start_POSTSUPERSCRIPT - 1 end_POSTSUPERSCRIPT ∑ start_POSTSUBSCRIPT italic_i = 1 end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_n end_POSTSUPERSCRIPT divide start_ARG ∥ bold_W start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT ∥ start_POSTSUPERSCRIPT 2 end_POSTSUPERSCRIPT ( italic_y start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT - bold_W start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_T end_POSTSUPERSCRIPT over^ start_ARG bold_italic_β end_ARG ) start_POSTSUPERSCRIPT 2 end_POSTSUPERSCRIPT end_ARG start_ARG italic_n end_ARG - divide start_ARG 1 end_ARG start_ARG italic_r end_ARG ( bold_Σ start_POSTSUBSCRIPT italic_u italic_u end_POSTSUBSCRIPT over^ start_ARG bold_italic_β end_ARG ) start_POSTSUBSCRIPT italic_j end_POSTSUBSCRIPT start_POSTSUPERSCRIPT 2 end_POSTSUPERSCRIPT .
By Assumption 2 and Holder inequality, we can achieve
∑ i = 1 n ‖ 𝐖 i ‖ 2 ( y i − 𝐖 i T 𝜷 ^ ) 2 n ≤ ( ∑ i = 1 n ‖ 𝐖 i ‖ 4 n ) 1 / 2 ( ∑ i = 1 n ( y i − 𝐖 i T 𝜷 ^ ) 4 n ) 1 / 2 = O P ( 1 ) . superscript subscript 𝑖 1 𝑛 superscript norm subscript 𝐖 𝑖 2 superscript subscript 𝑦 𝑖 superscript subscript 𝐖 𝑖 𝑇 ^ 𝜷 2 𝑛 superscript superscript subscript 𝑖 1 𝑛 superscript norm subscript 𝐖 𝑖 4 𝑛 1 2 superscript superscript subscript 𝑖 1 𝑛 superscript subscript 𝑦 𝑖 superscript subscript 𝐖 𝑖 𝑇 ^ 𝜷 4 𝑛 1 2 subscript 𝑂 𝑃 1 \sum_{i=1}^{n}\frac{\|\mathbf{W}_{i}\|^{2}(y_{i}-\mathbf{W}_{i}^{T}\hat{%
\boldsymbol{\beta}})^{2}}{n}\leq\left(\sum_{i=1}^{n}\frac{\|\mathbf{W}_{i}\|^{%
4}}{n}\right)^{1/2}\left(\sum_{i=1}^{n}\frac{(y_{i}-\mathbf{W}_{i}^{T}\hat{%
\boldsymbol{\beta}})^{4}}{n}\right)^{1/2}=O_{P}(1). ∑ start_POSTSUBSCRIPT italic_i = 1 end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_n end_POSTSUPERSCRIPT divide start_ARG ∥ bold_W start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT ∥ start_POSTSUPERSCRIPT 2 end_POSTSUPERSCRIPT ( italic_y start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT - bold_W start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_T end_POSTSUPERSCRIPT over^ start_ARG bold_italic_β end_ARG ) start_POSTSUPERSCRIPT 2 end_POSTSUPERSCRIPT end_ARG start_ARG italic_n end_ARG ≤ ( ∑ start_POSTSUBSCRIPT italic_i = 1 end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_n end_POSTSUPERSCRIPT divide start_ARG ∥ bold_W start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT ∥ start_POSTSUPERSCRIPT 4 end_POSTSUPERSCRIPT end_ARG start_ARG italic_n end_ARG ) start_POSTSUPERSCRIPT 1 / 2 end_POSTSUPERSCRIPT ( ∑ start_POSTSUBSCRIPT italic_i = 1 end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_n end_POSTSUPERSCRIPT divide start_ARG ( italic_y start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT - bold_W start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_T end_POSTSUPERSCRIPT over^ start_ARG bold_italic_β end_ARG ) start_POSTSUPERSCRIPT 4 end_POSTSUPERSCRIPT end_ARG start_ARG italic_n end_ARG ) start_POSTSUPERSCRIPT 1 / 2 end_POSTSUPERSCRIPT = italic_O start_POSTSUBSCRIPT italic_P end_POSTSUBSCRIPT ( 1 ) .
(S2.4)
According to Assumption 3, we can infer that V a r ( ℓ ˙ j * ( 𝜷 ^ ) | ℱ n ) = 1 r O P ( 1 ) O P ( 1 ) − O P ( r − 1 ) = O P ( r − 1 ) 𝑉 𝑎 𝑟 conditional superscript subscript ˙ ℓ 𝑗 ^ 𝜷 subscript ℱ 𝑛 1 𝑟 subscript 𝑂 𝑃 1 subscript 𝑂 𝑃 1 subscript 𝑂 𝑃 superscript 𝑟 1 subscript 𝑂 𝑃 superscript 𝑟 1 Var(\dot{\ell}_{j}^{*}(\hat{\boldsymbol{\beta}})|\mathcal{F}_{n})=\frac{1}{r}O%
_{P}(1)O_{P}(1)-O_{P}(r^{-1})=O_{P}(r^{-1}) italic_V italic_a italic_r ( over˙ start_ARG roman_ℓ end_ARG start_POSTSUBSCRIPT italic_j end_POSTSUBSCRIPT start_POSTSUPERSCRIPT * end_POSTSUPERSCRIPT ( over^ start_ARG bold_italic_β end_ARG ) | caligraphic_F start_POSTSUBSCRIPT italic_n end_POSTSUBSCRIPT ) = divide start_ARG 1 end_ARG start_ARG italic_r end_ARG italic_O start_POSTSUBSCRIPT italic_P end_POSTSUBSCRIPT ( 1 ) italic_O start_POSTSUBSCRIPT italic_P end_POSTSUBSCRIPT ( 1 ) - italic_O start_POSTSUBSCRIPT italic_P end_POSTSUBSCRIPT ( italic_r start_POSTSUPERSCRIPT - 1 end_POSTSUPERSCRIPT ) = italic_O start_POSTSUBSCRIPT italic_P end_POSTSUBSCRIPT ( italic_r start_POSTSUPERSCRIPT - 1 end_POSTSUPERSCRIPT ) .
From the Chebyshev inequality, for a sufficiently large M 𝑀 M italic_M , we have
P ( ‖ ℓ ˙ * ( 𝜷 ^ ) ‖ ≥ r − 1 / 2 M | ℱ n ) 𝑃 norm superscript ˙ ℓ ^ 𝜷 conditional superscript 𝑟 1 2 𝑀 subscript ℱ 𝑛 \displaystyle P(\|\dot{\ell}^{*}(\hat{\boldsymbol{\beta}})\|\geq r^{-1/2}M|%
\mathcal{F}_{n}) italic_P ( ∥ over˙ start_ARG roman_ℓ end_ARG start_POSTSUPERSCRIPT * end_POSTSUPERSCRIPT ( over^ start_ARG bold_italic_β end_ARG ) ∥ ≥ italic_r start_POSTSUPERSCRIPT - 1 / 2 end_POSTSUPERSCRIPT italic_M | caligraphic_F start_POSTSUBSCRIPT italic_n end_POSTSUBSCRIPT )
≤ r E ( ‖ ℓ ˙ * ( 𝜷 ^ ) ‖ 2 | ℱ n ) M 2 absent 𝑟 𝐸 conditional superscript norm superscript ˙ ℓ ^ 𝜷 2 subscript ℱ 𝑛 superscript 𝑀 2 \displaystyle\leq\frac{rE(\|\dot{\ell}^{*}(\hat{\boldsymbol{\beta}})\|^{2}|%
\mathcal{F}_{n})}{M^{2}} ≤ divide start_ARG italic_r italic_E ( ∥ over˙ start_ARG roman_ℓ end_ARG start_POSTSUPERSCRIPT * end_POSTSUPERSCRIPT ( over^ start_ARG bold_italic_β end_ARG ) ∥ start_POSTSUPERSCRIPT 2 end_POSTSUPERSCRIPT | caligraphic_F start_POSTSUBSCRIPT italic_n end_POSTSUBSCRIPT ) end_ARG start_ARG italic_M start_POSTSUPERSCRIPT 2 end_POSTSUPERSCRIPT end_ARG
= r ∑ j = 1 p E ( ℓ ˙ j * ( 𝜷 ^ ) 2 | ℱ n ) M 2 absent 𝑟 superscript subscript 𝑗 1 𝑝 𝐸 conditional superscript subscript ˙ ℓ 𝑗 superscript ^ 𝜷 2 subscript ℱ 𝑛 superscript 𝑀 2 \displaystyle=\frac{r\sum_{j=1}^{p}E(\dot{\ell}_{j}^{*}(\hat{\boldsymbol{\beta%
}})^{2}|\mathcal{F}_{n})}{M^{2}} = divide start_ARG italic_r ∑ start_POSTSUBSCRIPT italic_j = 1 end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_p end_POSTSUPERSCRIPT italic_E ( over˙ start_ARG roman_ℓ end_ARG start_POSTSUBSCRIPT italic_j end_POSTSUBSCRIPT start_POSTSUPERSCRIPT * end_POSTSUPERSCRIPT ( over^ start_ARG bold_italic_β end_ARG ) start_POSTSUPERSCRIPT 2 end_POSTSUPERSCRIPT | caligraphic_F start_POSTSUBSCRIPT italic_n end_POSTSUBSCRIPT ) end_ARG start_ARG italic_M start_POSTSUPERSCRIPT 2 end_POSTSUPERSCRIPT end_ARG
= O P ( 1 ) M 2 → 0 , n , r → ∞ . formulae-sequence absent subscript 𝑂 𝑃 1 superscript 𝑀 2 → 0 → 𝑛 𝑟
\displaystyle=\frac{O_{P}(1)}{M^{2}}\rightarrow 0,n,r\rightarrow\infty. = divide start_ARG italic_O start_POSTSUBSCRIPT italic_P end_POSTSUBSCRIPT ( 1 ) end_ARG start_ARG italic_M start_POSTSUPERSCRIPT 2 end_POSTSUPERSCRIPT end_ARG → 0 , italic_n , italic_r → ∞ .
Thus, the equation (S2.2 ) is derived.
In order to prove (S2.3 ), we directly calculate
E ( ℋ ~ W | ℱ n ) = ℋ W . 𝐸 conditional subscript ~ ℋ 𝑊 subscript ℱ 𝑛 subscript ℋ 𝑊 E(\tilde{\mathcal{H}}_{W}|\mathcal{F}_{n})=\mathcal{H}_{W}. italic_E ( over~ start_ARG caligraphic_H end_ARG start_POSTSUBSCRIPT italic_W end_POSTSUBSCRIPT | caligraphic_F start_POSTSUBSCRIPT italic_n end_POSTSUBSCRIPT ) = caligraphic_H start_POSTSUBSCRIPT italic_W end_POSTSUBSCRIPT .
For any element ℋ ~ W j 1 j 2 , 1 ≤ j 1 , j 2 ≤ p formulae-sequence superscript subscript ~ ℋ 𝑊 subscript 𝑗 1 subscript 𝑗 2 1
subscript 𝑗 1 subscript 𝑗 2 𝑝 \tilde{\mathcal{H}}_{W}^{j_{1}j_{2}},1\leq j_{1},j_{2}\leq p over~ start_ARG caligraphic_H end_ARG start_POSTSUBSCRIPT italic_W end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_j start_POSTSUBSCRIPT 1 end_POSTSUBSCRIPT italic_j start_POSTSUBSCRIPT 2 end_POSTSUBSCRIPT end_POSTSUPERSCRIPT , 1 ≤ italic_j start_POSTSUBSCRIPT 1 end_POSTSUBSCRIPT , italic_j start_POSTSUBSCRIPT 2 end_POSTSUBSCRIPT ≤ italic_p of ℋ ~ W subscript ~ ℋ 𝑊 \tilde{\mathcal{H}}_{W} over~ start_ARG caligraphic_H end_ARG start_POSTSUBSCRIPT italic_W end_POSTSUBSCRIPT , using Assumptions 2, 3 and Cauchy-Schwarz inequality, it can be concluded that
V a r ( ℋ ~ W j 1 j 2 | ℱ n ) = 𝑉 𝑎 𝑟 conditional superscript subscript ~ ℋ 𝑊 subscript 𝑗 1 subscript 𝑗 2 subscript ℱ 𝑛 absent \displaystyle Var(\tilde{\mathcal{H}}_{W}^{j_{1}j_{2}}|\mathcal{F}_{n})= italic_V italic_a italic_r ( over~ start_ARG caligraphic_H end_ARG start_POSTSUBSCRIPT italic_W end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_j start_POSTSUBSCRIPT 1 end_POSTSUBSCRIPT italic_j start_POSTSUBSCRIPT 2 end_POSTSUBSCRIPT end_POSTSUPERSCRIPT | caligraphic_F start_POSTSUBSCRIPT italic_n end_POSTSUBSCRIPT ) =
E { ℋ ~ W j 1 j 2 − ℋ W j 1 j 2 | ℱ n } 2 𝐸 superscript conditional-set superscript subscript ~ ℋ 𝑊 subscript 𝑗 1 subscript 𝑗 2 superscript subscript ℋ 𝑊 subscript 𝑗 1 subscript 𝑗 2 subscript ℱ 𝑛 2 \displaystyle E\{\tilde{\mathcal{H}}_{W}^{j_{1}j_{2}}-\mathcal{H}_{W}^{j_{1}j_%
{2}}|\mathcal{F}_{n}\}^{2} italic_E { over~ start_ARG caligraphic_H end_ARG start_POSTSUBSCRIPT italic_W end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_j start_POSTSUBSCRIPT 1 end_POSTSUBSCRIPT italic_j start_POSTSUBSCRIPT 2 end_POSTSUBSCRIPT end_POSTSUPERSCRIPT - caligraphic_H start_POSTSUBSCRIPT italic_W end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_j start_POSTSUBSCRIPT 1 end_POSTSUBSCRIPT italic_j start_POSTSUBSCRIPT 2 end_POSTSUBSCRIPT end_POSTSUPERSCRIPT | caligraphic_F start_POSTSUBSCRIPT italic_n end_POSTSUBSCRIPT } start_POSTSUPERSCRIPT 2 end_POSTSUPERSCRIPT
= \displaystyle= =
E { 1 n r ∑ i = 1 r 1 π i * w i j 1 * w i j 2 * − 1 n ∑ i = 1 n w i j 1 w i j 2 | ℱ n } 2 𝐸 superscript conditional-set 1 𝑛 𝑟 superscript subscript 𝑖 1 𝑟 1 superscript subscript 𝜋 𝑖 superscript subscript 𝑤 𝑖 subscript 𝑗 1 superscript subscript 𝑤 𝑖 subscript 𝑗 2 1 𝑛 superscript subscript 𝑖 1 𝑛 subscript 𝑤 𝑖 subscript 𝑗 1 subscript 𝑤 𝑖 subscript 𝑗 2 subscript ℱ 𝑛 2 \displaystyle E\left\{\frac{1}{nr}\sum_{i=1}^{r}\frac{1}{\pi_{i}^{*}}w_{ij_{1}%
}^{*}w_{ij_{2}}^{*}-\frac{1}{n}\sum_{i=1}^{n}w_{ij_{1}}w_{ij_{2}}\bigg{|}%
\mathcal{F}_{n}\right\}^{2} italic_E { divide start_ARG 1 end_ARG start_ARG italic_n italic_r end_ARG ∑ start_POSTSUBSCRIPT italic_i = 1 end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_r end_POSTSUPERSCRIPT divide start_ARG 1 end_ARG start_ARG italic_π start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT start_POSTSUPERSCRIPT * end_POSTSUPERSCRIPT end_ARG italic_w start_POSTSUBSCRIPT italic_i italic_j start_POSTSUBSCRIPT 1 end_POSTSUBSCRIPT end_POSTSUBSCRIPT start_POSTSUPERSCRIPT * end_POSTSUPERSCRIPT italic_w start_POSTSUBSCRIPT italic_i italic_j start_POSTSUBSCRIPT 2 end_POSTSUBSCRIPT end_POSTSUBSCRIPT start_POSTSUPERSCRIPT * end_POSTSUPERSCRIPT - divide start_ARG 1 end_ARG start_ARG italic_n end_ARG ∑ start_POSTSUBSCRIPT italic_i = 1 end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_n end_POSTSUPERSCRIPT italic_w start_POSTSUBSCRIPT italic_i italic_j start_POSTSUBSCRIPT 1 end_POSTSUBSCRIPT end_POSTSUBSCRIPT italic_w start_POSTSUBSCRIPT italic_i italic_j start_POSTSUBSCRIPT 2 end_POSTSUBSCRIPT end_POSTSUBSCRIPT | caligraphic_F start_POSTSUBSCRIPT italic_n end_POSTSUBSCRIPT } start_POSTSUPERSCRIPT 2 end_POSTSUPERSCRIPT
= \displaystyle= =
E ( 1 n r ∑ i = 1 r 1 π i * w i j 1 * w i j 2 * | ℱ n ) 2 − ( 1 n ∑ i = 1 n w i j 1 w i j 2 ) 2 𝐸 superscript conditional 1 𝑛 𝑟 superscript subscript 𝑖 1 𝑟 1 superscript subscript 𝜋 𝑖 superscript subscript 𝑤 𝑖 subscript 𝑗 1 superscript subscript 𝑤 𝑖 subscript 𝑗 2 subscript ℱ 𝑛 2 superscript 1 𝑛 superscript subscript 𝑖 1 𝑛 subscript 𝑤 𝑖 subscript 𝑗 1 subscript 𝑤 𝑖 subscript 𝑗 2 2 \displaystyle E\left(\frac{1}{nr}\sum_{i=1}^{r}\frac{1}{\pi_{i}^{*}}w_{ij_{1}}%
^{*}w_{ij_{2}}^{*}|\mathcal{F}_{n}\right)^{2}-\left(\frac{1}{n}\sum_{i=1}^{n}w%
_{ij_{1}}w_{ij_{2}}\right)^{2} italic_E ( divide start_ARG 1 end_ARG start_ARG italic_n italic_r end_ARG ∑ start_POSTSUBSCRIPT italic_i = 1 end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_r end_POSTSUPERSCRIPT divide start_ARG 1 end_ARG start_ARG italic_π start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT start_POSTSUPERSCRIPT * end_POSTSUPERSCRIPT end_ARG italic_w start_POSTSUBSCRIPT italic_i italic_j start_POSTSUBSCRIPT 1 end_POSTSUBSCRIPT end_POSTSUBSCRIPT start_POSTSUPERSCRIPT * end_POSTSUPERSCRIPT italic_w start_POSTSUBSCRIPT italic_i italic_j start_POSTSUBSCRIPT 2 end_POSTSUBSCRIPT end_POSTSUBSCRIPT start_POSTSUPERSCRIPT * end_POSTSUPERSCRIPT | caligraphic_F start_POSTSUBSCRIPT italic_n end_POSTSUBSCRIPT ) start_POSTSUPERSCRIPT 2 end_POSTSUPERSCRIPT - ( divide start_ARG 1 end_ARG start_ARG italic_n end_ARG ∑ start_POSTSUBSCRIPT italic_i = 1 end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_n end_POSTSUPERSCRIPT italic_w start_POSTSUBSCRIPT italic_i italic_j start_POSTSUBSCRIPT 1 end_POSTSUBSCRIPT end_POSTSUBSCRIPT italic_w start_POSTSUBSCRIPT italic_i italic_j start_POSTSUBSCRIPT 2 end_POSTSUBSCRIPT end_POSTSUBSCRIPT ) start_POSTSUPERSCRIPT 2 end_POSTSUPERSCRIPT
= \displaystyle= =
1 n 2 r ∑ i = 1 n π i ⋅ ( w i j 1 w i j 2 π i ) 2 − 1 n 2 ∑ i = 1 n ( w i j 1 w i j 2 ) 2 1 superscript 𝑛 2 𝑟 superscript subscript 𝑖 1 𝑛 ⋅ subscript 𝜋 𝑖 superscript subscript 𝑤 𝑖 subscript 𝑗 1 subscript 𝑤 𝑖 subscript 𝑗 2 subscript 𝜋 𝑖 2 1 superscript 𝑛 2 superscript subscript 𝑖 1 𝑛 superscript subscript 𝑤 𝑖 subscript 𝑗 1 subscript 𝑤 𝑖 subscript 𝑗 2 2 \displaystyle\frac{1}{n^{2}r}\sum_{i=1}^{n}\pi_{i}\cdot\left(\frac{w_{ij_{1}}w%
_{ij_{2}}}{\pi_{i}}\right)^{2}-\frac{1}{n^{2}}\sum_{i=1}^{n}(w_{ij_{1}}w_{ij_{%
2}})^{2} divide start_ARG 1 end_ARG start_ARG italic_n start_POSTSUPERSCRIPT 2 end_POSTSUPERSCRIPT italic_r end_ARG ∑ start_POSTSUBSCRIPT italic_i = 1 end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_n end_POSTSUPERSCRIPT italic_π start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT ⋅ ( divide start_ARG italic_w start_POSTSUBSCRIPT italic_i italic_j start_POSTSUBSCRIPT 1 end_POSTSUBSCRIPT end_POSTSUBSCRIPT italic_w start_POSTSUBSCRIPT italic_i italic_j start_POSTSUBSCRIPT 2 end_POSTSUBSCRIPT end_POSTSUBSCRIPT end_ARG start_ARG italic_π start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT end_ARG ) start_POSTSUPERSCRIPT 2 end_POSTSUPERSCRIPT - divide start_ARG 1 end_ARG start_ARG italic_n start_POSTSUPERSCRIPT 2 end_POSTSUPERSCRIPT end_ARG ∑ start_POSTSUBSCRIPT italic_i = 1 end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_n end_POSTSUPERSCRIPT ( italic_w start_POSTSUBSCRIPT italic_i italic_j start_POSTSUBSCRIPT 1 end_POSTSUBSCRIPT end_POSTSUBSCRIPT italic_w start_POSTSUBSCRIPT italic_i italic_j start_POSTSUBSCRIPT 2 end_POSTSUBSCRIPT end_POSTSUBSCRIPT ) start_POSTSUPERSCRIPT 2 end_POSTSUPERSCRIPT
= \displaystyle= =
1 n 2 r ∑ i = 1 n ( w i j 1 w i j 2 ) 2 π i − 1 n 2 ∑ i = 1 n ( w i j 1 w i j 2 ) 2 1 superscript 𝑛 2 𝑟 superscript subscript 𝑖 1 𝑛 superscript subscript 𝑤 𝑖 subscript 𝑗 1 subscript 𝑤 𝑖 subscript 𝑗 2 2 subscript 𝜋 𝑖 1 superscript 𝑛 2 superscript subscript 𝑖 1 𝑛 superscript subscript 𝑤 𝑖 subscript 𝑗 1 subscript 𝑤 𝑖 subscript 𝑗 2 2 \displaystyle\frac{1}{n^{2}r}\sum_{i=1}^{n}\frac{(w_{ij_{1}}w_{ij_{2}})^{2}}{%
\pi_{i}}-\frac{1}{n^{2}}\sum_{i=1}^{n}(w_{ij_{1}}w_{ij_{2}})^{2} divide start_ARG 1 end_ARG start_ARG italic_n start_POSTSUPERSCRIPT 2 end_POSTSUPERSCRIPT italic_r end_ARG ∑ start_POSTSUBSCRIPT italic_i = 1 end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_n end_POSTSUPERSCRIPT divide start_ARG ( italic_w start_POSTSUBSCRIPT italic_i italic_j start_POSTSUBSCRIPT 1 end_POSTSUBSCRIPT end_POSTSUBSCRIPT italic_w start_POSTSUBSCRIPT italic_i italic_j start_POSTSUBSCRIPT 2 end_POSTSUBSCRIPT end_POSTSUBSCRIPT ) start_POSTSUPERSCRIPT 2 end_POSTSUPERSCRIPT end_ARG start_ARG italic_π start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT end_ARG - divide start_ARG 1 end_ARG start_ARG italic_n start_POSTSUPERSCRIPT 2 end_POSTSUPERSCRIPT end_ARG ∑ start_POSTSUBSCRIPT italic_i = 1 end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_n end_POSTSUPERSCRIPT ( italic_w start_POSTSUBSCRIPT italic_i italic_j start_POSTSUBSCRIPT 1 end_POSTSUBSCRIPT end_POSTSUBSCRIPT italic_w start_POSTSUBSCRIPT italic_i italic_j start_POSTSUBSCRIPT 2 end_POSTSUBSCRIPT end_POSTSUBSCRIPT ) start_POSTSUPERSCRIPT 2 end_POSTSUPERSCRIPT
≤ \displaystyle\leq ≤
1 r max i = 1 , … , n ( n π i ) − 1 ∑ i = 1 n ‖ 𝐖 i ‖ 4 n \displaystyle\frac{1}{r}\max_{i=1,\ldots,n}(n\pi_{i})^{-1}\sum_{i=1}^{n}\frac{%
\|\mathbf{W}_{i}\|^{4}}{n} divide start_ARG 1 end_ARG start_ARG italic_r end_ARG roman_max start_POSTSUBSCRIPT italic_i = 1 , … , italic_n end_POSTSUBSCRIPT ( italic_n italic_π start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT ) start_POSTSUPERSCRIPT - 1 end_POSTSUPERSCRIPT ∑ start_POSTSUBSCRIPT italic_i = 1 end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_n end_POSTSUPERSCRIPT divide start_ARG ∥ bold_W start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT ∥ start_POSTSUPERSCRIPT 4 end_POSTSUPERSCRIPT end_ARG start_ARG italic_n end_ARG
= \displaystyle= =
1 r O P ( 1 ) O P ( 1 ) = O P ( r − 1 ) . 1 𝑟 subscript 𝑂 𝑃 1 subscript 𝑂 𝑃 1 subscript 𝑂 𝑃 superscript 𝑟 1 \displaystyle\frac{1}{r}O_{P}(1)O_{P}(1)=O_{P}(r^{-1}). divide start_ARG 1 end_ARG start_ARG italic_r end_ARG italic_O start_POSTSUBSCRIPT italic_P end_POSTSUBSCRIPT ( 1 ) italic_O start_POSTSUBSCRIPT italic_P end_POSTSUBSCRIPT ( 1 ) = italic_O start_POSTSUBSCRIPT italic_P end_POSTSUBSCRIPT ( italic_r start_POSTSUPERSCRIPT - 1 end_POSTSUPERSCRIPT ) .
Using the Chebyshev inequality, for sufficiently large M 𝑀 M italic_M , we have
P ( ‖ ℋ ~ W − ℋ W ‖ ≥ r − 1 / 2 M | ℱ n ) 𝑃 norm subscript ~ ℋ 𝑊 subscript ℋ 𝑊 conditional superscript 𝑟 1 2 𝑀 subscript ℱ 𝑛 \displaystyle P(\|\tilde{\mathcal{H}}_{W}-\mathcal{H}_{W}\|\geq r^{-1/2}M|%
\mathcal{F}_{n}) italic_P ( ∥ over~ start_ARG caligraphic_H end_ARG start_POSTSUBSCRIPT italic_W end_POSTSUBSCRIPT - caligraphic_H start_POSTSUBSCRIPT italic_W end_POSTSUBSCRIPT ∥ ≥ italic_r start_POSTSUPERSCRIPT - 1 / 2 end_POSTSUPERSCRIPT italic_M | caligraphic_F start_POSTSUBSCRIPT italic_n end_POSTSUBSCRIPT )
≤ r E ( ‖ ℋ ~ W ‖ 2 | ℱ n ) M 2 absent 𝑟 𝐸 conditional superscript norm subscript ~ ℋ 𝑊 2 subscript ℱ 𝑛 superscript 𝑀 2 \displaystyle\leq\frac{rE(\|\tilde{\mathcal{H}}_{W}\|^{2}|\mathcal{F}_{n})}{M^%
{2}} ≤ divide start_ARG italic_r italic_E ( ∥ over~ start_ARG caligraphic_H end_ARG start_POSTSUBSCRIPT italic_W end_POSTSUBSCRIPT ∥ start_POSTSUPERSCRIPT 2 end_POSTSUPERSCRIPT | caligraphic_F start_POSTSUBSCRIPT italic_n end_POSTSUBSCRIPT ) end_ARG start_ARG italic_M start_POSTSUPERSCRIPT 2 end_POSTSUPERSCRIPT end_ARG
= r ∑ j 1 = 1 p ∑ j 2 = 1 p E ( ℋ ~ W j 1 j 2 | ℱ n ) 2 M 2 absent 𝑟 superscript subscript subscript 𝑗 1 1 𝑝 superscript subscript subscript 𝑗 2 1 𝑝 𝐸 superscript conditional superscript subscript ~ ℋ 𝑊 subscript 𝑗 1 subscript 𝑗 2 subscript ℱ 𝑛 2 superscript 𝑀 2 \displaystyle=\frac{r\sum_{j_{1}=1}^{p}\sum_{j_{2}=1}^{p}E(\tilde{\mathcal{H}}%
_{W}^{j_{1}j_{2}}|\mathcal{F}_{n})^{2}}{M^{2}} = divide start_ARG italic_r ∑ start_POSTSUBSCRIPT italic_j start_POSTSUBSCRIPT 1 end_POSTSUBSCRIPT = 1 end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_p end_POSTSUPERSCRIPT ∑ start_POSTSUBSCRIPT italic_j start_POSTSUBSCRIPT 2 end_POSTSUBSCRIPT = 1 end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_p end_POSTSUPERSCRIPT italic_E ( over~ start_ARG caligraphic_H end_ARG start_POSTSUBSCRIPT italic_W end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_j start_POSTSUBSCRIPT 1 end_POSTSUBSCRIPT italic_j start_POSTSUBSCRIPT 2 end_POSTSUBSCRIPT end_POSTSUPERSCRIPT | caligraphic_F start_POSTSUBSCRIPT italic_n end_POSTSUBSCRIPT ) start_POSTSUPERSCRIPT 2 end_POSTSUPERSCRIPT end_ARG start_ARG italic_M start_POSTSUPERSCRIPT 2 end_POSTSUPERSCRIPT end_ARG
= O P ( 1 ) M 2 → 0 , n , r → ∞ . formulae-sequence absent subscript 𝑂 𝑃 1 superscript 𝑀 2 → 0 → 𝑛 𝑟
\displaystyle=\frac{O_{P}(1)}{M^{2}}\rightarrow 0,n,r\rightarrow\infty. = divide start_ARG italic_O start_POSTSUBSCRIPT italic_P end_POSTSUBSCRIPT ( 1 ) end_ARG start_ARG italic_M start_POSTSUPERSCRIPT 2 end_POSTSUPERSCRIPT end_ARG → 0 , italic_n , italic_r → ∞ .
Thus, the equation (S2.3 ) is proved.
By (S2.3 ) and Assumption 1, we can obtain ℋ ~ W − 1 = O P | ℱ n ( 1 ) superscript subscript ~ ℋ 𝑊 1 subscript 𝑂 conditional 𝑃 subscript ℱ 𝑛 1 \tilde{\mathcal{H}}_{W}^{-1}=O_{P|\mathcal{F}_{n}}(1) over~ start_ARG caligraphic_H end_ARG start_POSTSUBSCRIPT italic_W end_POSTSUBSCRIPT start_POSTSUPERSCRIPT - 1 end_POSTSUPERSCRIPT = italic_O start_POSTSUBSCRIPT italic_P | caligraphic_F start_POSTSUBSCRIPT italic_n end_POSTSUBSCRIPT end_POSTSUBSCRIPT ( 1 ) . Therefore, combining (S2.1 ), (S2.2 ) and (S2.3 ), we have
𝜷 ~ − 𝜷 ^ = O P | ℱ n ( r − 1 / 2 ) . ~ 𝜷 ^ 𝜷 subscript 𝑂 conditional 𝑃 subscript ℱ 𝑛 superscript 𝑟 1 2 \tilde{\boldsymbol{\beta}}-\hat{\boldsymbol{\beta}}=O_{P|\mathcal{F}_{n}}(r^{-%
1/2}). over~ start_ARG bold_italic_β end_ARG - over^ start_ARG bold_italic_β end_ARG = italic_O start_POSTSUBSCRIPT italic_P | caligraphic_F start_POSTSUBSCRIPT italic_n end_POSTSUBSCRIPT end_POSTSUBSCRIPT ( italic_r start_POSTSUPERSCRIPT - 1 / 2 end_POSTSUPERSCRIPT ) .
Then the theorem is proved.
S3 Proof of Theorem 2
Note that
ℓ ˙ * ( 𝜷 ^ ) = 1 r ∑ i = 1 r { 1 n π i * [ − 𝐖 i * ( y i * − 𝐖 i * T 𝜷 ^ ) ] − 𝚺 u u 𝜷 ^ } = 1 r ∑ i = 1 r 𝝃 i , superscript ˙ ℓ ^ 𝜷 1 𝑟 superscript subscript 𝑖 1 𝑟 1 𝑛 superscript subscript 𝜋 𝑖 delimited-[] superscript subscript 𝐖 𝑖 superscript subscript 𝑦 𝑖 superscript subscript 𝐖 𝑖 absent 𝑇 ^ 𝜷 subscript 𝚺 𝑢 𝑢 ^ 𝜷 1 𝑟 superscript subscript 𝑖 1 𝑟 subscript 𝝃 𝑖 \dot{\ell}^{*}(\hat{\boldsymbol{\beta}})=\frac{1}{r}\sum_{i=1}^{r}\left\{\frac%
{1}{n\pi_{i}^{*}}[-\mathbf{W}_{i}^{*}(y_{i}^{*}-\mathbf{W}_{i}^{*T}\hat{%
\boldsymbol{\beta}})]-\boldsymbol{\Sigma}_{uu}\hat{\boldsymbol{\beta}}\right\}%
=\frac{1}{r}\sum_{i=1}^{r}\boldsymbol{\xi}_{i}, over˙ start_ARG roman_ℓ end_ARG start_POSTSUPERSCRIPT * end_POSTSUPERSCRIPT ( over^ start_ARG bold_italic_β end_ARG ) = divide start_ARG 1 end_ARG start_ARG italic_r end_ARG ∑ start_POSTSUBSCRIPT italic_i = 1 end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_r end_POSTSUPERSCRIPT { divide start_ARG 1 end_ARG start_ARG italic_n italic_π start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT start_POSTSUPERSCRIPT * end_POSTSUPERSCRIPT end_ARG [ - bold_W start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT start_POSTSUPERSCRIPT * end_POSTSUPERSCRIPT ( italic_y start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT start_POSTSUPERSCRIPT * end_POSTSUPERSCRIPT - bold_W start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT start_POSTSUPERSCRIPT * italic_T end_POSTSUPERSCRIPT over^ start_ARG bold_italic_β end_ARG ) ] - bold_Σ start_POSTSUBSCRIPT italic_u italic_u end_POSTSUBSCRIPT over^ start_ARG bold_italic_β end_ARG } = divide start_ARG 1 end_ARG start_ARG italic_r end_ARG ∑ start_POSTSUBSCRIPT italic_i = 1 end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_r end_POSTSUPERSCRIPT bold_italic_ξ start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT ,
(S3.1)
where 𝝃 i = 1 n π i * [ − 𝐖 i * ( y i * − 𝐖 i * T 𝜷 ^ ) ] − 𝚺 u u 𝜷 ^ subscript 𝝃 𝑖 1 𝑛 superscript subscript 𝜋 𝑖 delimited-[] superscript subscript 𝐖 𝑖 superscript subscript 𝑦 𝑖 superscript subscript 𝐖 𝑖 absent 𝑇 ^ 𝜷 subscript 𝚺 𝑢 𝑢 ^ 𝜷 \boldsymbol{\xi}_{i}=\frac{1}{n\pi_{i}^{*}}[-\mathbf{W}_{i}^{*}(y_{i}^{*}-%
\mathbf{W}_{i}^{*T}\hat{\boldsymbol{\beta}})]-\boldsymbol{\Sigma}_{uu}\hat{%
\boldsymbol{\beta}} bold_italic_ξ start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT = divide start_ARG 1 end_ARG start_ARG italic_n italic_π start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT start_POSTSUPERSCRIPT * end_POSTSUPERSCRIPT end_ARG [ - bold_W start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT start_POSTSUPERSCRIPT * end_POSTSUPERSCRIPT ( italic_y start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT start_POSTSUPERSCRIPT * end_POSTSUPERSCRIPT - bold_W start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT start_POSTSUPERSCRIPT * italic_T end_POSTSUPERSCRIPT over^ start_ARG bold_italic_β end_ARG ) ] - bold_Σ start_POSTSUBSCRIPT italic_u italic_u end_POSTSUBSCRIPT over^ start_ARG bold_italic_β end_ARG is an independent random vector.
Then it can be directly calculated to obtain
E ( 𝝃 i | ℱ n ) 𝐸 conditional subscript 𝝃 𝑖 subscript ℱ 𝑛 \displaystyle E(\boldsymbol{\xi}_{i}|\mathcal{F}_{n}) italic_E ( bold_italic_ξ start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT | caligraphic_F start_POSTSUBSCRIPT italic_n end_POSTSUBSCRIPT )
= 1 n ∑ i = 1 n [ − 𝐖 i ( y i − 𝐖 i T 𝜷 ^ ) ] − 𝚺 u u 𝜷 ^ = 𝟎 , absent 1 𝑛 superscript subscript 𝑖 1 𝑛 delimited-[] subscript 𝐖 𝑖 subscript 𝑦 𝑖 superscript subscript 𝐖 𝑖 𝑇 ^ 𝜷 subscript 𝚺 𝑢 𝑢 ^ 𝜷 0 \displaystyle=\frac{1}{n}\sum_{i=1}^{n}[-\mathbf{W}_{i}(y_{i}-\mathbf{W}_{i}^{%
T}\hat{\boldsymbol{\beta}})]-\boldsymbol{\Sigma}_{uu}\hat{\boldsymbol{\beta}}=%
\mathbf{0}, = divide start_ARG 1 end_ARG start_ARG italic_n end_ARG ∑ start_POSTSUBSCRIPT italic_i = 1 end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_n end_POSTSUPERSCRIPT [ - bold_W start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT ( italic_y start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT - bold_W start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_T end_POSTSUPERSCRIPT over^ start_ARG bold_italic_β end_ARG ) ] - bold_Σ start_POSTSUBSCRIPT italic_u italic_u end_POSTSUBSCRIPT over^ start_ARG bold_italic_β end_ARG = bold_0 ,
(S3.2)
V a r ( 𝝃 i | ℱ n ) 𝑉 𝑎 𝑟 conditional subscript 𝝃 𝑖 subscript ℱ 𝑛 \displaystyle Var(\boldsymbol{\xi}_{i}|\mathcal{F}_{n}) italic_V italic_a italic_r ( bold_italic_ξ start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT | caligraphic_F start_POSTSUBSCRIPT italic_n end_POSTSUBSCRIPT )
= 1 n 2 ∑ i = 1 n 𝐖 i 𝐖 i T ( y i − 𝐖 i T 𝜷 ^ ) 2 π i − ( 𝚺 u u 𝜷 ^ ) ⊗ 2 = r V c . absent 1 superscript 𝑛 2 superscript subscript 𝑖 1 𝑛 subscript 𝐖 𝑖 superscript subscript 𝐖 𝑖 𝑇 superscript subscript 𝑦 𝑖 superscript subscript 𝐖 𝑖 𝑇 ^ 𝜷 2 subscript 𝜋 𝑖 superscript subscript 𝚺 𝑢 𝑢 ^ 𝜷 tensor-product absent 2 𝑟 subscript 𝑉 𝑐 \displaystyle=\frac{1}{n^{2}}\sum_{i=1}^{n}\frac{\mathbf{W}_{i}\mathbf{W}_{i}^%
{T}(y_{i}-\mathbf{W}_{i}^{T}\hat{\boldsymbol{\beta}})^{2}}{\pi_{i}}-(%
\boldsymbol{\Sigma}_{uu}\hat{\boldsymbol{\beta}})^{\otimes 2}=rV_{c}. = divide start_ARG 1 end_ARG start_ARG italic_n start_POSTSUPERSCRIPT 2 end_POSTSUPERSCRIPT end_ARG ∑ start_POSTSUBSCRIPT italic_i = 1 end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_n end_POSTSUPERSCRIPT divide start_ARG bold_W start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT bold_W start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_T end_POSTSUPERSCRIPT ( italic_y start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT - bold_W start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_T end_POSTSUPERSCRIPT over^ start_ARG bold_italic_β end_ARG ) start_POSTSUPERSCRIPT 2 end_POSTSUPERSCRIPT end_ARG start_ARG italic_π start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT end_ARG - ( bold_Σ start_POSTSUBSCRIPT italic_u italic_u end_POSTSUBSCRIPT over^ start_ARG bold_italic_β end_ARG ) start_POSTSUPERSCRIPT ⊗ 2 end_POSTSUPERSCRIPT = italic_r italic_V start_POSTSUBSCRIPT italic_c end_POSTSUBSCRIPT .
According to the C r subscript 𝐶 𝑟 C_{r} italic_C start_POSTSUBSCRIPT italic_r end_POSTSUBSCRIPT inequality, Assumptions 3 and 4, we have
∑ i = 1 r E { ‖ r − 1 / 2 𝝃 i ‖ 2 I ( ‖ r − 1 / 2 𝝃 i ‖ > ε ) | ℱ n } superscript subscript 𝑖 1 𝑟 𝐸 conditional superscript norm superscript 𝑟 1 2 subscript 𝝃 𝑖 2 𝐼 norm superscript 𝑟 1 2 subscript 𝝃 𝑖 𝜀 subscript ℱ 𝑛 \displaystyle\sum_{i=1}^{r}E\{\|r^{-1/2}\boldsymbol{\xi}_{i}\|^{2}I(\|r^{-1/2}%
\boldsymbol{\xi}_{i}\|>\varepsilon)|\mathcal{F}_{n}\} ∑ start_POSTSUBSCRIPT italic_i = 1 end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_r end_POSTSUPERSCRIPT italic_E { ∥ italic_r start_POSTSUPERSCRIPT - 1 / 2 end_POSTSUPERSCRIPT bold_italic_ξ start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT ∥ start_POSTSUPERSCRIPT 2 end_POSTSUPERSCRIPT italic_I ( ∥ italic_r start_POSTSUPERSCRIPT - 1 / 2 end_POSTSUPERSCRIPT bold_italic_ξ start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT ∥ > italic_ε ) | caligraphic_F start_POSTSUBSCRIPT italic_n end_POSTSUBSCRIPT }
≤ \displaystyle\leq ≤
1 ε δ r 1 + δ / 2 ∑ i = 1 r E { ‖ 𝝃 i ‖ 2 + δ | ℱ n } 1 superscript 𝜀 𝛿 superscript 𝑟 1 𝛿 2 superscript subscript 𝑖 1 𝑟 𝐸 conditional superscript norm subscript 𝝃 𝑖 2 𝛿 subscript ℱ 𝑛 \displaystyle\frac{1}{\varepsilon^{\delta}r^{1+\delta/2}}\sum_{i=1}^{r}E\{\|%
\boldsymbol{\xi}_{i}\|^{2+\delta}|\mathcal{F}_{n}\} divide start_ARG 1 end_ARG start_ARG italic_ε start_POSTSUPERSCRIPT italic_δ end_POSTSUPERSCRIPT italic_r start_POSTSUPERSCRIPT 1 + italic_δ / 2 end_POSTSUPERSCRIPT end_ARG ∑ start_POSTSUBSCRIPT italic_i = 1 end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_r end_POSTSUPERSCRIPT italic_E { ∥ bold_italic_ξ start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT ∥ start_POSTSUPERSCRIPT 2 + italic_δ end_POSTSUPERSCRIPT | caligraphic_F start_POSTSUBSCRIPT italic_n end_POSTSUBSCRIPT }
= \displaystyle= =
1 ε δ r 1 + δ / 2 ∑ i = 1 r E { ‖ 1 n π i * [ − 𝐖 i * ( y i * − 𝐖 i * T 𝜷 ^ ) ] − 𝚺 u u 𝜷 ^ ‖ 2 + δ | ℱ n } 1 superscript 𝜀 𝛿 superscript 𝑟 1 𝛿 2 superscript subscript 𝑖 1 𝑟 𝐸 conditional superscript norm 1 𝑛 superscript subscript 𝜋 𝑖 delimited-[] superscript subscript 𝐖 𝑖 superscript subscript 𝑦 𝑖 superscript subscript 𝐖 𝑖 absent 𝑇 ^ 𝜷 subscript 𝚺 𝑢 𝑢 ^ 𝜷 2 𝛿 subscript ℱ 𝑛 \displaystyle\frac{1}{\varepsilon^{\delta}r^{1+\delta/2}}\sum_{i=1}^{r}E\left%
\{\left\|\frac{1}{n\pi_{i}^{*}}[-\mathbf{W}_{i}^{*}(y_{i}^{*}-\mathbf{W}_{i}^{%
*T}\hat{\boldsymbol{\beta}})]-\boldsymbol{\Sigma}_{uu}\hat{\boldsymbol{\beta}}%
\right\|^{2+\delta}\bigg{|}\mathcal{F}_{n}\right\} divide start_ARG 1 end_ARG start_ARG italic_ε start_POSTSUPERSCRIPT italic_δ end_POSTSUPERSCRIPT italic_r start_POSTSUPERSCRIPT 1 + italic_δ / 2 end_POSTSUPERSCRIPT end_ARG ∑ start_POSTSUBSCRIPT italic_i = 1 end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_r end_POSTSUPERSCRIPT italic_E { ∥ divide start_ARG 1 end_ARG start_ARG italic_n italic_π start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT start_POSTSUPERSCRIPT * end_POSTSUPERSCRIPT end_ARG [ - bold_W start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT start_POSTSUPERSCRIPT * end_POSTSUPERSCRIPT ( italic_y start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT start_POSTSUPERSCRIPT * end_POSTSUPERSCRIPT - bold_W start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT start_POSTSUPERSCRIPT * italic_T end_POSTSUPERSCRIPT over^ start_ARG bold_italic_β end_ARG ) ] - bold_Σ start_POSTSUBSCRIPT italic_u italic_u end_POSTSUBSCRIPT over^ start_ARG bold_italic_β end_ARG ∥ start_POSTSUPERSCRIPT 2 + italic_δ end_POSTSUPERSCRIPT | caligraphic_F start_POSTSUBSCRIPT italic_n end_POSTSUBSCRIPT }
≤ \displaystyle\leq ≤
2 1 + δ ε δ r 1 + δ / 2 ∑ i = 1 r { E [ ‖ 1 n π i * 𝐖 i * ( y i * − 𝐖 i * T 𝜷 ^ ) ‖ 2 + δ | ℱ n ] + E [ ‖ 𝚺 u u 𝜷 ^ ‖ 2 + δ | ℱ n ] } superscript 2 1 𝛿 superscript 𝜀 𝛿 superscript 𝑟 1 𝛿 2 superscript subscript 𝑖 1 𝑟 𝐸 delimited-[] conditional superscript norm 1 𝑛 superscript subscript 𝜋 𝑖 superscript subscript 𝐖 𝑖 superscript subscript 𝑦 𝑖 superscript subscript 𝐖 𝑖 absent 𝑇 ^ 𝜷 2 𝛿 subscript ℱ 𝑛 𝐸 delimited-[] conditional superscript norm subscript 𝚺 𝑢 𝑢 ^ 𝜷 2 𝛿 subscript ℱ 𝑛 \displaystyle\frac{2^{1+\delta}}{\varepsilon^{\delta}r^{1+\delta/2}}\sum_{i=1}%
^{r}\left\{E\left[\left\|\frac{1}{n\pi_{i}^{*}}\mathbf{W}_{i}^{*}(y_{i}^{*}-%
\mathbf{W}_{i}^{*T}\hat{\boldsymbol{\beta}})\right\|^{2+\delta}|\mathcal{F}_{n%
}\right]+E[\|\boldsymbol{\Sigma}_{uu}\hat{\boldsymbol{\beta}}\|^{2+\delta}|%
\mathcal{F}_{n}]\right\} divide start_ARG 2 start_POSTSUPERSCRIPT 1 + italic_δ end_POSTSUPERSCRIPT end_ARG start_ARG italic_ε start_POSTSUPERSCRIPT italic_δ end_POSTSUPERSCRIPT italic_r start_POSTSUPERSCRIPT 1 + italic_δ / 2 end_POSTSUPERSCRIPT end_ARG ∑ start_POSTSUBSCRIPT italic_i = 1 end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_r end_POSTSUPERSCRIPT { italic_E [ ∥ divide start_ARG 1 end_ARG start_ARG italic_n italic_π start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT start_POSTSUPERSCRIPT * end_POSTSUPERSCRIPT end_ARG bold_W start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT start_POSTSUPERSCRIPT * end_POSTSUPERSCRIPT ( italic_y start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT start_POSTSUPERSCRIPT * end_POSTSUPERSCRIPT - bold_W start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT start_POSTSUPERSCRIPT * italic_T end_POSTSUPERSCRIPT over^ start_ARG bold_italic_β end_ARG ) ∥ start_POSTSUPERSCRIPT 2 + italic_δ end_POSTSUPERSCRIPT | caligraphic_F start_POSTSUBSCRIPT italic_n end_POSTSUBSCRIPT ] + italic_E [ ∥ bold_Σ start_POSTSUBSCRIPT italic_u italic_u end_POSTSUBSCRIPT over^ start_ARG bold_italic_β end_ARG ∥ start_POSTSUPERSCRIPT 2 + italic_δ end_POSTSUPERSCRIPT | caligraphic_F start_POSTSUBSCRIPT italic_n end_POSTSUBSCRIPT ] }
= \displaystyle= =
2 1 + δ ε δ r δ / 2 { ∑ i = 1 n ‖ 𝐖 i ‖ 2 + δ ( y i − 𝐖 i T 𝜷 ^ ) 2 + δ n 2 + δ π i 1 + δ + ‖ 𝚺 u u 𝜷 ^ ‖ 2 + δ } superscript 2 1 𝛿 superscript 𝜀 𝛿 superscript 𝑟 𝛿 2 superscript subscript 𝑖 1 𝑛 superscript norm subscript 𝐖 𝑖 2 𝛿 superscript subscript 𝑦 𝑖 superscript subscript 𝐖 𝑖 𝑇 ^ 𝜷 2 𝛿 superscript 𝑛 2 𝛿 superscript subscript 𝜋 𝑖 1 𝛿 superscript norm subscript 𝚺 𝑢 𝑢 ^ 𝜷 2 𝛿 \displaystyle\frac{2^{1+\delta}}{\varepsilon^{\delta}r^{\delta/2}}\left\{\sum_%
{i=1}^{n}\frac{\|\mathbf{W}_{i}\|^{2+\delta}(y_{i}-\mathbf{W}_{i}^{T}\hat{%
\boldsymbol{\beta}})^{2+\delta}}{n^{2+\delta}\pi_{i}^{1+\delta}}+\|\boldsymbol%
{\Sigma}_{uu}\hat{\boldsymbol{\beta}}\|^{2+\delta}\right\} divide start_ARG 2 start_POSTSUPERSCRIPT 1 + italic_δ end_POSTSUPERSCRIPT end_ARG start_ARG italic_ε start_POSTSUPERSCRIPT italic_δ end_POSTSUPERSCRIPT italic_r start_POSTSUPERSCRIPT italic_δ / 2 end_POSTSUPERSCRIPT end_ARG { ∑ start_POSTSUBSCRIPT italic_i = 1 end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_n end_POSTSUPERSCRIPT divide start_ARG ∥ bold_W start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT ∥ start_POSTSUPERSCRIPT 2 + italic_δ end_POSTSUPERSCRIPT ( italic_y start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT - bold_W start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_T end_POSTSUPERSCRIPT over^ start_ARG bold_italic_β end_ARG ) start_POSTSUPERSCRIPT 2 + italic_δ end_POSTSUPERSCRIPT end_ARG start_ARG italic_n start_POSTSUPERSCRIPT 2 + italic_δ end_POSTSUPERSCRIPT italic_π start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT start_POSTSUPERSCRIPT 1 + italic_δ end_POSTSUPERSCRIPT end_ARG + ∥ bold_Σ start_POSTSUBSCRIPT italic_u italic_u end_POSTSUBSCRIPT over^ start_ARG bold_italic_β end_ARG ∥ start_POSTSUPERSCRIPT 2 + italic_δ end_POSTSUPERSCRIPT }
≤ \displaystyle\leq ≤
2 1 + δ ε δ r δ / 2 { max i = 1 , … , n ( n π i ) − 1 − δ ∑ i = 1 n ( y i − 𝐖 i T 𝜷 ^ ) 2 + δ ‖ 𝐖 i ‖ 2 + δ n + ∥ 𝚺 u u 𝜷 ^ ∥ 2 + δ } \displaystyle\frac{2^{1+\delta}}{\varepsilon^{\delta}r^{\delta/2}}\left\{\max_%
{i=1,\ldots,n}(n\pi_{i})^{-1-\delta}\sum_{i=1}^{n}\frac{(y_{i}-\mathbf{W}_{i}^%
{T}\hat{\boldsymbol{\beta}})^{2+\delta}\|\mathbf{W}_{i}\|^{2+\delta}}{n}+\|%
\boldsymbol{\Sigma}_{uu}\hat{\boldsymbol{\beta}}\|^{2+\delta}\right\} divide start_ARG 2 start_POSTSUPERSCRIPT 1 + italic_δ end_POSTSUPERSCRIPT end_ARG start_ARG italic_ε start_POSTSUPERSCRIPT italic_δ end_POSTSUPERSCRIPT italic_r start_POSTSUPERSCRIPT italic_δ / 2 end_POSTSUPERSCRIPT end_ARG { roman_max start_POSTSUBSCRIPT italic_i = 1 , … , italic_n end_POSTSUBSCRIPT ( italic_n italic_π start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT ) start_POSTSUPERSCRIPT - 1 - italic_δ end_POSTSUPERSCRIPT ∑ start_POSTSUBSCRIPT italic_i = 1 end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_n end_POSTSUPERSCRIPT divide start_ARG ( italic_y start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT - bold_W start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_T end_POSTSUPERSCRIPT over^ start_ARG bold_italic_β end_ARG ) start_POSTSUPERSCRIPT 2 + italic_δ end_POSTSUPERSCRIPT ∥ bold_W start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT ∥ start_POSTSUPERSCRIPT 2 + italic_δ end_POSTSUPERSCRIPT end_ARG start_ARG italic_n end_ARG + ∥ bold_Σ start_POSTSUBSCRIPT italic_u italic_u end_POSTSUBSCRIPT over^ start_ARG bold_italic_β end_ARG ∥ start_POSTSUPERSCRIPT 2 + italic_δ end_POSTSUPERSCRIPT }
= \displaystyle= =
O P ( r − δ / 2 ) . subscript 𝑂 𝑃 superscript 𝑟 𝛿 2 \displaystyle O_{P}(r^{-\delta/2}). italic_O start_POSTSUBSCRIPT italic_P end_POSTSUBSCRIPT ( italic_r start_POSTSUPERSCRIPT - italic_δ / 2 end_POSTSUPERSCRIPT ) .
In the light of the Lindeberg-Feller central limit theorem, it follows that
( ∑ i = 1 r V a r ( 𝝃 i | ℱ n ) ) − 1 / 2 ∑ i = 1 r 𝝃 i = V c − 1 / 2 ℓ ˙ * ( 𝜷 ^ ) → 𝑑 N p ( 𝟎 , I ) . superscript superscript subscript 𝑖 1 𝑟 𝑉 𝑎 𝑟 conditional subscript 𝝃 𝑖 subscript ℱ 𝑛 1 2 superscript subscript 𝑖 1 𝑟 subscript 𝝃 𝑖 superscript subscript 𝑉 𝑐 1 2 superscript ˙ ℓ ^ 𝜷 𝑑 → subscript 𝑁 𝑝 0 𝐼 \left(\sum_{i=1}^{r}Var(\boldsymbol{\xi}_{i}|\mathcal{F}_{n})\right)^{-1/2}%
\sum_{i=1}^{r}\boldsymbol{\xi}_{i}=V_{c}^{-1/2}\dot{\ell}^{*}(\hat{\boldsymbol%
{\beta}})\xrightarrow{d}N_{p}(\mathbf{0},I). ( ∑ start_POSTSUBSCRIPT italic_i = 1 end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_r end_POSTSUPERSCRIPT italic_V italic_a italic_r ( bold_italic_ξ start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT | caligraphic_F start_POSTSUBSCRIPT italic_n end_POSTSUBSCRIPT ) ) start_POSTSUPERSCRIPT - 1 / 2 end_POSTSUPERSCRIPT ∑ start_POSTSUBSCRIPT italic_i = 1 end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_r end_POSTSUPERSCRIPT bold_italic_ξ start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT = italic_V start_POSTSUBSCRIPT italic_c end_POSTSUBSCRIPT start_POSTSUPERSCRIPT - 1 / 2 end_POSTSUPERSCRIPT over˙ start_ARG roman_ℓ end_ARG start_POSTSUPERSCRIPT * end_POSTSUPERSCRIPT ( over^ start_ARG bold_italic_β end_ARG ) start_ARROW overitalic_d → end_ARROW italic_N start_POSTSUBSCRIPT italic_p end_POSTSUBSCRIPT ( bold_0 , italic_I ) .
(S3.3)
By (S2.3 ), we can obtain
ℋ ~ W − 1 − ℋ W − 1 = − ℋ W − 1 ( ℋ ~ W − ℋ W ) ℋ ~ W − 1 = O P | ℱ n ( r − 1 / 2 ) . superscript subscript ~ ℋ 𝑊 1 superscript subscript ℋ 𝑊 1 superscript subscript ℋ 𝑊 1 subscript ~ ℋ 𝑊 subscript ℋ 𝑊 superscript subscript ~ ℋ 𝑊 1 subscript 𝑂 conditional 𝑃 subscript ℱ 𝑛 superscript 𝑟 1 2 \tilde{\mathcal{H}}_{W}^{-1}-\mathcal{H}_{W}^{-1}=-\mathcal{H}_{W}^{-1}(\tilde%
{\mathcal{H}}_{W}-\mathcal{H}_{W})\tilde{\mathcal{H}}_{W}^{-1}=O_{P|\mathcal{F%
}_{n}}(r^{-1/2}). over~ start_ARG caligraphic_H end_ARG start_POSTSUBSCRIPT italic_W end_POSTSUBSCRIPT start_POSTSUPERSCRIPT - 1 end_POSTSUPERSCRIPT - caligraphic_H start_POSTSUBSCRIPT italic_W end_POSTSUBSCRIPT start_POSTSUPERSCRIPT - 1 end_POSTSUPERSCRIPT = - caligraphic_H start_POSTSUBSCRIPT italic_W end_POSTSUBSCRIPT start_POSTSUPERSCRIPT - 1 end_POSTSUPERSCRIPT ( over~ start_ARG caligraphic_H end_ARG start_POSTSUBSCRIPT italic_W end_POSTSUBSCRIPT - caligraphic_H start_POSTSUBSCRIPT italic_W end_POSTSUBSCRIPT ) over~ start_ARG caligraphic_H end_ARG start_POSTSUBSCRIPT italic_W end_POSTSUBSCRIPT start_POSTSUPERSCRIPT - 1 end_POSTSUPERSCRIPT = italic_O start_POSTSUBSCRIPT italic_P | caligraphic_F start_POSTSUBSCRIPT italic_n end_POSTSUBSCRIPT end_POSTSUBSCRIPT ( italic_r start_POSTSUPERSCRIPT - 1 / 2 end_POSTSUPERSCRIPT ) .
(S3.4)
By Assumption 1, ℋ W subscript ℋ 𝑊 \mathcal{H}_{W} caligraphic_H start_POSTSUBSCRIPT italic_W end_POSTSUBSCRIPT converges to a positive definite matrix, then ℋ W − 1 = O P ( 1 ) superscript subscript ℋ 𝑊 1 subscript 𝑂 𝑃 1 \mathcal{H}_{W}^{-1}=O_{P}(1) caligraphic_H start_POSTSUBSCRIPT italic_W end_POSTSUBSCRIPT start_POSTSUPERSCRIPT - 1 end_POSTSUPERSCRIPT = italic_O start_POSTSUBSCRIPT italic_P end_POSTSUBSCRIPT ( 1 ) . And due to (S3.2 ), we have
V = ℋ W − 1 V c ℋ W − 1 = 1 r ℋ W − 1 ( r V c ) ℋ W − 1 = O P ( r − 1 ) . 𝑉 superscript subscript ℋ 𝑊 1 subscript 𝑉 𝑐 superscript subscript ℋ 𝑊 1 1 𝑟 superscript subscript ℋ 𝑊 1 𝑟 subscript 𝑉 𝑐 superscript subscript ℋ 𝑊 1 subscript 𝑂 𝑃 superscript 𝑟 1 V=\mathcal{H}_{W}^{-1}V_{c}\mathcal{H}_{W}^{-1}=\frac{1}{r}\mathcal{H}_{W}^{-1%
}(rV_{c})\mathcal{H}_{W}^{-1}=O_{P}(r^{-1}). italic_V = caligraphic_H start_POSTSUBSCRIPT italic_W end_POSTSUBSCRIPT start_POSTSUPERSCRIPT - 1 end_POSTSUPERSCRIPT italic_V start_POSTSUBSCRIPT italic_c end_POSTSUBSCRIPT caligraphic_H start_POSTSUBSCRIPT italic_W end_POSTSUBSCRIPT start_POSTSUPERSCRIPT - 1 end_POSTSUPERSCRIPT = divide start_ARG 1 end_ARG start_ARG italic_r end_ARG caligraphic_H start_POSTSUBSCRIPT italic_W end_POSTSUBSCRIPT start_POSTSUPERSCRIPT - 1 end_POSTSUPERSCRIPT ( italic_r italic_V start_POSTSUBSCRIPT italic_c end_POSTSUBSCRIPT ) caligraphic_H start_POSTSUBSCRIPT italic_W end_POSTSUBSCRIPT start_POSTSUPERSCRIPT - 1 end_POSTSUPERSCRIPT = italic_O start_POSTSUBSCRIPT italic_P end_POSTSUBSCRIPT ( italic_r start_POSTSUPERSCRIPT - 1 end_POSTSUPERSCRIPT ) .
(S3.5)
Therefore, combining (S2.1 ), (S3.4 ) and (S3.5 ), then
V − 1 / 2 ( 𝜷 ~ − 𝜷 ^ ) superscript 𝑉 1 2 ~ 𝜷 ^ 𝜷 \displaystyle V^{-1/2}(\tilde{\boldsymbol{\beta}}-\hat{\boldsymbol{\beta}}) italic_V start_POSTSUPERSCRIPT - 1 / 2 end_POSTSUPERSCRIPT ( over~ start_ARG bold_italic_β end_ARG - over^ start_ARG bold_italic_β end_ARG )
= − V − 1 / 2 ℋ ~ W − 1 ℓ ˙ * ( 𝜷 ^ ) absent superscript 𝑉 1 2 superscript subscript ~ ℋ 𝑊 1 superscript ˙ ℓ ^ 𝜷 \displaystyle=-V^{-1/2}\tilde{\mathcal{H}}_{W}^{-1}\dot{\ell}^{*}(\hat{%
\boldsymbol{\beta}}) = - italic_V start_POSTSUPERSCRIPT - 1 / 2 end_POSTSUPERSCRIPT over~ start_ARG caligraphic_H end_ARG start_POSTSUBSCRIPT italic_W end_POSTSUBSCRIPT start_POSTSUPERSCRIPT - 1 end_POSTSUPERSCRIPT over˙ start_ARG roman_ℓ end_ARG start_POSTSUPERSCRIPT * end_POSTSUPERSCRIPT ( over^ start_ARG bold_italic_β end_ARG )
= − V − 1 / 2 ℋ W − 1 ℓ ˙ * ( 𝜷 ^ ) − V − 1 / 2 ( ℋ ~ W − 1 − ℋ W − 1 ) ℓ ˙ * ( 𝜷 ^ ) absent superscript 𝑉 1 2 superscript subscript ℋ 𝑊 1 superscript ˙ ℓ ^ 𝜷 superscript 𝑉 1 2 superscript subscript ~ ℋ 𝑊 1 superscript subscript ℋ 𝑊 1 superscript ˙ ℓ ^ 𝜷 \displaystyle=-V^{-1/2}\mathcal{H}_{W}^{-1}\dot{\ell}^{*}(\hat{\boldsymbol{%
\beta}})-V^{-1/2}(\tilde{\mathcal{H}}_{W}^{-1}-\mathcal{H}_{W}^{-1})\dot{\ell}%
^{*}(\hat{\boldsymbol{\beta}}) = - italic_V start_POSTSUPERSCRIPT - 1 / 2 end_POSTSUPERSCRIPT caligraphic_H start_POSTSUBSCRIPT italic_W end_POSTSUBSCRIPT start_POSTSUPERSCRIPT - 1 end_POSTSUPERSCRIPT over˙ start_ARG roman_ℓ end_ARG start_POSTSUPERSCRIPT * end_POSTSUPERSCRIPT ( over^ start_ARG bold_italic_β end_ARG ) - italic_V start_POSTSUPERSCRIPT - 1 / 2 end_POSTSUPERSCRIPT ( over~ start_ARG caligraphic_H end_ARG start_POSTSUBSCRIPT italic_W end_POSTSUBSCRIPT start_POSTSUPERSCRIPT - 1 end_POSTSUPERSCRIPT - caligraphic_H start_POSTSUBSCRIPT italic_W end_POSTSUBSCRIPT start_POSTSUPERSCRIPT - 1 end_POSTSUPERSCRIPT ) over˙ start_ARG roman_ℓ end_ARG start_POSTSUPERSCRIPT * end_POSTSUPERSCRIPT ( over^ start_ARG bold_italic_β end_ARG )
= − V − 1 / 2 ℋ W − 1 V c 1 / 2 V c − 1 / 2 ℓ ˙ * ( 𝜷 ^ ) + O P | ℱ n ( r − 1 / 2 ) . absent superscript 𝑉 1 2 superscript subscript ℋ 𝑊 1 superscript subscript 𝑉 𝑐 1 2 superscript subscript 𝑉 𝑐 1 2 superscript ˙ ℓ ^ 𝜷 subscript 𝑂 conditional 𝑃 subscript ℱ 𝑛 superscript 𝑟 1 2 \displaystyle=-V^{-1/2}\mathcal{H}_{W}^{-1}V_{c}^{1/2}V_{c}^{-1/2}\dot{\ell}^{%
*}(\hat{\boldsymbol{\beta}})+O_{P|\mathcal{F}_{n}}(r^{-1/2}). = - italic_V start_POSTSUPERSCRIPT - 1 / 2 end_POSTSUPERSCRIPT caligraphic_H start_POSTSUBSCRIPT italic_W end_POSTSUBSCRIPT start_POSTSUPERSCRIPT - 1 end_POSTSUPERSCRIPT italic_V start_POSTSUBSCRIPT italic_c end_POSTSUBSCRIPT start_POSTSUPERSCRIPT 1 / 2 end_POSTSUPERSCRIPT italic_V start_POSTSUBSCRIPT italic_c end_POSTSUBSCRIPT start_POSTSUPERSCRIPT - 1 / 2 end_POSTSUPERSCRIPT over˙ start_ARG roman_ℓ end_ARG start_POSTSUPERSCRIPT * end_POSTSUPERSCRIPT ( over^ start_ARG bold_italic_β end_ARG ) + italic_O start_POSTSUBSCRIPT italic_P | caligraphic_F start_POSTSUBSCRIPT italic_n end_POSTSUBSCRIPT end_POSTSUBSCRIPT ( italic_r start_POSTSUPERSCRIPT - 1 / 2 end_POSTSUPERSCRIPT ) .
Note that
V − 1 / 2 ℋ W − 1 V c 1 / 2 ( V − 1 / 2 ℋ W − 1 V c 1 / 2 ) T = V − 1 / 2 ℋ W − 1 V c 1 / 2 V c 1 / 2 ℋ W − 1 V − 1 / 2 = I , superscript 𝑉 1 2 superscript subscript ℋ 𝑊 1 superscript subscript 𝑉 𝑐 1 2 superscript superscript 𝑉 1 2 superscript subscript ℋ 𝑊 1 superscript subscript 𝑉 𝑐 1 2 𝑇 superscript 𝑉 1 2 superscript subscript ℋ 𝑊 1 superscript subscript 𝑉 𝑐 1 2 superscript subscript 𝑉 𝑐 1 2 superscript subscript ℋ 𝑊 1 superscript 𝑉 1 2 𝐼 V^{-1/2}\mathcal{H}_{W}^{-1}V_{c}^{1/2}(V^{-1/2}\mathcal{H}_{W}^{-1}V_{c}^{1/2%
})^{T}=V^{-1/2}\mathcal{H}_{W}^{-1}V_{c}^{1/2}V_{c}^{1/2}\mathcal{H}_{W}^{-1}V%
^{-1/2}=I, italic_V start_POSTSUPERSCRIPT - 1 / 2 end_POSTSUPERSCRIPT caligraphic_H start_POSTSUBSCRIPT italic_W end_POSTSUBSCRIPT start_POSTSUPERSCRIPT - 1 end_POSTSUPERSCRIPT italic_V start_POSTSUBSCRIPT italic_c end_POSTSUBSCRIPT start_POSTSUPERSCRIPT 1 / 2 end_POSTSUPERSCRIPT ( italic_V start_POSTSUPERSCRIPT - 1 / 2 end_POSTSUPERSCRIPT caligraphic_H start_POSTSUBSCRIPT italic_W end_POSTSUBSCRIPT start_POSTSUPERSCRIPT - 1 end_POSTSUPERSCRIPT italic_V start_POSTSUBSCRIPT italic_c end_POSTSUBSCRIPT start_POSTSUPERSCRIPT 1 / 2 end_POSTSUPERSCRIPT ) start_POSTSUPERSCRIPT italic_T end_POSTSUPERSCRIPT = italic_V start_POSTSUPERSCRIPT - 1 / 2 end_POSTSUPERSCRIPT caligraphic_H start_POSTSUBSCRIPT italic_W end_POSTSUBSCRIPT start_POSTSUPERSCRIPT - 1 end_POSTSUPERSCRIPT italic_V start_POSTSUBSCRIPT italic_c end_POSTSUBSCRIPT start_POSTSUPERSCRIPT 1 / 2 end_POSTSUPERSCRIPT italic_V start_POSTSUBSCRIPT italic_c end_POSTSUBSCRIPT start_POSTSUPERSCRIPT 1 / 2 end_POSTSUPERSCRIPT caligraphic_H start_POSTSUBSCRIPT italic_W end_POSTSUBSCRIPT start_POSTSUPERSCRIPT - 1 end_POSTSUPERSCRIPT italic_V start_POSTSUPERSCRIPT - 1 / 2 end_POSTSUPERSCRIPT = italic_I ,
According to the Slutsky theorem, we have
V − 1 / 2 ( 𝜷 ~ − 𝜷 ^ ) → 𝑑 N p ( 𝟎 , I ) , r , n → ∞ . formulae-sequence 𝑑 → superscript 𝑉 1 2 ~ 𝜷 ^ 𝜷 subscript 𝑁 𝑝 0 𝐼 𝑟
→ 𝑛 V^{-1/2}(\tilde{\boldsymbol{\beta}}-\hat{\boldsymbol{\beta}})\xrightarrow{d}N_%
{p}(\mathbf{0},I),r,n\rightarrow\infty. italic_V start_POSTSUPERSCRIPT - 1 / 2 end_POSTSUPERSCRIPT ( over~ start_ARG bold_italic_β end_ARG - over^ start_ARG bold_italic_β end_ARG ) start_ARROW overitalic_d → end_ARROW italic_N start_POSTSUBSCRIPT italic_p end_POSTSUBSCRIPT ( bold_0 , italic_I ) , italic_r , italic_n → ∞ .
(S3.6)
Then the theorem is proved.
S6 Proof of Theorem 6
First, we consider the case where m = 1 𝑚 1 m=1 italic_m = 1 . Note that
𝜷 ˇ − 𝜷 ^ ˇ 𝜷 ^ 𝜷 \displaystyle\check{\boldsymbol{\beta}}-\hat{\boldsymbol{\beta}} overroman_ˇ start_ARG bold_italic_β end_ARG - over^ start_ARG bold_italic_β end_ARG
= ( 1 n ∑ i = 1 n ψ i 𝐖 i 𝐖 i T − 𝚺 u u ) − 1 ⋅ [ 1 n ∑ i = 1 n ψ i 𝐖 i ( y i − 𝐖 i T 𝜷 ^ ) + 𝚺 u u 𝜷 ^ ] absent ⋅ superscript 1 𝑛 superscript subscript 𝑖 1 𝑛 subscript 𝜓 𝑖 subscript 𝐖 𝑖 superscript subscript 𝐖 𝑖 𝑇 subscript 𝚺 𝑢 𝑢 1 delimited-[] 1 𝑛 superscript subscript 𝑖 1 𝑛 subscript 𝜓 𝑖 subscript 𝐖 𝑖 subscript 𝑦 𝑖 superscript subscript 𝐖 𝑖 𝑇 ^ 𝜷 subscript 𝚺 𝑢 𝑢 ^ 𝜷 \displaystyle=\left(\frac{1}{n}\sum_{i=1}^{n}\psi_{i}\mathbf{W}_{i}\mathbf{W}_%
{i}^{T}-\boldsymbol{\Sigma}_{uu}\right)^{-1}\cdot\left[\frac{1}{n}\sum_{i=1}^{%
n}\psi_{i}\mathbf{W}_{i}(y_{i}-\mathbf{W}_{i}^{T}\hat{\boldsymbol{\beta}})+%
\boldsymbol{\Sigma}_{uu}\hat{\boldsymbol{\beta}}\right] = ( divide start_ARG 1 end_ARG start_ARG italic_n end_ARG ∑ start_POSTSUBSCRIPT italic_i = 1 end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_n end_POSTSUPERSCRIPT italic_ψ start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT bold_W start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT bold_W start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_T end_POSTSUPERSCRIPT - bold_Σ start_POSTSUBSCRIPT italic_u italic_u end_POSTSUBSCRIPT ) start_POSTSUPERSCRIPT - 1 end_POSTSUPERSCRIPT ⋅ [ divide start_ARG 1 end_ARG start_ARG italic_n end_ARG ∑ start_POSTSUBSCRIPT italic_i = 1 end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_n end_POSTSUPERSCRIPT italic_ψ start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT bold_W start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT ( italic_y start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT - bold_W start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_T end_POSTSUPERSCRIPT over^ start_ARG bold_italic_β end_ARG ) + bold_Σ start_POSTSUBSCRIPT italic_u italic_u end_POSTSUBSCRIPT over^ start_ARG bold_italic_β end_ARG ]
(S6.1)
= − ( ℋ ˇ W ) − 1 L ˙ * ( 𝜷 ^ ) , absent superscript subscript ˇ ℋ 𝑊 1 superscript ˙ 𝐿 ^ 𝜷 \displaystyle=-(\check{\mathcal{H}}_{W})^{-1}\dot{L}^{*}(\hat{\boldsymbol{%
\beta}}), = - ( overroman_ˇ start_ARG caligraphic_H end_ARG start_POSTSUBSCRIPT italic_W end_POSTSUBSCRIPT ) start_POSTSUPERSCRIPT - 1 end_POSTSUPERSCRIPT over˙ start_ARG italic_L end_ARG start_POSTSUPERSCRIPT * end_POSTSUPERSCRIPT ( over^ start_ARG bold_italic_β end_ARG ) ,
where ℋ ˇ W = 1 n ∑ i = 1 n ψ i ( 𝐖 i 𝐖 i T ) − 𝚺 u u , L ˙ * ( 𝜷 ) = 1 n ∑ i = 1 n ψ i [ − 𝐖 i ( y i − 𝐖 i T 𝜷 ) ] − 𝚺 u u 𝜷 . formulae-sequence subscript ˇ ℋ 𝑊 1 𝑛 superscript subscript 𝑖 1 𝑛 subscript 𝜓 𝑖 subscript 𝐖 𝑖 superscript subscript 𝐖 𝑖 𝑇 subscript 𝚺 𝑢 𝑢 superscript ˙ 𝐿 𝜷 1 𝑛 superscript subscript 𝑖 1 𝑛 subscript 𝜓 𝑖 delimited-[] subscript 𝐖 𝑖 subscript 𝑦 𝑖 superscript subscript 𝐖 𝑖 𝑇 𝜷 subscript 𝚺 𝑢 𝑢 𝜷 \check{\mathcal{H}}_{W}=\frac{1}{n}\sum_{i=1}^{n}\psi_{i}(\mathbf{W}_{i}%
\mathbf{W}_{i}^{T})-\boldsymbol{\Sigma}_{uu},\dot{L}^{*}(\boldsymbol{\beta})=%
\frac{1}{n}\sum_{i=1}^{n}\psi_{i}[-\mathbf{W}_{i}(y_{i}-\mathbf{W}_{i}^{T}%
\boldsymbol{\beta})]\\
-\boldsymbol{\Sigma}_{uu}\boldsymbol{\beta}. overroman_ˇ start_ARG caligraphic_H end_ARG start_POSTSUBSCRIPT italic_W end_POSTSUBSCRIPT = divide start_ARG 1 end_ARG start_ARG italic_n end_ARG ∑ start_POSTSUBSCRIPT italic_i = 1 end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_n end_POSTSUPERSCRIPT italic_ψ start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT ( bold_W start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT bold_W start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_T end_POSTSUPERSCRIPT ) - bold_Σ start_POSTSUBSCRIPT italic_u italic_u end_POSTSUBSCRIPT , over˙ start_ARG italic_L end_ARG start_POSTSUPERSCRIPT * end_POSTSUPERSCRIPT ( bold_italic_β ) = divide start_ARG 1 end_ARG start_ARG italic_n end_ARG ∑ start_POSTSUBSCRIPT italic_i = 1 end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_n end_POSTSUPERSCRIPT italic_ψ start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT [ - bold_W start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT ( italic_y start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT - bold_W start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_T end_POSTSUPERSCRIPT bold_italic_β ) ] - bold_Σ start_POSTSUBSCRIPT italic_u italic_u end_POSTSUBSCRIPT bold_italic_β .
Therefore, it is only necessary to demonstrate that
L ˙ * ( 𝜷 ^ ) = O P | ℱ n ( r − 1 / 2 ) , superscript ˙ 𝐿 ^ 𝜷 subscript 𝑂 conditional 𝑃 subscript ℱ 𝑛 superscript 𝑟 1 2 \dot{L}^{*}(\hat{\boldsymbol{\beta}})=O_{P|\mathcal{F}_{n}}(r^{-1/2}), over˙ start_ARG italic_L end_ARG start_POSTSUPERSCRIPT * end_POSTSUPERSCRIPT ( over^ start_ARG bold_italic_β end_ARG ) = italic_O start_POSTSUBSCRIPT italic_P | caligraphic_F start_POSTSUBSCRIPT italic_n end_POSTSUBSCRIPT end_POSTSUBSCRIPT ( italic_r start_POSTSUPERSCRIPT - 1 / 2 end_POSTSUPERSCRIPT ) ,
(S6.2)
and
ℋ ˇ W − ℋ W = O P | ℱ n ( r − 1 / 2 ) , subscript ˇ ℋ 𝑊 subscript ℋ 𝑊 subscript 𝑂 conditional 𝑃 subscript ℱ 𝑛 superscript 𝑟 1 2 \check{\mathcal{H}}_{W}-\mathcal{H}_{W}=O_{P|\mathcal{F}_{n}}(r^{-1/2}), overroman_ˇ start_ARG caligraphic_H end_ARG start_POSTSUBSCRIPT italic_W end_POSTSUBSCRIPT - caligraphic_H start_POSTSUBSCRIPT italic_W end_POSTSUBSCRIPT = italic_O start_POSTSUBSCRIPT italic_P | caligraphic_F start_POSTSUBSCRIPT italic_n end_POSTSUBSCRIPT end_POSTSUBSCRIPT ( italic_r start_POSTSUPERSCRIPT - 1 / 2 end_POSTSUPERSCRIPT ) ,
(S6.3)
where ℋ W = 1 n ∑ i = 1 n 𝐖 i 𝐖 i T − 𝚺 u u . subscript ℋ 𝑊 1 𝑛 superscript subscript 𝑖 1 𝑛 subscript 𝐖 𝑖 superscript subscript 𝐖 𝑖 𝑇 subscript 𝚺 𝑢 𝑢 \mathcal{H}_{W}=\frac{1}{n}\sum_{i=1}^{n}\mathbf{W}_{i}\mathbf{W}_{i}^{T}-%
\boldsymbol{\Sigma}_{uu}. caligraphic_H start_POSTSUBSCRIPT italic_W end_POSTSUBSCRIPT = divide start_ARG 1 end_ARG start_ARG italic_n end_ARG ∑ start_POSTSUBSCRIPT italic_i = 1 end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_n end_POSTSUPERSCRIPT bold_W start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT bold_W start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_T end_POSTSUPERSCRIPT - bold_Σ start_POSTSUBSCRIPT italic_u italic_u end_POSTSUBSCRIPT .
Note that
E ( ψ i ) = E ( μ i ν i ) = q n ⋅ 1 q n = 1 , 𝐸 subscript 𝜓 𝑖 𝐸 subscript 𝜇 𝑖 subscript 𝜈 𝑖 ⋅ subscript 𝑞 𝑛 1 subscript 𝑞 𝑛 1 E(\psi_{i})=E(\mu_{i}\nu_{i})=q_{n}\cdot\frac{1}{q_{n}}=1, italic_E ( italic_ψ start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT ) = italic_E ( italic_μ start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT italic_ν start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT ) = italic_q start_POSTSUBSCRIPT italic_n end_POSTSUBSCRIPT ⋅ divide start_ARG 1 end_ARG start_ARG italic_q start_POSTSUBSCRIPT italic_n end_POSTSUBSCRIPT end_ARG = 1 ,
E ( ψ i 2 ) = ( q n ( 1 − q n ) + q n 2 ) ⋅ ( b n 2 + 1 q n 2 ) = q n b n 2 + 1 q n , 𝐸 superscript subscript 𝜓 𝑖 2 ⋅ subscript 𝑞 𝑛 1 subscript 𝑞 𝑛 superscript subscript 𝑞 𝑛 2 superscript subscript 𝑏 𝑛 2 1 superscript subscript 𝑞 𝑛 2 subscript 𝑞 𝑛 superscript subscript 𝑏 𝑛 2 1 subscript 𝑞 𝑛 E(\psi_{i}^{2})=(q_{n}(1-q_{n})+q_{n}^{2})\cdot\left(b_{n}^{2}+\frac{1}{q_{n}^%
{2}}\right)=q_{n}b_{n}^{2}+\frac{1}{q_{n}}, italic_E ( italic_ψ start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT start_POSTSUPERSCRIPT 2 end_POSTSUPERSCRIPT ) = ( italic_q start_POSTSUBSCRIPT italic_n end_POSTSUBSCRIPT ( 1 - italic_q start_POSTSUBSCRIPT italic_n end_POSTSUBSCRIPT ) + italic_q start_POSTSUBSCRIPT italic_n end_POSTSUBSCRIPT start_POSTSUPERSCRIPT 2 end_POSTSUPERSCRIPT ) ⋅ ( italic_b start_POSTSUBSCRIPT italic_n end_POSTSUBSCRIPT start_POSTSUPERSCRIPT 2 end_POSTSUPERSCRIPT + divide start_ARG 1 end_ARG start_ARG italic_q start_POSTSUBSCRIPT italic_n end_POSTSUBSCRIPT start_POSTSUPERSCRIPT 2 end_POSTSUPERSCRIPT end_ARG ) = italic_q start_POSTSUBSCRIPT italic_n end_POSTSUBSCRIPT italic_b start_POSTSUBSCRIPT italic_n end_POSTSUBSCRIPT start_POSTSUPERSCRIPT 2 end_POSTSUPERSCRIPT + divide start_ARG 1 end_ARG start_ARG italic_q start_POSTSUBSCRIPT italic_n end_POSTSUBSCRIPT end_ARG ,
and
V a r ( ψ i ) = E ( ψ i 2 ) − ( E ( ψ i ) ) 2 = q n b n 2 + 1 q n − 1 = n a n r , 𝑉 𝑎 𝑟 subscript 𝜓 𝑖 𝐸 superscript subscript 𝜓 𝑖 2 superscript 𝐸 subscript 𝜓 𝑖 2 subscript 𝑞 𝑛 superscript subscript 𝑏 𝑛 2 1 subscript 𝑞 𝑛 1 𝑛 subscript 𝑎 𝑛 𝑟 Var(\psi_{i})=E(\psi_{i}^{2})-(E(\psi_{i}))^{2}=q_{n}b_{n}^{2}+\frac{1}{q_{n}}%
-1=\frac{na_{n}}{r}, italic_V italic_a italic_r ( italic_ψ start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT ) = italic_E ( italic_ψ start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT start_POSTSUPERSCRIPT 2 end_POSTSUPERSCRIPT ) - ( italic_E ( italic_ψ start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT ) ) start_POSTSUPERSCRIPT 2 end_POSTSUPERSCRIPT = italic_q start_POSTSUBSCRIPT italic_n end_POSTSUBSCRIPT italic_b start_POSTSUBSCRIPT italic_n end_POSTSUBSCRIPT start_POSTSUPERSCRIPT 2 end_POSTSUPERSCRIPT + divide start_ARG 1 end_ARG start_ARG italic_q start_POSTSUBSCRIPT italic_n end_POSTSUBSCRIPT end_ARG - 1 = divide start_ARG italic_n italic_a start_POSTSUBSCRIPT italic_n end_POSTSUBSCRIPT end_ARG start_ARG italic_r end_ARG ,
where a n = b n 2 q n 2 − q n + 1 . subscript 𝑎 𝑛 superscript subscript 𝑏 𝑛 2 superscript subscript 𝑞 𝑛 2 subscript 𝑞 𝑛 1 a_{n}=b_{n}^{2}q_{n}^{2}-q_{n}+1. italic_a start_POSTSUBSCRIPT italic_n end_POSTSUBSCRIPT = italic_b start_POSTSUBSCRIPT italic_n end_POSTSUBSCRIPT start_POSTSUPERSCRIPT 2 end_POSTSUPERSCRIPT italic_q start_POSTSUBSCRIPT italic_n end_POSTSUBSCRIPT start_POSTSUPERSCRIPT 2 end_POSTSUPERSCRIPT - italic_q start_POSTSUBSCRIPT italic_n end_POSTSUBSCRIPT + 1 .
According to the Assumption 5, as q n → 0 → subscript 𝑞 𝑛 0 q_{n}\rightarrow 0 italic_q start_POSTSUBSCRIPT italic_n end_POSTSUBSCRIPT → 0 , lim sup n → ∞ a n = lim sup n → ∞ q n V a r ( ψ ) = lim sup n → ∞ q n ( E ( ψ i 2 ) − 1 ) = < ∞ . \limsup\limits_{n\rightarrow\infty}a_{n}=\limsup\limits_{n\rightarrow\infty}q_%
{n}Var(\psi)=\limsup\limits_{n\rightarrow\infty}q_{n}(E(\psi_{i}^{2})-1)=<\infty. lim sup start_POSTSUBSCRIPT italic_n → ∞ end_POSTSUBSCRIPT italic_a start_POSTSUBSCRIPT italic_n end_POSTSUBSCRIPT = lim sup start_POSTSUBSCRIPT italic_n → ∞ end_POSTSUBSCRIPT italic_q start_POSTSUBSCRIPT italic_n end_POSTSUBSCRIPT italic_V italic_a italic_r ( italic_ψ ) = lim sup start_POSTSUBSCRIPT italic_n → ∞ end_POSTSUBSCRIPT italic_q start_POSTSUBSCRIPT italic_n end_POSTSUBSCRIPT ( italic_E ( italic_ψ start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT start_POSTSUPERSCRIPT 2 end_POSTSUPERSCRIPT ) - 1 ) = < ∞ .
To prove (S6.2 ), we calculate directly to obtain
E ( L ˙ * ( 𝜷 ^ ) | ℱ n ) 𝐸 conditional superscript ˙ 𝐿 ^ 𝜷 subscript ℱ 𝑛 \displaystyle E(\dot{L}^{*}(\hat{\boldsymbol{\beta}})|\mathcal{F}_{n}) italic_E ( over˙ start_ARG italic_L end_ARG start_POSTSUPERSCRIPT * end_POSTSUPERSCRIPT ( over^ start_ARG bold_italic_β end_ARG ) | caligraphic_F start_POSTSUBSCRIPT italic_n end_POSTSUBSCRIPT )
= E { 1 n ∑ i = 1 n ψ i [ − 𝐖 i ( y i − 𝐖 i T 𝜷 ^ ) ] − 𝚺 u u 𝜷 ^ | ℱ n } absent 𝐸 conditional-set 1 𝑛 superscript subscript 𝑖 1 𝑛 subscript 𝜓 𝑖 delimited-[] subscript 𝐖 𝑖 subscript 𝑦 𝑖 superscript subscript 𝐖 𝑖 𝑇 ^ 𝜷 subscript 𝚺 𝑢 𝑢 ^ 𝜷 subscript ℱ 𝑛 \displaystyle=E\left\{\frac{1}{n}\sum_{i=1}^{n}\psi_{i}[-\mathbf{W}_{i}(y_{i}-%
\mathbf{W}_{i}^{T}\hat{\boldsymbol{\beta}})]-\boldsymbol{\Sigma}_{uu}\hat{%
\boldsymbol{\beta}}\bigg{|}\mathcal{F}_{n}\right\} = italic_E { divide start_ARG 1 end_ARG start_ARG italic_n end_ARG ∑ start_POSTSUBSCRIPT italic_i = 1 end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_n end_POSTSUPERSCRIPT italic_ψ start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT [ - bold_W start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT ( italic_y start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT - bold_W start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_T end_POSTSUPERSCRIPT over^ start_ARG bold_italic_β end_ARG ) ] - bold_Σ start_POSTSUBSCRIPT italic_u italic_u end_POSTSUBSCRIPT over^ start_ARG bold_italic_β end_ARG | caligraphic_F start_POSTSUBSCRIPT italic_n end_POSTSUBSCRIPT }
= 1 n ∑ i = 1 n [ − 𝐖 i ( y i − 𝐖 i T 𝜷 ^ ) ] − 𝚺 u u 𝜷 ^ = 𝟎 . absent 1 𝑛 superscript subscript 𝑖 1 𝑛 delimited-[] subscript 𝐖 𝑖 subscript 𝑦 𝑖 superscript subscript 𝐖 𝑖 𝑇 ^ 𝜷 subscript 𝚺 𝑢 𝑢 ^ 𝜷 0 \displaystyle=\frac{1}{n}\sum_{i=1}^{n}[-\mathbf{W}_{i}(y_{i}-\mathbf{W}_{i}^{%
T}\hat{\boldsymbol{\beta}})]-\boldsymbol{\Sigma}_{uu}\hat{\boldsymbol{\beta}}=%
\mathbf{0}. = divide start_ARG 1 end_ARG start_ARG italic_n end_ARG ∑ start_POSTSUBSCRIPT italic_i = 1 end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_n end_POSTSUPERSCRIPT [ - bold_W start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT ( italic_y start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT - bold_W start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_T end_POSTSUPERSCRIPT over^ start_ARG bold_italic_β end_ARG ) ] - bold_Σ start_POSTSUBSCRIPT italic_u italic_u end_POSTSUBSCRIPT over^ start_ARG bold_italic_β end_ARG = bold_0 .
For the j 𝑗 j italic_j -th element of L ˙ * ( 𝜷 ^ ) superscript ˙ 𝐿 ^ 𝜷 \dot{L}^{*}(\hat{\boldsymbol{\beta}}) over˙ start_ARG italic_L end_ARG start_POSTSUPERSCRIPT * end_POSTSUPERSCRIPT ( over^ start_ARG bold_italic_β end_ARG ) , represented as L ˙ j * ( 𝜷 ^ ) = 1 n ∑ i = 1 n ψ i [ − w i j ( y i − 𝐖 i T 𝜷 ^ ) ] − ( 𝚺 u u 𝜷 ^ ) j superscript subscript ˙ 𝐿 𝑗 ^ 𝜷 1 𝑛 superscript subscript 𝑖 1 𝑛 subscript 𝜓 𝑖 delimited-[] subscript 𝑤 𝑖 𝑗 subscript 𝑦 𝑖 superscript subscript 𝐖 𝑖 𝑇 ^ 𝜷 subscript subscript 𝚺 𝑢 𝑢 ^ 𝜷 𝑗 \dot{L}_{j}^{*}(\hat{\boldsymbol{\beta}})=\frac{1}{n}\sum_{i=1}^{n}\psi_{i}[-w%
_{ij}(y_{i}-\mathbf{W}_{i}^{T}\hat{\boldsymbol{\beta}})]-(\boldsymbol{\Sigma}_%
{uu}\hat{\boldsymbol{\beta}})_{j} over˙ start_ARG italic_L end_ARG start_POSTSUBSCRIPT italic_j end_POSTSUBSCRIPT start_POSTSUPERSCRIPT * end_POSTSUPERSCRIPT ( over^ start_ARG bold_italic_β end_ARG ) = divide start_ARG 1 end_ARG start_ARG italic_n end_ARG ∑ start_POSTSUBSCRIPT italic_i = 1 end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_n end_POSTSUPERSCRIPT italic_ψ start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT [ - italic_w start_POSTSUBSCRIPT italic_i italic_j end_POSTSUBSCRIPT ( italic_y start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT - bold_W start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_T end_POSTSUPERSCRIPT over^ start_ARG bold_italic_β end_ARG ) ] - ( bold_Σ start_POSTSUBSCRIPT italic_u italic_u end_POSTSUBSCRIPT over^ start_ARG bold_italic_β end_ARG ) start_POSTSUBSCRIPT italic_j end_POSTSUBSCRIPT .
By (S2.4 ) and Assumption 6, we have
V a r ( L ˙ j * ( 𝜷 ^ ) | ℱ n ) 𝑉 𝑎 𝑟 conditional superscript subscript ˙ 𝐿 𝑗 ^ 𝜷 subscript ℱ 𝑛 \displaystyle Var(\dot{L}_{j}^{*}(\hat{\boldsymbol{\beta}})|\mathcal{F}_{n}) italic_V italic_a italic_r ( over˙ start_ARG italic_L end_ARG start_POSTSUBSCRIPT italic_j end_POSTSUBSCRIPT start_POSTSUPERSCRIPT * end_POSTSUPERSCRIPT ( over^ start_ARG bold_italic_β end_ARG ) | caligraphic_F start_POSTSUBSCRIPT italic_n end_POSTSUBSCRIPT )
= V a r { 1 n ∑ i = 1 n ψ i [ − w i j ( y i − 𝐖 i T 𝜷 ^ ) ] − ( 𝚺 u u 𝜷 ^ ) j | ℱ n } absent 𝑉 𝑎 𝑟 conditional-set 1 𝑛 superscript subscript 𝑖 1 𝑛 subscript 𝜓 𝑖 delimited-[] subscript 𝑤 𝑖 𝑗 subscript 𝑦 𝑖 superscript subscript 𝐖 𝑖 𝑇 ^ 𝜷 subscript subscript 𝚺 𝑢 𝑢 ^ 𝜷 𝑗 subscript ℱ 𝑛 \displaystyle=Var\left\{\frac{1}{n}\sum_{i=1}^{n}\psi_{i}[-w_{ij}(y_{i}-%
\mathbf{W}_{i}^{T}\hat{\boldsymbol{\beta}})]-(\boldsymbol{\Sigma}_{uu}\hat{%
\boldsymbol{\beta}})_{j}\bigg{|}\mathcal{F}_{n}\right\} = italic_V italic_a italic_r { divide start_ARG 1 end_ARG start_ARG italic_n end_ARG ∑ start_POSTSUBSCRIPT italic_i = 1 end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_n end_POSTSUPERSCRIPT italic_ψ start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT [ - italic_w start_POSTSUBSCRIPT italic_i italic_j end_POSTSUBSCRIPT ( italic_y start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT - bold_W start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_T end_POSTSUPERSCRIPT over^ start_ARG bold_italic_β end_ARG ) ] - ( bold_Σ start_POSTSUBSCRIPT italic_u italic_u end_POSTSUBSCRIPT over^ start_ARG bold_italic_β end_ARG ) start_POSTSUBSCRIPT italic_j end_POSTSUBSCRIPT | caligraphic_F start_POSTSUBSCRIPT italic_n end_POSTSUBSCRIPT }
= 1 n 2 n a n r ∑ i = 1 n w i j 2 ( y i − 𝐖 i T 𝜷 ^ ) 2 absent 1 superscript 𝑛 2 𝑛 subscript 𝑎 𝑛 𝑟 superscript subscript 𝑖 1 𝑛 superscript subscript 𝑤 𝑖 𝑗 2 superscript subscript 𝑦 𝑖 superscript subscript 𝐖 𝑖 𝑇 ^ 𝜷 2 \displaystyle=\frac{1}{n^{2}}\frac{na_{n}}{r}\sum_{i=1}^{n}w_{ij}^{2}(y_{i}-%
\mathbf{W}_{i}^{T}\hat{\boldsymbol{\beta}})^{2} = divide start_ARG 1 end_ARG start_ARG italic_n start_POSTSUPERSCRIPT 2 end_POSTSUPERSCRIPT end_ARG divide start_ARG italic_n italic_a start_POSTSUBSCRIPT italic_n end_POSTSUBSCRIPT end_ARG start_ARG italic_r end_ARG ∑ start_POSTSUBSCRIPT italic_i = 1 end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_n end_POSTSUPERSCRIPT italic_w start_POSTSUBSCRIPT italic_i italic_j end_POSTSUBSCRIPT start_POSTSUPERSCRIPT 2 end_POSTSUPERSCRIPT ( italic_y start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT - bold_W start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_T end_POSTSUPERSCRIPT over^ start_ARG bold_italic_β end_ARG ) start_POSTSUPERSCRIPT 2 end_POSTSUPERSCRIPT
≤ a n r ∑ i = 1 n ‖ 𝐖 i ‖ 2 ( y i − 𝐖 i T 𝜷 ^ ) 2 n absent subscript 𝑎 𝑛 𝑟 superscript subscript 𝑖 1 𝑛 superscript norm subscript 𝐖 𝑖 2 superscript subscript 𝑦 𝑖 superscript subscript 𝐖 𝑖 𝑇 ^ 𝜷 2 𝑛 \displaystyle\leq\frac{a_{n}}{r}\sum_{i=1}^{n}\frac{\|\mathbf{W}_{i}\|^{2}(y_{%
i}-\mathbf{W}_{i}^{T}\hat{\boldsymbol{\beta}})^{2}}{n} ≤ divide start_ARG italic_a start_POSTSUBSCRIPT italic_n end_POSTSUBSCRIPT end_ARG start_ARG italic_r end_ARG ∑ start_POSTSUBSCRIPT italic_i = 1 end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_n end_POSTSUPERSCRIPT divide start_ARG ∥ bold_W start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT ∥ start_POSTSUPERSCRIPT 2 end_POSTSUPERSCRIPT ( italic_y start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT - bold_W start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_T end_POSTSUPERSCRIPT over^ start_ARG bold_italic_β end_ARG ) start_POSTSUPERSCRIPT 2 end_POSTSUPERSCRIPT end_ARG start_ARG italic_n end_ARG
= 1 r O P ( 1 ) = O P ( r − 1 ) . absent 1 𝑟 subscript 𝑂 𝑃 1 subscript 𝑂 𝑃 superscript 𝑟 1 \displaystyle=\frac{1}{r}O_{P}(1)=O_{P}(r^{-1}). = divide start_ARG 1 end_ARG start_ARG italic_r end_ARG italic_O start_POSTSUBSCRIPT italic_P end_POSTSUBSCRIPT ( 1 ) = italic_O start_POSTSUBSCRIPT italic_P end_POSTSUBSCRIPT ( italic_r start_POSTSUPERSCRIPT - 1 end_POSTSUPERSCRIPT ) .
From the Chebyshev inequality, for sufficiently large M 𝑀 M italic_M , we have
P ( ‖ L ˙ * ( 𝜷 ^ ) ‖ ≥ r − 1 / 2 M | ℱ n ) 𝑃 norm superscript ˙ 𝐿 ^ 𝜷 conditional superscript 𝑟 1 2 𝑀 subscript ℱ 𝑛 \displaystyle P(\|\dot{L}^{*}(\hat{\boldsymbol{\beta}})\|\geq r^{-1/2}M|%
\mathcal{F}_{n}) italic_P ( ∥ over˙ start_ARG italic_L end_ARG start_POSTSUPERSCRIPT * end_POSTSUPERSCRIPT ( over^ start_ARG bold_italic_β end_ARG ) ∥ ≥ italic_r start_POSTSUPERSCRIPT - 1 / 2 end_POSTSUPERSCRIPT italic_M | caligraphic_F start_POSTSUBSCRIPT italic_n end_POSTSUBSCRIPT )
≤ r E ( ‖ L ˙ * ( 𝜷 ^ ) ‖ 2 | ℱ n ) M 2 absent 𝑟 𝐸 conditional superscript norm superscript ˙ 𝐿 ^ 𝜷 2 subscript ℱ 𝑛 superscript 𝑀 2 \displaystyle\leq\frac{rE(\|\dot{L}^{*}(\hat{\boldsymbol{\beta}})\|^{2}|%
\mathcal{F}_{n})}{M^{2}} ≤ divide start_ARG italic_r italic_E ( ∥ over˙ start_ARG italic_L end_ARG start_POSTSUPERSCRIPT * end_POSTSUPERSCRIPT ( over^ start_ARG bold_italic_β end_ARG ) ∥ start_POSTSUPERSCRIPT 2 end_POSTSUPERSCRIPT | caligraphic_F start_POSTSUBSCRIPT italic_n end_POSTSUBSCRIPT ) end_ARG start_ARG italic_M start_POSTSUPERSCRIPT 2 end_POSTSUPERSCRIPT end_ARG
= r ∑ j = 1 p E ( L ˙ j * ( 𝜷 ^ ) | ℱ n ) 2 M 2 absent 𝑟 superscript subscript 𝑗 1 𝑝 𝐸 superscript conditional superscript subscript ˙ 𝐿 𝑗 ^ 𝜷 subscript ℱ 𝑛 2 superscript 𝑀 2 \displaystyle=\frac{r\sum_{j=1}^{p}E(\dot{L}_{j}^{*}(\hat{\boldsymbol{\beta}})%
|\mathcal{F}_{n})^{2}}{M^{2}} = divide start_ARG italic_r ∑ start_POSTSUBSCRIPT italic_j = 1 end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_p end_POSTSUPERSCRIPT italic_E ( over˙ start_ARG italic_L end_ARG start_POSTSUBSCRIPT italic_j end_POSTSUBSCRIPT start_POSTSUPERSCRIPT * end_POSTSUPERSCRIPT ( over^ start_ARG bold_italic_β end_ARG ) | caligraphic_F start_POSTSUBSCRIPT italic_n end_POSTSUBSCRIPT ) start_POSTSUPERSCRIPT 2 end_POSTSUPERSCRIPT end_ARG start_ARG italic_M start_POSTSUPERSCRIPT 2 end_POSTSUPERSCRIPT end_ARG
= O P ( 1 ) M 2 → 0 , n , r → ∞ . formulae-sequence absent subscript 𝑂 𝑃 1 superscript 𝑀 2 → 0 → 𝑛 𝑟
\displaystyle=\frac{O_{P}(1)}{M^{2}}\rightarrow 0,n,r\rightarrow\infty. = divide start_ARG italic_O start_POSTSUBSCRIPT italic_P end_POSTSUBSCRIPT ( 1 ) end_ARG start_ARG italic_M start_POSTSUPERSCRIPT 2 end_POSTSUPERSCRIPT end_ARG → 0 , italic_n , italic_r → ∞ .
Thus, the equation (S6.2 ) is proved.
To prove (S6.3 ), We calculate directly to obtain
E ( ℋ ˇ W | ℱ n ) = ℋ W , 𝐸 conditional subscript ˇ ℋ 𝑊 subscript ℱ 𝑛 subscript ℋ 𝑊 E(\check{\mathcal{H}}_{W}|\mathcal{F}_{n})=\mathcal{H}_{W}, italic_E ( overroman_ˇ start_ARG caligraphic_H end_ARG start_POSTSUBSCRIPT italic_W end_POSTSUBSCRIPT | caligraphic_F start_POSTSUBSCRIPT italic_n end_POSTSUBSCRIPT ) = caligraphic_H start_POSTSUBSCRIPT italic_W end_POSTSUBSCRIPT ,
For any element ℋ ˇ W j 1 j 2 , 1 ≤ j 1 , j 2 ≤ p formulae-sequence superscript subscript ˇ ℋ 𝑊 subscript 𝑗 1 subscript 𝑗 2 1
subscript 𝑗 1 subscript 𝑗 2 𝑝 \check{\mathcal{H}}_{W}^{j_{1}j_{2}},1\leq j_{1},j_{2}\leq p overroman_ˇ start_ARG caligraphic_H end_ARG start_POSTSUBSCRIPT italic_W end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_j start_POSTSUBSCRIPT 1 end_POSTSUBSCRIPT italic_j start_POSTSUBSCRIPT 2 end_POSTSUBSCRIPT end_POSTSUPERSCRIPT , 1 ≤ italic_j start_POSTSUBSCRIPT 1 end_POSTSUBSCRIPT , italic_j start_POSTSUBSCRIPT 2 end_POSTSUBSCRIPT ≤ italic_p of ℋ ˇ W subscript ˇ ℋ 𝑊 \check{\mathcal{H}}_{W} overroman_ˇ start_ARG caligraphic_H end_ARG start_POSTSUBSCRIPT italic_W end_POSTSUBSCRIPT , by Assumptions 2 and 5, we have
V a r ( ℋ ˇ W j 1 j 2 | ℱ n ) 𝑉 𝑎 𝑟 conditional superscript subscript ˇ ℋ 𝑊 subscript 𝑗 1 subscript 𝑗 2 subscript ℱ 𝑛 \displaystyle Var(\check{\mathcal{H}}_{W}^{j_{1}j_{2}}|\mathcal{F}_{n}) italic_V italic_a italic_r ( overroman_ˇ start_ARG caligraphic_H end_ARG start_POSTSUBSCRIPT italic_W end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_j start_POSTSUBSCRIPT 1 end_POSTSUBSCRIPT italic_j start_POSTSUBSCRIPT 2 end_POSTSUBSCRIPT end_POSTSUPERSCRIPT | caligraphic_F start_POSTSUBSCRIPT italic_n end_POSTSUBSCRIPT )
= V a r [ 1 n ∑ i = 1 n ψ i ( W i j 1 W i j 2 ) − ( 𝚺 u u ) j 1 j 2 | ℱ n ] absent 𝑉 𝑎 𝑟 delimited-[] 1 𝑛 superscript subscript 𝑖 1 𝑛 subscript 𝜓 𝑖 subscript 𝑊 𝑖 subscript 𝑗 1 subscript 𝑊 𝑖 subscript 𝑗 2 conditional subscript subscript 𝚺 𝑢 𝑢 subscript 𝑗 1 subscript 𝑗 2 subscript ℱ 𝑛 \displaystyle=Var\left[\frac{1}{n}\sum_{i=1}^{n}\psi_{i}(W_{ij_{1}}W_{ij_{2}})%
-(\boldsymbol{\Sigma}_{uu})_{j_{1}j_{2}}\Big{|}\mathcal{F}_{n}\right] = italic_V italic_a italic_r [ divide start_ARG 1 end_ARG start_ARG italic_n end_ARG ∑ start_POSTSUBSCRIPT italic_i = 1 end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_n end_POSTSUPERSCRIPT italic_ψ start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT ( italic_W start_POSTSUBSCRIPT italic_i italic_j start_POSTSUBSCRIPT 1 end_POSTSUBSCRIPT end_POSTSUBSCRIPT italic_W start_POSTSUBSCRIPT italic_i italic_j start_POSTSUBSCRIPT 2 end_POSTSUBSCRIPT end_POSTSUBSCRIPT ) - ( bold_Σ start_POSTSUBSCRIPT italic_u italic_u end_POSTSUBSCRIPT ) start_POSTSUBSCRIPT italic_j start_POSTSUBSCRIPT 1 end_POSTSUBSCRIPT italic_j start_POSTSUBSCRIPT 2 end_POSTSUBSCRIPT end_POSTSUBSCRIPT | caligraphic_F start_POSTSUBSCRIPT italic_n end_POSTSUBSCRIPT ]
= 1 n 2 n a n r ∑ i = 1 n ( W i j 1 W i j 2 ) 2 absent 1 superscript 𝑛 2 𝑛 subscript 𝑎 𝑛 𝑟 superscript subscript 𝑖 1 𝑛 superscript subscript 𝑊 𝑖 subscript 𝑗 1 subscript 𝑊 𝑖 subscript 𝑗 2 2 \displaystyle=\frac{1}{n^{2}}\frac{na_{n}}{r}\sum_{i=1}^{n}(W_{ij_{1}}W_{ij_{2%
}})^{2} = divide start_ARG 1 end_ARG start_ARG italic_n start_POSTSUPERSCRIPT 2 end_POSTSUPERSCRIPT end_ARG divide start_ARG italic_n italic_a start_POSTSUBSCRIPT italic_n end_POSTSUBSCRIPT end_ARG start_ARG italic_r end_ARG ∑ start_POSTSUBSCRIPT italic_i = 1 end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_n end_POSTSUPERSCRIPT ( italic_W start_POSTSUBSCRIPT italic_i italic_j start_POSTSUBSCRIPT 1 end_POSTSUBSCRIPT end_POSTSUBSCRIPT italic_W start_POSTSUBSCRIPT italic_i italic_j start_POSTSUBSCRIPT 2 end_POSTSUBSCRIPT end_POSTSUBSCRIPT ) start_POSTSUPERSCRIPT 2 end_POSTSUPERSCRIPT
≤ a n r ∑ i = 1 n ‖ 𝐖 i ‖ 4 n = 1 r O P ( 1 ) absent subscript 𝑎 𝑛 𝑟 superscript subscript 𝑖 1 𝑛 superscript norm subscript 𝐖 𝑖 4 𝑛 1 𝑟 subscript 𝑂 𝑃 1 \displaystyle\leq\frac{a_{n}}{r}\sum_{i=1}^{n}\frac{\|\mathbf{W}_{i}\|^{4}}{n}%
=\frac{1}{r}O_{P}(1) ≤ divide start_ARG italic_a start_POSTSUBSCRIPT italic_n end_POSTSUBSCRIPT end_ARG start_ARG italic_r end_ARG ∑ start_POSTSUBSCRIPT italic_i = 1 end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_n end_POSTSUPERSCRIPT divide start_ARG ∥ bold_W start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT ∥ start_POSTSUPERSCRIPT 4 end_POSTSUPERSCRIPT end_ARG start_ARG italic_n end_ARG = divide start_ARG 1 end_ARG start_ARG italic_r end_ARG italic_O start_POSTSUBSCRIPT italic_P end_POSTSUBSCRIPT ( 1 )
= O P ( r − 1 ) . absent subscript 𝑂 𝑃 superscript 𝑟 1 \displaystyle=O_{P}(r^{-1}). = italic_O start_POSTSUBSCRIPT italic_P end_POSTSUBSCRIPT ( italic_r start_POSTSUPERSCRIPT - 1 end_POSTSUPERSCRIPT ) .
From the Chebyshev inequality, for sufficiently large M 𝑀 M italic_M , we have
P ( ‖ ℋ ˇ W − ℋ W ‖ ≥ r − 1 / 2 M | ℱ n ) 𝑃 norm subscript ˇ ℋ 𝑊 subscript ℋ 𝑊 conditional superscript 𝑟 1 2 𝑀 subscript ℱ 𝑛 \displaystyle P(\|\check{\mathcal{H}}_{W}-\mathcal{H}_{W}\|\geq r^{-1/2}M|%
\mathcal{F}_{n}) italic_P ( ∥ overroman_ˇ start_ARG caligraphic_H end_ARG start_POSTSUBSCRIPT italic_W end_POSTSUBSCRIPT - caligraphic_H start_POSTSUBSCRIPT italic_W end_POSTSUBSCRIPT ∥ ≥ italic_r start_POSTSUPERSCRIPT - 1 / 2 end_POSTSUPERSCRIPT italic_M | caligraphic_F start_POSTSUBSCRIPT italic_n end_POSTSUBSCRIPT )
≤ r E ( ‖ ℋ ˇ W ‖ 2 | ℱ n ) M 2 absent 𝑟 𝐸 conditional superscript norm subscript ˇ ℋ 𝑊 2 subscript ℱ 𝑛 superscript 𝑀 2 \displaystyle\leq\frac{rE(\|\check{\mathcal{H}}_{W}\|^{2}|\mathcal{F}_{n})}{M^%
{2}} ≤ divide start_ARG italic_r italic_E ( ∥ overroman_ˇ start_ARG caligraphic_H end_ARG start_POSTSUBSCRIPT italic_W end_POSTSUBSCRIPT ∥ start_POSTSUPERSCRIPT 2 end_POSTSUPERSCRIPT | caligraphic_F start_POSTSUBSCRIPT italic_n end_POSTSUBSCRIPT ) end_ARG start_ARG italic_M start_POSTSUPERSCRIPT 2 end_POSTSUPERSCRIPT end_ARG
= r ∑ j 1 = 1 p ∑ j 2 = 1 p E ( ℋ ˇ W j 1 j 2 | ℱ n ) 2 M 2 absent 𝑟 superscript subscript subscript 𝑗 1 1 𝑝 superscript subscript subscript 𝑗 2 1 𝑝 𝐸 superscript conditional superscript subscript ˇ ℋ 𝑊 subscript 𝑗 1 subscript 𝑗 2 subscript ℱ 𝑛 2 superscript 𝑀 2 \displaystyle=\frac{r\sum_{j_{1}=1}^{p}\sum_{j_{2}=1}^{p}E(\check{\mathcal{H}}%
_{W}^{j_{1}j_{2}}|\mathcal{F}_{n})^{2}}{M^{2}} = divide start_ARG italic_r ∑ start_POSTSUBSCRIPT italic_j start_POSTSUBSCRIPT 1 end_POSTSUBSCRIPT = 1 end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_p end_POSTSUPERSCRIPT ∑ start_POSTSUBSCRIPT italic_j start_POSTSUBSCRIPT 2 end_POSTSUBSCRIPT = 1 end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_p end_POSTSUPERSCRIPT italic_E ( overroman_ˇ start_ARG caligraphic_H end_ARG start_POSTSUBSCRIPT italic_W end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_j start_POSTSUBSCRIPT 1 end_POSTSUBSCRIPT italic_j start_POSTSUBSCRIPT 2 end_POSTSUBSCRIPT end_POSTSUPERSCRIPT | caligraphic_F start_POSTSUBSCRIPT italic_n end_POSTSUBSCRIPT ) start_POSTSUPERSCRIPT 2 end_POSTSUPERSCRIPT end_ARG start_ARG italic_M start_POSTSUPERSCRIPT 2 end_POSTSUPERSCRIPT end_ARG
= O P ( 1 ) M 2 → 0 , n , r → ∞ . formulae-sequence absent subscript 𝑂 𝑃 1 superscript 𝑀 2 → 0 → 𝑛 𝑟
\displaystyle=\frac{O_{P}(1)}{M^{2}}\rightarrow 0,n,r\rightarrow\infty. = divide start_ARG italic_O start_POSTSUBSCRIPT italic_P end_POSTSUBSCRIPT ( 1 ) end_ARG start_ARG italic_M start_POSTSUPERSCRIPT 2 end_POSTSUPERSCRIPT end_ARG → 0 , italic_n , italic_r → ∞ .
Thus, the equation (S6.3 ) is proved.
By (S6.3 ) and Assumption 1, we have ℋ ˇ W − 1 = O P | ℱ n ( 1 ) superscript subscript ˇ ℋ 𝑊 1 subscript 𝑂 conditional 𝑃 subscript ℱ 𝑛 1 \check{\mathcal{H}}_{W}^{-1}=O_{P|\mathcal{F}_{n}}(1) overroman_ˇ start_ARG caligraphic_H end_ARG start_POSTSUBSCRIPT italic_W end_POSTSUBSCRIPT start_POSTSUPERSCRIPT - 1 end_POSTSUPERSCRIPT = italic_O start_POSTSUBSCRIPT italic_P | caligraphic_F start_POSTSUBSCRIPT italic_n end_POSTSUBSCRIPT end_POSTSUBSCRIPT ( 1 ) . Therefore, combining (S6.1 ), (S6.2 ) and (S6.3 ), then
𝜷 ˇ − 𝜷 ^ = O P | ℱ n ( r − 1 / 2 ) . ˇ 𝜷 ^ 𝜷 subscript 𝑂 conditional 𝑃 subscript ℱ 𝑛 superscript 𝑟 1 2 \check{\boldsymbol{\beta}}-\hat{\boldsymbol{\beta}}=O_{P|\mathcal{F}_{n}}(r^{-%
1/2}). overroman_ˇ start_ARG bold_italic_β end_ARG - over^ start_ARG bold_italic_β end_ARG = italic_O start_POSTSUBSCRIPT italic_P | caligraphic_F start_POSTSUBSCRIPT italic_n end_POSTSUBSCRIPT end_POSTSUBSCRIPT ( italic_r start_POSTSUPERSCRIPT - 1 / 2 end_POSTSUPERSCRIPT ) .
(S6.4)
As m > 1 𝑚 1 m>1 italic_m > 1 , we have 𝜷 ˇ ( m ) = 1 m ∑ k = 1 m 𝜷 ˇ k . superscript ˇ 𝜷 𝑚 1 𝑚 superscript subscript 𝑘 1 𝑚 subscript ˇ 𝜷 𝑘 \check{\boldsymbol{\beta}}^{(m)}=\frac{1}{m}\sum_{k=1}^{m}\check{\boldsymbol{%
\beta}}_{k}. overroman_ˇ start_ARG bold_italic_β end_ARG start_POSTSUPERSCRIPT ( italic_m ) end_POSTSUPERSCRIPT = divide start_ARG 1 end_ARG start_ARG italic_m end_ARG ∑ start_POSTSUBSCRIPT italic_k = 1 end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_m end_POSTSUPERSCRIPT overroman_ˇ start_ARG bold_italic_β end_ARG start_POSTSUBSCRIPT italic_k end_POSTSUBSCRIPT .
Then according to the weak law of large numbers, it follows that
𝜷 ˇ ( m ) − 𝜷 ^ = 1 m ∑ k = 1 m 𝜷 ˇ k − 𝜷 ^ = 1 m ∑ k = 1 m ( 𝜷 ˇ k − 𝜷 ^ ) = O P | ℱ n ( ( m r ) − 1 / 2 ) . superscript ˇ 𝜷 𝑚 ^ 𝜷 1 𝑚 superscript subscript 𝑘 1 𝑚 subscript ˇ 𝜷 𝑘 ^ 𝜷 1 𝑚 superscript subscript 𝑘 1 𝑚 subscript ˇ 𝜷 𝑘 ^ 𝜷 subscript 𝑂 conditional 𝑃 subscript ℱ 𝑛 superscript 𝑚 𝑟 1 2 \check{\boldsymbol{\beta}}^{(m)}-\hat{\boldsymbol{\beta}}=\frac{1}{m}\sum_{k=1%
}^{m}\check{\boldsymbol{\beta}}_{k}-\hat{\boldsymbol{\beta}}=\frac{1}{m}\sum_{%
k=1}^{m}(\check{\boldsymbol{\beta}}_{k}-\hat{\boldsymbol{\beta}})=O_{P|%
\mathcal{F}_{n}}((mr)^{-1/2}). overroman_ˇ start_ARG bold_italic_β end_ARG start_POSTSUPERSCRIPT ( italic_m ) end_POSTSUPERSCRIPT - over^ start_ARG bold_italic_β end_ARG = divide start_ARG 1 end_ARG start_ARG italic_m end_ARG ∑ start_POSTSUBSCRIPT italic_k = 1 end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_m end_POSTSUPERSCRIPT overroman_ˇ start_ARG bold_italic_β end_ARG start_POSTSUBSCRIPT italic_k end_POSTSUBSCRIPT - over^ start_ARG bold_italic_β end_ARG = divide start_ARG 1 end_ARG start_ARG italic_m end_ARG ∑ start_POSTSUBSCRIPT italic_k = 1 end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_m end_POSTSUPERSCRIPT ( overroman_ˇ start_ARG bold_italic_β end_ARG start_POSTSUBSCRIPT italic_k end_POSTSUBSCRIPT - over^ start_ARG bold_italic_β end_ARG ) = italic_O start_POSTSUBSCRIPT italic_P | caligraphic_F start_POSTSUBSCRIPT italic_n end_POSTSUBSCRIPT end_POSTSUBSCRIPT ( ( italic_m italic_r ) start_POSTSUPERSCRIPT - 1 / 2 end_POSTSUPERSCRIPT ) .
Then the theorem is proved.
S7 Proof of Theorem 7
Firstly, we prove the case where m = 1 𝑚 1 m=1 italic_m = 1 .
Because
L ˙ * ( 𝜷 ^ ) = 1 n ∑ i = 1 n { ψ i [ − 𝐖 i ( y i − 𝐖 i T 𝜷 ^ ) ] − 𝚺 u u 𝜷 ^ } = 1 r ∑ i = 1 n 𝜼 i , superscript ˙ 𝐿 ^ 𝜷 1 𝑛 superscript subscript 𝑖 1 𝑛 subscript 𝜓 𝑖 delimited-[] subscript 𝐖 𝑖 subscript 𝑦 𝑖 superscript subscript 𝐖 𝑖 𝑇 ^ 𝜷 subscript 𝚺 𝑢 𝑢 ^ 𝜷 1 𝑟 superscript subscript 𝑖 1 𝑛 subscript 𝜼 𝑖 \dot{L}^{*}(\hat{\boldsymbol{\beta}})=\frac{1}{n}\sum_{i=1}^{n}\{\psi_{i}[-%
\mathbf{W}_{i}(y_{i}-\mathbf{W}_{i}^{T}\hat{\boldsymbol{\beta}})]-\boldsymbol{%
\Sigma}_{uu}\hat{\boldsymbol{\beta}}\}=\frac{1}{\sqrt{r}}\sum_{i=1}^{n}%
\boldsymbol{\eta}_{i}, over˙ start_ARG italic_L end_ARG start_POSTSUPERSCRIPT * end_POSTSUPERSCRIPT ( over^ start_ARG bold_italic_β end_ARG ) = divide start_ARG 1 end_ARG start_ARG italic_n end_ARG ∑ start_POSTSUBSCRIPT italic_i = 1 end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_n end_POSTSUPERSCRIPT { italic_ψ start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT [ - bold_W start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT ( italic_y start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT - bold_W start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_T end_POSTSUPERSCRIPT over^ start_ARG bold_italic_β end_ARG ) ] - bold_Σ start_POSTSUBSCRIPT italic_u italic_u end_POSTSUBSCRIPT over^ start_ARG bold_italic_β end_ARG } = divide start_ARG 1 end_ARG start_ARG square-root start_ARG italic_r end_ARG end_ARG ∑ start_POSTSUBSCRIPT italic_i = 1 end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_n end_POSTSUPERSCRIPT bold_italic_η start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT ,
(S7.1)
where 𝜼 i = r n { ψ i [ − 𝐖 i ( y i − 𝐖 i T 𝜷 ^ ) ] − 𝚺 u u 𝜷 ^ } subscript 𝜼 𝑖 𝑟 𝑛 subscript 𝜓 𝑖 delimited-[] subscript 𝐖 𝑖 subscript 𝑦 𝑖 superscript subscript 𝐖 𝑖 𝑇 ^ 𝜷 subscript 𝚺 𝑢 𝑢 ^ 𝜷 \boldsymbol{\eta}_{i}=\frac{\sqrt{r}}{n}\{\psi_{i}[-\mathbf{W}_{i}(y_{i}-%
\mathbf{W}_{i}^{T}\hat{\boldsymbol{\beta}})]-\boldsymbol{\Sigma}_{uu}\hat{%
\boldsymbol{\beta}}\} bold_italic_η start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT = divide start_ARG square-root start_ARG italic_r end_ARG end_ARG start_ARG italic_n end_ARG { italic_ψ start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT [ - bold_W start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT ( italic_y start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT - bold_W start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_T end_POSTSUPERSCRIPT over^ start_ARG bold_italic_β end_ARG ) ] - bold_Σ start_POSTSUBSCRIPT italic_u italic_u end_POSTSUBSCRIPT over^ start_ARG bold_italic_β end_ARG } is an independent random vector.
Note that
E ( 𝜼 i | ℱ n ) 𝐸 conditional subscript 𝜼 𝑖 subscript ℱ 𝑛 \displaystyle E(\boldsymbol{\eta}_{i}|\mathcal{F}_{n}) italic_E ( bold_italic_η start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT | caligraphic_F start_POSTSUBSCRIPT italic_n end_POSTSUBSCRIPT )
= r n [ − 𝐖 i ( y i − 𝐖 i T 𝜷 ^ ) − 𝚺 u u 𝜷 ^ ] , absent 𝑟 𝑛 delimited-[] subscript 𝐖 𝑖 subscript 𝑦 𝑖 superscript subscript 𝐖 𝑖 𝑇 ^ 𝜷 subscript 𝚺 𝑢 𝑢 ^ 𝜷 \displaystyle=\frac{\sqrt{r}}{n}[-\mathbf{W}_{i}(y_{i}-\mathbf{W}_{i}^{T}\hat{%
\boldsymbol{\beta}})-\boldsymbol{\Sigma}_{uu}\hat{\boldsymbol{\beta}}], = divide start_ARG square-root start_ARG italic_r end_ARG end_ARG start_ARG italic_n end_ARG [ - bold_W start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT ( italic_y start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT - bold_W start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_T end_POSTSUPERSCRIPT over^ start_ARG bold_italic_β end_ARG ) - bold_Σ start_POSTSUBSCRIPT italic_u italic_u end_POSTSUBSCRIPT over^ start_ARG bold_italic_β end_ARG ] ,
V a r ( 𝜼 i | ℱ n ) 𝑉 𝑎 𝑟 conditional subscript 𝜼 𝑖 subscript ℱ 𝑛 \displaystyle Var(\boldsymbol{\eta}_{i}|\mathcal{F}_{n}) italic_V italic_a italic_r ( bold_italic_η start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT | caligraphic_F start_POSTSUBSCRIPT italic_n end_POSTSUBSCRIPT )
= r n 2 n a n r [ − 𝐖 i ( y i − 𝐖 i T 𝜷 ^ ) ] ⊗ 2 = a n n 𝐖 i 𝐖 i T ( y i − 𝐖 i T 𝜷 ^ ) 2 . absent 𝑟 superscript 𝑛 2 𝑛 subscript 𝑎 𝑛 𝑟 superscript delimited-[] subscript 𝐖 𝑖 subscript 𝑦 𝑖 superscript subscript 𝐖 𝑖 𝑇 ^ 𝜷 tensor-product absent 2 subscript 𝑎 𝑛 𝑛 subscript 𝐖 𝑖 superscript subscript 𝐖 𝑖 𝑇 superscript subscript 𝑦 𝑖 superscript subscript 𝐖 𝑖 𝑇 ^ 𝜷 2 \displaystyle=\frac{r}{n^{2}}\frac{na_{n}}{r}[-\mathbf{W}_{i}(y_{i}-\mathbf{W}%
_{i}^{T}\hat{\boldsymbol{\beta}})]^{\otimes 2}=\frac{a_{n}}{n}\mathbf{W}_{i}%
\mathbf{W}_{i}^{T}(y_{i}-\mathbf{W}_{i}^{T}\hat{\boldsymbol{\beta}})^{2}. = divide start_ARG italic_r end_ARG start_ARG italic_n start_POSTSUPERSCRIPT 2 end_POSTSUPERSCRIPT end_ARG divide start_ARG italic_n italic_a start_POSTSUBSCRIPT italic_n end_POSTSUBSCRIPT end_ARG start_ARG italic_r end_ARG [ - bold_W start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT ( italic_y start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT - bold_W start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_T end_POSTSUPERSCRIPT over^ start_ARG bold_italic_β end_ARG ) ] start_POSTSUPERSCRIPT ⊗ 2 end_POSTSUPERSCRIPT = divide start_ARG italic_a start_POSTSUBSCRIPT italic_n end_POSTSUBSCRIPT end_ARG start_ARG italic_n end_ARG bold_W start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT bold_W start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_T end_POSTSUPERSCRIPT ( italic_y start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT - bold_W start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_T end_POSTSUPERSCRIPT over^ start_ARG bold_italic_β end_ARG ) start_POSTSUPERSCRIPT 2 end_POSTSUPERSCRIPT .
Then by using Assumptions 2 and 5, we obtain
∑ i = 1 n E ( 𝜼 i | ℱ n ) = r n ∑ i = 1 n [ − 𝐖 i ( y i − 𝐖 i T 𝜷 ^ ) − 𝚺 u u 𝜷 ^ ] = 𝟎 , superscript subscript 𝑖 1 𝑛 𝐸 conditional subscript 𝜼 𝑖 subscript ℱ 𝑛 𝑟 𝑛 superscript subscript 𝑖 1 𝑛 delimited-[] subscript 𝐖 𝑖 subscript 𝑦 𝑖 superscript subscript 𝐖 𝑖 𝑇 ^ 𝜷 subscript 𝚺 𝑢 𝑢 ^ 𝜷 0 \sum_{i=1}^{n}E(\boldsymbol{\eta}_{i}|\mathcal{F}_{n})=\frac{\sqrt{r}}{n}\sum_%
{i=1}^{n}[-\mathbf{W}_{i}(y_{i}-\mathbf{W}_{i}^{T}\hat{\boldsymbol{\beta}})-%
\boldsymbol{\Sigma}_{uu}\hat{\boldsymbol{\beta}}]=\mathbf{0}, ∑ start_POSTSUBSCRIPT italic_i = 1 end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_n end_POSTSUPERSCRIPT italic_E ( bold_italic_η start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT | caligraphic_F start_POSTSUBSCRIPT italic_n end_POSTSUBSCRIPT ) = divide start_ARG square-root start_ARG italic_r end_ARG end_ARG start_ARG italic_n end_ARG ∑ start_POSTSUBSCRIPT italic_i = 1 end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_n end_POSTSUPERSCRIPT [ - bold_W start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT ( italic_y start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT - bold_W start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_T end_POSTSUPERSCRIPT over^ start_ARG bold_italic_β end_ARG ) - bold_Σ start_POSTSUBSCRIPT italic_u italic_u end_POSTSUBSCRIPT over^ start_ARG bold_italic_β end_ARG ] = bold_0 ,
∑ i = 1 n V a r ( 𝜼 i | ℱ n ) = a n n ∑ i = 1 n 𝐖 i 𝐖 i T ( y i − 𝐖 i T 𝜷 ^ ) 2 = a n Σ c . superscript subscript 𝑖 1 𝑛 𝑉 𝑎 𝑟 conditional subscript 𝜼 𝑖 subscript ℱ 𝑛 subscript 𝑎 𝑛 𝑛 superscript subscript 𝑖 1 𝑛 subscript 𝐖 𝑖 superscript subscript 𝐖 𝑖 𝑇 superscript subscript 𝑦 𝑖 superscript subscript 𝐖 𝑖 𝑇 ^ 𝜷 2 subscript 𝑎 𝑛 subscript Σ 𝑐 \sum_{i=1}^{n}Var(\boldsymbol{\eta}_{i}|\mathcal{F}_{n})=\frac{a_{n}}{n}\sum_{%
i=1}^{n}\mathbf{W}_{i}\mathbf{W}_{i}^{T}(y_{i}-\mathbf{W}_{i}^{T}\hat{%
\boldsymbol{\beta}})^{2}=a_{n}\Sigma_{c}. ∑ start_POSTSUBSCRIPT italic_i = 1 end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_n end_POSTSUPERSCRIPT italic_V italic_a italic_r ( bold_italic_η start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT | caligraphic_F start_POSTSUBSCRIPT italic_n end_POSTSUBSCRIPT ) = divide start_ARG italic_a start_POSTSUBSCRIPT italic_n end_POSTSUBSCRIPT end_ARG start_ARG italic_n end_ARG ∑ start_POSTSUBSCRIPT italic_i = 1 end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_n end_POSTSUPERSCRIPT bold_W start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT bold_W start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_T end_POSTSUPERSCRIPT ( italic_y start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT - bold_W start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_T end_POSTSUPERSCRIPT over^ start_ARG bold_italic_β end_ARG ) start_POSTSUPERSCRIPT 2 end_POSTSUPERSCRIPT = italic_a start_POSTSUBSCRIPT italic_n end_POSTSUBSCRIPT roman_Σ start_POSTSUBSCRIPT italic_c end_POSTSUBSCRIPT .
(S7.2)
According to the C r subscript 𝐶 𝑟 C_{r} italic_C start_POSTSUBSCRIPT italic_r end_POSTSUBSCRIPT inequality, Assumptions 4 and 5, we have
∑ i = 1 n E { ‖ 𝜼 i ‖ 2 I ( ‖ 𝜼 i ‖ > ε ) | ℱ n } superscript subscript 𝑖 1 𝑛 𝐸 conditional superscript norm subscript 𝜼 𝑖 2 𝐼 norm subscript 𝜼 𝑖 𝜀 subscript ℱ 𝑛 \displaystyle\sum_{i=1}^{n}E\{\|\boldsymbol{\eta}_{i}\|^{2}I(\|\boldsymbol{%
\eta}_{i}\|>\varepsilon)|\mathcal{F}_{n}\} ∑ start_POSTSUBSCRIPT italic_i = 1 end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_n end_POSTSUPERSCRIPT italic_E { ∥ bold_italic_η start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT ∥ start_POSTSUPERSCRIPT 2 end_POSTSUPERSCRIPT italic_I ( ∥ bold_italic_η start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT ∥ > italic_ε ) | caligraphic_F start_POSTSUBSCRIPT italic_n end_POSTSUBSCRIPT }
≤ \displaystyle\leq ≤
ε − α ∑ i = 1 n E { ‖ 𝜼 i ‖ 2 + α | ℱ n } superscript 𝜀 𝛼 superscript subscript 𝑖 1 𝑛 𝐸 conditional superscript norm subscript 𝜼 𝑖 2 𝛼 subscript ℱ 𝑛 \displaystyle\varepsilon^{-\alpha}\sum_{i=1}^{n}E\{\|\boldsymbol{\eta}_{i}\|^{%
2+\alpha}|\mathcal{F}_{n}\} italic_ε start_POSTSUPERSCRIPT - italic_α end_POSTSUPERSCRIPT ∑ start_POSTSUBSCRIPT italic_i = 1 end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_n end_POSTSUPERSCRIPT italic_E { ∥ bold_italic_η start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT ∥ start_POSTSUPERSCRIPT 2 + italic_α end_POSTSUPERSCRIPT | caligraphic_F start_POSTSUBSCRIPT italic_n end_POSTSUBSCRIPT }
= \displaystyle= =
ε − α ∑ i = 1 n E { ‖ r n { ψ i [ − 𝐖 i ( y i − 𝐖 i T 𝜷 ^ ) ] − 𝚺 u u 𝜷 ^ } ‖ 2 + α | ℱ n } superscript 𝜀 𝛼 superscript subscript 𝑖 1 𝑛 𝐸 conditional superscript norm 𝑟 𝑛 subscript 𝜓 𝑖 delimited-[] subscript 𝐖 𝑖 subscript 𝑦 𝑖 superscript subscript 𝐖 𝑖 𝑇 ^ 𝜷 subscript 𝚺 𝑢 𝑢 ^ 𝜷 2 𝛼 subscript ℱ 𝑛 \displaystyle\varepsilon^{-\alpha}\sum_{i=1}^{n}E\left\{\left\|\frac{\sqrt{r}}%
{n}\{\psi_{i}[-\mathbf{W}_{i}(y_{i}-\mathbf{W}_{i}^{T}\hat{\boldsymbol{\beta}}%
)]-\boldsymbol{\Sigma}_{uu}\hat{\boldsymbol{\beta}}\}\right\|^{2+\alpha}\bigg{%
|}\mathcal{F}_{n}\right\} italic_ε start_POSTSUPERSCRIPT - italic_α end_POSTSUPERSCRIPT ∑ start_POSTSUBSCRIPT italic_i = 1 end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_n end_POSTSUPERSCRIPT italic_E { ∥ divide start_ARG square-root start_ARG italic_r end_ARG end_ARG start_ARG italic_n end_ARG { italic_ψ start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT [ - bold_W start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT ( italic_y start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT - bold_W start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_T end_POSTSUPERSCRIPT over^ start_ARG bold_italic_β end_ARG ) ] - bold_Σ start_POSTSUBSCRIPT italic_u italic_u end_POSTSUBSCRIPT over^ start_ARG bold_italic_β end_ARG } ∥ start_POSTSUPERSCRIPT 2 + italic_α end_POSTSUPERSCRIPT | caligraphic_F start_POSTSUBSCRIPT italic_n end_POSTSUBSCRIPT }
= \displaystyle= =
r 1 + α 2 ε α n 2 + α ∑ i = 1 n E { ‖ ψ i 𝐖 i ( y i − 𝐖 i T 𝜷 ^ ) + 𝚺 u u 𝜷 ^ ‖ 2 + α | ℱ n } superscript 𝑟 1 𝛼 2 superscript 𝜀 𝛼 superscript 𝑛 2 𝛼 superscript subscript 𝑖 1 𝑛 𝐸 conditional superscript norm subscript 𝜓 𝑖 subscript 𝐖 𝑖 subscript 𝑦 𝑖 superscript subscript 𝐖 𝑖 𝑇 ^ 𝜷 subscript 𝚺 𝑢 𝑢 ^ 𝜷 2 𝛼 subscript ℱ 𝑛 \displaystyle\frac{r^{1+\frac{\alpha}{2}}}{\varepsilon^{\alpha}n^{2+\alpha}}%
\sum_{i=1}^{n}E\{\|\psi_{i}\mathbf{W}_{i}(y_{i}-\mathbf{W}_{i}^{T}\hat{%
\boldsymbol{\beta}})+\boldsymbol{\Sigma}_{uu}\hat{\boldsymbol{\beta}}\|^{2+%
\alpha}|\mathcal{F}_{n}\} divide start_ARG italic_r start_POSTSUPERSCRIPT 1 + divide start_ARG italic_α end_ARG start_ARG 2 end_ARG end_POSTSUPERSCRIPT end_ARG start_ARG italic_ε start_POSTSUPERSCRIPT italic_α end_POSTSUPERSCRIPT italic_n start_POSTSUPERSCRIPT 2 + italic_α end_POSTSUPERSCRIPT end_ARG ∑ start_POSTSUBSCRIPT italic_i = 1 end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_n end_POSTSUPERSCRIPT italic_E { ∥ italic_ψ start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT bold_W start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT ( italic_y start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT - bold_W start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_T end_POSTSUPERSCRIPT over^ start_ARG bold_italic_β end_ARG ) + bold_Σ start_POSTSUBSCRIPT italic_u italic_u end_POSTSUBSCRIPT over^ start_ARG bold_italic_β end_ARG ∥ start_POSTSUPERSCRIPT 2 + italic_α end_POSTSUPERSCRIPT | caligraphic_F start_POSTSUBSCRIPT italic_n end_POSTSUBSCRIPT }
≤ \displaystyle\leq ≤
r 1 + α 2 2 1 + α ε α n 2 + α ∑ i = 1 n { E [ ‖ ψ i 𝐖 i ( y i − 𝐖 i T 𝜷 ^ ) ‖ 2 + α | ℱ n ] + E [ ‖ 𝚺 u u 𝜷 ^ ‖ 2 + α | ℱ n ] } superscript 𝑟 1 𝛼 2 superscript 2 1 𝛼 superscript 𝜀 𝛼 superscript 𝑛 2 𝛼 superscript subscript 𝑖 1 𝑛 𝐸 delimited-[] conditional superscript norm subscript 𝜓 𝑖 subscript 𝐖 𝑖 subscript 𝑦 𝑖 superscript subscript 𝐖 𝑖 𝑇 ^ 𝜷 2 𝛼 subscript ℱ 𝑛 𝐸 delimited-[] conditional superscript norm subscript 𝚺 𝑢 𝑢 ^ 𝜷 2 𝛼 subscript ℱ 𝑛 \displaystyle\frac{r^{1+\frac{\alpha}{2}}2^{1+\alpha}}{\varepsilon^{\alpha}n^{%
2+\alpha}}\sum_{i=1}^{n}\{E[\|\psi_{i}\mathbf{W}_{i}(y_{i}-\mathbf{W}_{i}^{T}%
\hat{\boldsymbol{\beta}})\|^{2+\alpha}|\mathcal{F}_{n}]+E[\|\boldsymbol{\Sigma%
}_{uu}\hat{\boldsymbol{\beta}}\|^{2+\alpha}|\mathcal{F}_{n}]\} divide start_ARG italic_r start_POSTSUPERSCRIPT 1 + divide start_ARG italic_α end_ARG start_ARG 2 end_ARG end_POSTSUPERSCRIPT 2 start_POSTSUPERSCRIPT 1 + italic_α end_POSTSUPERSCRIPT end_ARG start_ARG italic_ε start_POSTSUPERSCRIPT italic_α end_POSTSUPERSCRIPT italic_n start_POSTSUPERSCRIPT 2 + italic_α end_POSTSUPERSCRIPT end_ARG ∑ start_POSTSUBSCRIPT italic_i = 1 end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_n end_POSTSUPERSCRIPT { italic_E [ ∥ italic_ψ start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT bold_W start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT ( italic_y start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT - bold_W start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_T end_POSTSUPERSCRIPT over^ start_ARG bold_italic_β end_ARG ) ∥ start_POSTSUPERSCRIPT 2 + italic_α end_POSTSUPERSCRIPT | caligraphic_F start_POSTSUBSCRIPT italic_n end_POSTSUBSCRIPT ] + italic_E [ ∥ bold_Σ start_POSTSUBSCRIPT italic_u italic_u end_POSTSUBSCRIPT over^ start_ARG bold_italic_β end_ARG ∥ start_POSTSUPERSCRIPT 2 + italic_α end_POSTSUPERSCRIPT | caligraphic_F start_POSTSUBSCRIPT italic_n end_POSTSUBSCRIPT ] }
= \displaystyle= =
r 1 + α 2 2 1 + α ε α n 1 + α { E ( ψ ) 2 + α 1 n ∑ i = 1 n ‖ 𝐖 i ‖ 2 + α ( y i − 𝐖 i T 𝜷 ^ ) 2 + α + ‖ 𝚺 u u 𝜷 ^ ‖ 2 + α } superscript 𝑟 1 𝛼 2 superscript 2 1 𝛼 superscript 𝜀 𝛼 superscript 𝑛 1 𝛼 𝐸 superscript 𝜓 2 𝛼 1 𝑛 superscript subscript 𝑖 1 𝑛 superscript norm subscript 𝐖 𝑖 2 𝛼 superscript subscript 𝑦 𝑖 superscript subscript 𝐖 𝑖 𝑇 ^ 𝜷 2 𝛼 superscript norm subscript 𝚺 𝑢 𝑢 ^ 𝜷 2 𝛼 \displaystyle\frac{r^{1+\frac{\alpha}{2}}2^{1+\alpha}}{\varepsilon^{\alpha}n^{%
1+\alpha}}\left\{E(\psi)^{2+\alpha}\frac{1}{n}\sum_{i=1}^{n}\|\mathbf{W}_{i}\|%
^{2+\alpha}(y_{i}-\mathbf{W}_{i}^{T}\hat{\boldsymbol{\beta}})^{2+\alpha}+\|%
\boldsymbol{\Sigma}_{uu}\hat{\boldsymbol{\beta}}\|^{2+\alpha}\right\} divide start_ARG italic_r start_POSTSUPERSCRIPT 1 + divide start_ARG italic_α end_ARG start_ARG 2 end_ARG end_POSTSUPERSCRIPT 2 start_POSTSUPERSCRIPT 1 + italic_α end_POSTSUPERSCRIPT end_ARG start_ARG italic_ε start_POSTSUPERSCRIPT italic_α end_POSTSUPERSCRIPT italic_n start_POSTSUPERSCRIPT 1 + italic_α end_POSTSUPERSCRIPT end_ARG { italic_E ( italic_ψ ) start_POSTSUPERSCRIPT 2 + italic_α end_POSTSUPERSCRIPT divide start_ARG 1 end_ARG start_ARG italic_n end_ARG ∑ start_POSTSUBSCRIPT italic_i = 1 end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_n end_POSTSUPERSCRIPT ∥ bold_W start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT ∥ start_POSTSUPERSCRIPT 2 + italic_α end_POSTSUPERSCRIPT ( italic_y start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT - bold_W start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_T end_POSTSUPERSCRIPT over^ start_ARG bold_italic_β end_ARG ) start_POSTSUPERSCRIPT 2 + italic_α end_POSTSUPERSCRIPT + ∥ bold_Σ start_POSTSUBSCRIPT italic_u italic_u end_POSTSUBSCRIPT over^ start_ARG bold_italic_β end_ARG ∥ start_POSTSUPERSCRIPT 2 + italic_α end_POSTSUPERSCRIPT }
= \displaystyle= =
O P ( r − α / 2 ) . subscript 𝑂 𝑃 superscript 𝑟 𝛼 2 \displaystyle O_{P}(r^{-\alpha/2}). italic_O start_POSTSUBSCRIPT italic_P end_POSTSUBSCRIPT ( italic_r start_POSTSUPERSCRIPT - italic_α / 2 end_POSTSUPERSCRIPT ) .
Therefore, the Lindeberg-Feller condition is satisfied. According to the Lindeberg-Feller central limit theorem, we have
( ∑ i = 1 n V a r ( 𝜼 i | ℱ n ) ) − 1 / 2 ∑ i = 1 n 𝜼 i = r a n Σ c − 1 / 2 L ˙ * ( 𝜷 ^ ) → 𝑑 N p ( 𝟎 , I ) . superscript superscript subscript 𝑖 1 𝑛 𝑉 𝑎 𝑟 conditional subscript 𝜼 𝑖 subscript ℱ 𝑛 1 2 superscript subscript 𝑖 1 𝑛 subscript 𝜼 𝑖 𝑟 subscript 𝑎 𝑛 superscript subscript Σ 𝑐 1 2 superscript ˙ 𝐿 ^ 𝜷 𝑑 → subscript 𝑁 𝑝 0 𝐼 \left(\sum_{i=1}^{n}Var(\boldsymbol{\eta}_{i}|\mathcal{F}_{n})\right)^{-1/2}%
\sum_{i=1}^{n}\boldsymbol{\eta}_{i}=\sqrt{\frac{r}{a_{n}}}\Sigma_{c}^{-1/2}%
\dot{L}^{*}(\hat{\boldsymbol{\beta}})\xrightarrow{d}N_{p}(\mathbf{0},I). ( ∑ start_POSTSUBSCRIPT italic_i = 1 end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_n end_POSTSUPERSCRIPT italic_V italic_a italic_r ( bold_italic_η start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT | caligraphic_F start_POSTSUBSCRIPT italic_n end_POSTSUBSCRIPT ) ) start_POSTSUPERSCRIPT - 1 / 2 end_POSTSUPERSCRIPT ∑ start_POSTSUBSCRIPT italic_i = 1 end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_n end_POSTSUPERSCRIPT bold_italic_η start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT = square-root start_ARG divide start_ARG italic_r end_ARG start_ARG italic_a start_POSTSUBSCRIPT italic_n end_POSTSUBSCRIPT end_ARG end_ARG roman_Σ start_POSTSUBSCRIPT italic_c end_POSTSUBSCRIPT start_POSTSUPERSCRIPT - 1 / 2 end_POSTSUPERSCRIPT over˙ start_ARG italic_L end_ARG start_POSTSUPERSCRIPT * end_POSTSUPERSCRIPT ( over^ start_ARG bold_italic_β end_ARG ) start_ARROW overitalic_d → end_ARROW italic_N start_POSTSUBSCRIPT italic_p end_POSTSUBSCRIPT ( bold_0 , italic_I ) .
(S7.3)
By (S6.3 ), we have
ℋ ˇ W − 1 − ℋ W − 1 = − ℋ W − 1 ( ℋ ˇ W − ℋ W ) ℋ ˇ W − 1 = O P | ℱ n ( r − 1 / 2 ) , superscript subscript ˇ ℋ 𝑊 1 superscript subscript ℋ 𝑊 1 superscript subscript ℋ 𝑊 1 subscript ˇ ℋ 𝑊 subscript ℋ 𝑊 superscript subscript ˇ ℋ 𝑊 1 subscript 𝑂 conditional 𝑃 subscript ℱ 𝑛 superscript 𝑟 1 2 \check{\mathcal{H}}_{W}^{-1}-\mathcal{H}_{W}^{-1}=-\mathcal{H}_{W}^{-1}(\check%
{\mathcal{H}}_{W}-\mathcal{H}_{W})\check{\mathcal{H}}_{W}^{-1}=O_{P|\mathcal{F%
}_{n}}(r^{-1/2}), overroman_ˇ start_ARG caligraphic_H end_ARG start_POSTSUBSCRIPT italic_W end_POSTSUBSCRIPT start_POSTSUPERSCRIPT - 1 end_POSTSUPERSCRIPT - caligraphic_H start_POSTSUBSCRIPT italic_W end_POSTSUBSCRIPT start_POSTSUPERSCRIPT - 1 end_POSTSUPERSCRIPT = - caligraphic_H start_POSTSUBSCRIPT italic_W end_POSTSUBSCRIPT start_POSTSUPERSCRIPT - 1 end_POSTSUPERSCRIPT ( overroman_ˇ start_ARG caligraphic_H end_ARG start_POSTSUBSCRIPT italic_W end_POSTSUBSCRIPT - caligraphic_H start_POSTSUBSCRIPT italic_W end_POSTSUBSCRIPT ) overroman_ˇ start_ARG caligraphic_H end_ARG start_POSTSUBSCRIPT italic_W end_POSTSUBSCRIPT start_POSTSUPERSCRIPT - 1 end_POSTSUPERSCRIPT = italic_O start_POSTSUBSCRIPT italic_P | caligraphic_F start_POSTSUBSCRIPT italic_n end_POSTSUBSCRIPT end_POSTSUBSCRIPT ( italic_r start_POSTSUPERSCRIPT - 1 / 2 end_POSTSUPERSCRIPT ) ,
(S7.4)
By Assumption 1, ℋ W subscript ℋ 𝑊 \mathcal{H}_{W} caligraphic_H start_POSTSUBSCRIPT italic_W end_POSTSUBSCRIPT converges to a positive definite matrix, then ℋ W − 1 = O P ( 1 ) superscript subscript ℋ 𝑊 1 subscript 𝑂 𝑃 1 \mathcal{H}_{W}^{-1}=O_{P}(1) caligraphic_H start_POSTSUBSCRIPT italic_W end_POSTSUBSCRIPT start_POSTSUPERSCRIPT - 1 end_POSTSUPERSCRIPT = italic_O start_POSTSUBSCRIPT italic_P end_POSTSUBSCRIPT ( 1 ) .
And due to (S7.2 ), we obtain
Σ = ℋ W − 1 Σ c ℋ W − 1 = O P ( 1 ) . Σ superscript subscript ℋ 𝑊 1 subscript Σ 𝑐 superscript subscript ℋ 𝑊 1 subscript 𝑂 𝑃 1 \Sigma=\mathcal{H}_{W}^{-1}\Sigma_{c}\mathcal{H}_{W}^{-1}=O_{P}(1). roman_Σ = caligraphic_H start_POSTSUBSCRIPT italic_W end_POSTSUBSCRIPT start_POSTSUPERSCRIPT - 1 end_POSTSUPERSCRIPT roman_Σ start_POSTSUBSCRIPT italic_c end_POSTSUBSCRIPT caligraphic_H start_POSTSUBSCRIPT italic_W end_POSTSUBSCRIPT start_POSTSUPERSCRIPT - 1 end_POSTSUPERSCRIPT = italic_O start_POSTSUBSCRIPT italic_P end_POSTSUBSCRIPT ( 1 ) .
(S7.5)
Therefore, combining(S6.1 ), (S7.4 ) and (S7.5 ), we have
r a n Σ − 1 / 2 ( 𝜷 ˇ − 𝜷 ^ ) 𝑟 subscript 𝑎 𝑛 superscript Σ 1 2 ˇ 𝜷 ^ 𝜷 \displaystyle\sqrt{\frac{r}{a_{n}}}\Sigma^{-1/2}(\check{\boldsymbol{\beta}}-%
\hat{\boldsymbol{\beta}}) square-root start_ARG divide start_ARG italic_r end_ARG start_ARG italic_a start_POSTSUBSCRIPT italic_n end_POSTSUBSCRIPT end_ARG end_ARG roman_Σ start_POSTSUPERSCRIPT - 1 / 2 end_POSTSUPERSCRIPT ( overroman_ˇ start_ARG bold_italic_β end_ARG - over^ start_ARG bold_italic_β end_ARG )
= − r a n Σ − 1 / 2 ℋ ˇ W − 1 L ˙ * ( 𝜷 ^ ) absent 𝑟 subscript 𝑎 𝑛 superscript Σ 1 2 superscript subscript ˇ ℋ 𝑊 1 superscript ˙ 𝐿 ^ 𝜷 \displaystyle=-\sqrt{\frac{r}{a_{n}}}\Sigma^{-1/2}\check{\mathcal{H}}_{W}^{-1}%
\dot{L}^{*}(\hat{\boldsymbol{\beta}}) = - square-root start_ARG divide start_ARG italic_r end_ARG start_ARG italic_a start_POSTSUBSCRIPT italic_n end_POSTSUBSCRIPT end_ARG end_ARG roman_Σ start_POSTSUPERSCRIPT - 1 / 2 end_POSTSUPERSCRIPT overroman_ˇ start_ARG caligraphic_H end_ARG start_POSTSUBSCRIPT italic_W end_POSTSUBSCRIPT start_POSTSUPERSCRIPT - 1 end_POSTSUPERSCRIPT over˙ start_ARG italic_L end_ARG start_POSTSUPERSCRIPT * end_POSTSUPERSCRIPT ( over^ start_ARG bold_italic_β end_ARG )
= − r a n Σ − 1 / 2 ℋ W − 1 L ˙ * ( 𝜷 ^ ) − r a n Σ − 1 / 2 ( ℋ ˇ W − 1 − ℋ W − 1 ) L ˙ * ( 𝜷 ^ ) absent 𝑟 subscript 𝑎 𝑛 superscript Σ 1 2 superscript subscript ℋ 𝑊 1 superscript ˙ 𝐿 ^ 𝜷 𝑟 subscript 𝑎 𝑛 superscript Σ 1 2 superscript subscript ˇ ℋ 𝑊 1 superscript subscript ℋ 𝑊 1 superscript ˙ 𝐿 ^ 𝜷 \displaystyle=-\sqrt{\frac{r}{a_{n}}}\Sigma^{-1/2}\mathcal{H}_{W}^{-1}\dot{L}^%
{*}(\hat{\boldsymbol{\beta}})-\sqrt{\frac{r}{a_{n}}}\Sigma^{-1/2}(\check{%
\mathcal{H}}_{W}^{-1}-\mathcal{H}_{W}^{-1})\dot{L}^{*}(\hat{\boldsymbol{\beta}}) = - square-root start_ARG divide start_ARG italic_r end_ARG start_ARG italic_a start_POSTSUBSCRIPT italic_n end_POSTSUBSCRIPT end_ARG end_ARG roman_Σ start_POSTSUPERSCRIPT - 1 / 2 end_POSTSUPERSCRIPT caligraphic_H start_POSTSUBSCRIPT italic_W end_POSTSUBSCRIPT start_POSTSUPERSCRIPT - 1 end_POSTSUPERSCRIPT over˙ start_ARG italic_L end_ARG start_POSTSUPERSCRIPT * end_POSTSUPERSCRIPT ( over^ start_ARG bold_italic_β end_ARG ) - square-root start_ARG divide start_ARG italic_r end_ARG start_ARG italic_a start_POSTSUBSCRIPT italic_n end_POSTSUBSCRIPT end_ARG end_ARG roman_Σ start_POSTSUPERSCRIPT - 1 / 2 end_POSTSUPERSCRIPT ( overroman_ˇ start_ARG caligraphic_H end_ARG start_POSTSUBSCRIPT italic_W end_POSTSUBSCRIPT start_POSTSUPERSCRIPT - 1 end_POSTSUPERSCRIPT - caligraphic_H start_POSTSUBSCRIPT italic_W end_POSTSUBSCRIPT start_POSTSUPERSCRIPT - 1 end_POSTSUPERSCRIPT ) over˙ start_ARG italic_L end_ARG start_POSTSUPERSCRIPT * end_POSTSUPERSCRIPT ( over^ start_ARG bold_italic_β end_ARG )
= − r a n Σ − 1 / 2 ℋ W − 1 Σ c 1 / 2 Σ c − 1 / 2 L ˙ * ( 𝜷 ^ ) + O P | ℱ n ( r − 1 / 2 ) . absent 𝑟 subscript 𝑎 𝑛 superscript Σ 1 2 superscript subscript ℋ 𝑊 1 superscript subscript Σ 𝑐 1 2 superscript subscript Σ 𝑐 1 2 superscript ˙ 𝐿 ^ 𝜷 subscript 𝑂 conditional 𝑃 subscript ℱ 𝑛 superscript 𝑟 1 2 \displaystyle=-\sqrt{\frac{r}{a_{n}}}{\Sigma}^{-1/2}\mathcal{H}_{W}^{-1}\Sigma%
_{c}^{1/2}\Sigma_{c}^{-1/2}\dot{L}^{*}(\hat{\boldsymbol{\beta}})+O_{P|\mathcal%
{F}_{n}}(r^{-1/2}). = - square-root start_ARG divide start_ARG italic_r end_ARG start_ARG italic_a start_POSTSUBSCRIPT italic_n end_POSTSUBSCRIPT end_ARG end_ARG roman_Σ start_POSTSUPERSCRIPT - 1 / 2 end_POSTSUPERSCRIPT caligraphic_H start_POSTSUBSCRIPT italic_W end_POSTSUBSCRIPT start_POSTSUPERSCRIPT - 1 end_POSTSUPERSCRIPT roman_Σ start_POSTSUBSCRIPT italic_c end_POSTSUBSCRIPT start_POSTSUPERSCRIPT 1 / 2 end_POSTSUPERSCRIPT roman_Σ start_POSTSUBSCRIPT italic_c end_POSTSUBSCRIPT start_POSTSUPERSCRIPT - 1 / 2 end_POSTSUPERSCRIPT over˙ start_ARG italic_L end_ARG start_POSTSUPERSCRIPT * end_POSTSUPERSCRIPT ( over^ start_ARG bold_italic_β end_ARG ) + italic_O start_POSTSUBSCRIPT italic_P | caligraphic_F start_POSTSUBSCRIPT italic_n end_POSTSUBSCRIPT end_POSTSUBSCRIPT ( italic_r start_POSTSUPERSCRIPT - 1 / 2 end_POSTSUPERSCRIPT ) .
Note that
Σ − 1 / 2 ℋ W − 1 Σ c 1 / 2 ( Σ − 1 / 2 ℋ W − 1 Σ c 1 / 2 ) T = Σ − 1 / 2 ℋ W − 1 Σ c 1 / 2 Σ c 1 / 2 ℋ W − 1 Σ − 1 / 2 = I , superscript Σ 1 2 superscript subscript ℋ 𝑊 1 superscript subscript Σ 𝑐 1 2 superscript superscript Σ 1 2 superscript subscript ℋ 𝑊 1 superscript subscript Σ 𝑐 1 2 𝑇 superscript Σ 1 2 superscript subscript ℋ 𝑊 1 superscript subscript Σ 𝑐 1 2 superscript subscript Σ 𝑐 1 2 superscript subscript ℋ 𝑊 1 superscript Σ 1 2 𝐼 \Sigma^{-1/2}\mathcal{H}_{W}^{-1}\Sigma_{c}^{1/2}(\Sigma^{-1/2}\mathcal{H}_{W}%
^{-1}\Sigma_{c}^{1/2})^{T}=\Sigma^{-1/2}\mathcal{H}_{W}^{-1}\Sigma_{c}^{1/2}%
\Sigma_{c}^{1/2}\mathcal{H}_{W}^{-1}\Sigma^{-1/2}=I, roman_Σ start_POSTSUPERSCRIPT - 1 / 2 end_POSTSUPERSCRIPT caligraphic_H start_POSTSUBSCRIPT italic_W end_POSTSUBSCRIPT start_POSTSUPERSCRIPT - 1 end_POSTSUPERSCRIPT roman_Σ start_POSTSUBSCRIPT italic_c end_POSTSUBSCRIPT start_POSTSUPERSCRIPT 1 / 2 end_POSTSUPERSCRIPT ( roman_Σ start_POSTSUPERSCRIPT - 1 / 2 end_POSTSUPERSCRIPT caligraphic_H start_POSTSUBSCRIPT italic_W end_POSTSUBSCRIPT start_POSTSUPERSCRIPT - 1 end_POSTSUPERSCRIPT roman_Σ start_POSTSUBSCRIPT italic_c end_POSTSUBSCRIPT start_POSTSUPERSCRIPT 1 / 2 end_POSTSUPERSCRIPT ) start_POSTSUPERSCRIPT italic_T end_POSTSUPERSCRIPT = roman_Σ start_POSTSUPERSCRIPT - 1 / 2 end_POSTSUPERSCRIPT caligraphic_H start_POSTSUBSCRIPT italic_W end_POSTSUBSCRIPT start_POSTSUPERSCRIPT - 1 end_POSTSUPERSCRIPT roman_Σ start_POSTSUBSCRIPT italic_c end_POSTSUBSCRIPT start_POSTSUPERSCRIPT 1 / 2 end_POSTSUPERSCRIPT roman_Σ start_POSTSUBSCRIPT italic_c end_POSTSUBSCRIPT start_POSTSUPERSCRIPT 1 / 2 end_POSTSUPERSCRIPT caligraphic_H start_POSTSUBSCRIPT italic_W end_POSTSUBSCRIPT start_POSTSUPERSCRIPT - 1 end_POSTSUPERSCRIPT roman_Σ start_POSTSUPERSCRIPT - 1 / 2 end_POSTSUPERSCRIPT = italic_I ,
Using of the Slutsky theorem and (S7.3 ), we have
Σ − 1 / 2 r / a n ( 𝜷 ˇ − 𝜷 ^ ) → 𝑑 N p ( 𝟎 , I ) , r , n → ∞ . formulae-sequence 𝑑 → superscript Σ 1 2 𝑟 subscript 𝑎 𝑛 ˇ 𝜷 ^ 𝜷 subscript 𝑁 𝑝 0 𝐼 𝑟
→ 𝑛 \Sigma^{-1/2}\sqrt{r/a_{n}}(\check{\boldsymbol{\beta}}-\hat{\boldsymbol{\beta}%
})\xrightarrow{d}N_{p}(\mathbf{0},I),r,n\rightarrow\infty. roman_Σ start_POSTSUPERSCRIPT - 1 / 2 end_POSTSUPERSCRIPT square-root start_ARG italic_r / italic_a start_POSTSUBSCRIPT italic_n end_POSTSUBSCRIPT end_ARG ( overroman_ˇ start_ARG bold_italic_β end_ARG - over^ start_ARG bold_italic_β end_ARG ) start_ARROW overitalic_d → end_ARROW italic_N start_POSTSUBSCRIPT italic_p end_POSTSUBSCRIPT ( bold_0 , italic_I ) , italic_r , italic_n → ∞ .
(S7.6)
As m > 1 𝑚 1 m>1 italic_m > 1 , we have 𝜷 ˇ ( m ) = 1 m ∑ k = 1 m 𝜷 ˇ k , superscript ˇ 𝜷 𝑚 1 𝑚 superscript subscript 𝑘 1 𝑚 subscript ˇ 𝜷 𝑘 \check{\boldsymbol{\beta}}^{(m)}=\frac{1}{m}\sum_{k=1}^{m}\check{\boldsymbol{%
\beta}}_{k}, overroman_ˇ start_ARG bold_italic_β end_ARG start_POSTSUPERSCRIPT ( italic_m ) end_POSTSUPERSCRIPT = divide start_ARG 1 end_ARG start_ARG italic_m end_ARG ∑ start_POSTSUBSCRIPT italic_k = 1 end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_m end_POSTSUPERSCRIPT overroman_ˇ start_ARG bold_italic_β end_ARG start_POSTSUBSCRIPT italic_k end_POSTSUBSCRIPT ,
From the central limit theorem, it can be concluded that
r m a n ( 𝜷 ˇ ( m ) − 𝜷 ^ ) = 1 m ∑ k = 1 m r a n ( 𝜷 ˇ k − 𝜷 ^ ) → 𝑑 N p ( 𝟎 , Σ ) , r , n → ∞ . formulae-sequence 𝑟 𝑚 subscript 𝑎 𝑛 superscript ˇ 𝜷 𝑚 ^ 𝜷 1 𝑚 superscript subscript 𝑘 1 𝑚 𝑟 subscript 𝑎 𝑛 subscript ˇ 𝜷 𝑘 ^ 𝜷 𝑑 → subscript 𝑁 𝑝 0 Σ → 𝑟 𝑛
\displaystyle\sqrt{\frac{rm}{a_{n}}}(\check{\boldsymbol{\beta}}^{(m)}-\hat{%
\boldsymbol{\beta}})=\frac{1}{\sqrt{m}}\sum_{k=1}^{m}\sqrt{\frac{r}{a_{n}}}(%
\check{\boldsymbol{\beta}}_{k}-\hat{\boldsymbol{\beta}})\xrightarrow{d}N_{p}(%
\mathbf{0},\Sigma),r,n\rightarrow\infty. square-root start_ARG divide start_ARG italic_r italic_m end_ARG start_ARG italic_a start_POSTSUBSCRIPT italic_n end_POSTSUBSCRIPT end_ARG end_ARG ( overroman_ˇ start_ARG bold_italic_β end_ARG start_POSTSUPERSCRIPT ( italic_m ) end_POSTSUPERSCRIPT - over^ start_ARG bold_italic_β end_ARG ) = divide start_ARG 1 end_ARG start_ARG square-root start_ARG italic_m end_ARG end_ARG ∑ start_POSTSUBSCRIPT italic_k = 1 end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_m end_POSTSUPERSCRIPT square-root start_ARG divide start_ARG italic_r end_ARG start_ARG italic_a start_POSTSUBSCRIPT italic_n end_POSTSUBSCRIPT end_ARG end_ARG ( overroman_ˇ start_ARG bold_italic_β end_ARG start_POSTSUBSCRIPT italic_k end_POSTSUBSCRIPT - over^ start_ARG bold_italic_β end_ARG ) start_ARROW overitalic_d → end_ARROW italic_N start_POSTSUBSCRIPT italic_p end_POSTSUBSCRIPT ( bold_0 , roman_Σ ) , italic_r , italic_n → ∞ .
Then the theorem is proved.