Showing 1–15 of 15 results for author: Chan, K C

Balancing Method for Non-monotone Missing Data

Authors: Jianing Dong, Raymond K. W. Wong, Kwun Chuen Gary Chan

Abstract: Covariate balancing methods have been widely applied to single or monotone missing patterns and have certain advantages over likelihood-based methods and inverse probability weighting approaches based on standard logistic regression. In this paper, we consider non-monotone missing data under the complete-case missing variable condition (CCMV), which is a case of missing not at random (MNAR). Using… ▽ More Covariate balancing methods have been widely applied to single or monotone missing patterns and have certain advantages over likelihood-based methods and inverse probability weighting approaches based on standard logistic regression. In this paper, we consider non-monotone missing data under the complete-case missing variable condition (CCMV), which is a case of missing not at random (MNAR). Using relationships between each missing pattern and the complete-case subsample, a weighted estimator can be used for estimation, where the weight is a sum of ratios of conditional probability of observing a particular missing pattern versus that of observing the complete-case pattern, given the variables observed in the corresponding missing pattern. Plug-in estimators of the propensity ratios, however, can be unbounded and lead to unstable estimation. Using further relations between propensity ratios and balancing of moments across missing patterns, we employ tailored loss functions each encouraging empirical balance across patterns to estimate propensity ratios flexibly using functional basis expansion. We propose two penalizations to separately control propensity ratio model complexity and covariate imbalance. We study the asymptotic properties of the proposed estimators and show that they are consistent under mild smoothness assumptions. Asymptotic normality and efficiency are also developed. Numerical simulation results show that the proposed method achieves smaller mean squared errors than other methods. △ Less

Submitted 13 February, 2024; originally announced February 2024.

Integration of Summary Information from External Studies for Semiparametric Models

Authors: Jianxuan Zang, K. C. G. Chan, Fei Gao

Abstract: With the development of biomedical science, researchers have increasing access to an abundance of studies focusing on similar research questions. There is a growing interest in the integration of summary information from those studies to enhance the efficiency of estimation in their own internal studies. In this work, we present a comprehensive framework on integration of summary information from… ▽ More With the development of biomedical science, researchers have increasing access to an abundance of studies focusing on similar research questions. There is a growing interest in the integration of summary information from those studies to enhance the efficiency of estimation in their own internal studies. In this work, we present a comprehensive framework on integration of summary information from external studies when the data are modeled by semiparametric models. Our novel framework offers straightforward estimators that update conventional estimations with auxiliary information. It addresses computational challenges by capitalizing on the intricate mathematical structure inherent to the problem. We demonstrate the conditions when the proposed estimators are theoretically more efficient than initial estimate based solely on internal data. Several special cases such as proportional hazards model in survival analysis are provided with numerical examples. △ Less

Submitted 9 November, 2023; v1 submitted 8 November, 2023; originally announced November 2023.

Flexible Functional Treatment Effect Estimation

Authors: Jiayi Wang, Raymond K. W. Wong, Xiaoke Zhang, Kwun Chuen Gary Chan

Abstract: We study treatment effect estimation with functional treatments where the average potential outcome functional is a function of functions, in contrast to continuous treatment effect estimation where the target is a function of real numbers. By considering a flexible scalar-on-function marginal structural model, a weight-modified kernel ridge regression (WMKRR) is adopted for estimation. The weight… ▽ More We study treatment effect estimation with functional treatments where the average potential outcome functional is a function of functions, in contrast to continuous treatment effect estimation where the target is a function of real numbers. By considering a flexible scalar-on-function marginal structural model, a weight-modified kernel ridge regression (WMKRR) is adopted for estimation. The weights are constructed by directly minimizing the uniform balancing error resulting from a decomposition of the WMKRR estimator, instead of being estimated under a particular treatment selection model. Despite the complex structure of the uniform balancing error derived under WMKRR, finite-dimensional convex algorithms can be applied to efficiently solve for the proposed weights thanks to a representer theorem. The optimal convergence rate is shown to be attainable by the proposed WMKRR estimator without any smoothness assumption on the true weight function. Corresponding empirical performance is demonstrated by a simulation study and a real data application. △ Less

Submitted 14 September, 2023; originally announced September 2023.

An Effective Multivariate Normality Test via Hessians of Empirical Cumulant Generating Functions

Authors: Kwun Chuen Gary Chan, Hok Kan Ling, Chuan-Fa Tang, Sheung Chi Phillip Yam

Abstract: In this article, we propose a new class of consistent tests for $p$-variate normality. These tests are based on the characterization of the standard multivariate normal distribution, that the Hessian of the corresponding cumulant generating function is identical to the $p\times p$ identity matrix and the idea of decomposing the information from the joint distribution into the dependence copula and… ▽ More In this article, we propose a new class of consistent tests for $p$-variate normality. These tests are based on the characterization of the standard multivariate normal distribution, that the Hessian of the corresponding cumulant generating function is identical to the $p\times p$ identity matrix and the idea of decomposing the information from the joint distribution into the dependence copula and all marginal distributions. Under the null hypothesis of multivariate normality, our proposed test statistic is independent of the unknown mean vector and covariance matrix so that the distribution-free critical value of the test can be obtained by Monte Carlo simulation. We also derive the asymptotic null distribution of proposed test statistic and establish the consistency of the test against different fixed alternatives. Last but not least, a comprehensive and extensive Monte Carlo study also illustrates that our test is a superb yet computationally convenient competitor to many well-known existing test statistics. △ Less

Submitted 20 March, 2023; originally announced March 2023.

Data Harmonization Via Regularized Nonparametric Mixing Distribution Estimation

Authors: Steven Wilkins-Reeves, Yen-Chi Chen, Kwun Chuen Gary Chan

Abstract: Data harmonization is the process by which an equivalence is developed between two variables measuring a common trait. Our problem is motivated by dementia research in which multiple tests are used in practice to measure the same underlying cognitive ability such as language or memory. We connect this statistical problem to mixing distribution estimation. We introduce and study a non-parametric la… ▽ More Data harmonization is the process by which an equivalence is developed between two variables measuring a common trait. Our problem is motivated by dementia research in which multiple tests are used in practice to measure the same underlying cognitive ability such as language or memory. We connect this statistical problem to mixing distribution estimation. We introduce and study a non-parametric latent trait model, develop a method which enforces uniqueness of the regularized maximum likelihood estimator, show how a nonparametric EM algorithm will converge weakly to its maximizer, and additionally propose a faster algorithm for learning a discretized approximation of the latent distribution. Furthermore, we develop methods to assess goodness of fit for the mixing likelihood which is an area neglected in most mixing distribution estimation problems. We apply our method to the National Alzheimer's Coordination Center Uniform Data Set and show that we can use our method to convert between score measurements and account for the measurement error. We show that this method outperforms standard techniques commonly used in dementia research. Full code is available at https://github.com/SteveJWR/Data-Harmonization-Nonparametric. △ Less

Submitted 12 October, 2021; originally announced October 2021.

Comments: 46 pages, 15 figures

MSC Class: 62G05

Matrix Completion with Model-free Weighting

Authors: Jiayi Wang, Raymond K. W. Wong, Xiaojun Mao, Kwun Chuen Gary Chan

Abstract: In this paper, we propose a novel method for matrix completion under general non-uniform missing structures. By controlling an upper bound of a novel balancing error, we construct weights that can actively adjust for the non-uniformity in the empirical risk without explicitly modeling the observation probabilities, and can be computed efficiently via convex optimization. The recovered matrix based… ▽ More In this paper, we propose a novel method for matrix completion under general non-uniform missing structures. By controlling an upper bound of a novel balancing error, we construct weights that can actively adjust for the non-uniformity in the empirical risk without explicitly modeling the observation probabilities, and can be computed efficiently via convex optimization. The recovered matrix based on the proposed weighted empirical risk enjoys appealing theoretical guarantees. In particular, the proposed method achieves a stronger guarantee than existing work in terms of the scaling with respect to the observation probabilities, under asymptotically heterogeneous missing settings (where entry-wise observation probabilities can be of different orders). These settings can be regarded as a better theoretical model of missing patterns with highly varying probabilities. We also provide a new minimax lower bound under a class of heterogeneous settings. Numerical experiments are also provided to demonstrate the effectiveness of the proposed method. △ Less

Submitted 9 June, 2021; originally announced June 2021.

Comments: Proceedings of the 38th International Conference on Machine Learning, PMLR 139, 2021

Estimation of Partially Conditional Average Treatment Effect by Hybrid Kernel-covariate Balancing

Authors: Jiayi Wang, Raymond K. W. Wong, Shu Yang, Kwun Chuen Gary Chan

Abstract: We study nonparametric estimation for the partially conditional average treatment effect, defined as the treatment effect function over an interested subset of confounders. We propose a hybrid kernel weighting estimator where the weights aim to control the balancing error of any function of the confounders from a reproducing kernel Hilbert space after kernel smoothing over the subset of interested… ▽ More We study nonparametric estimation for the partially conditional average treatment effect, defined as the treatment effect function over an interested subset of confounders. We propose a hybrid kernel weighting estimator where the weights aim to control the balancing error of any function of the confounders from a reproducing kernel Hilbert space after kernel smoothing over the subset of interested variables. In addition, we present an augmented version of our estimator which can incorporate estimations of outcome mean functions. Based on the representer theorem, gradient-based algorithms can be applied for solving the corresponding infinite-dimensional optimization problem. Asymptotic properties are studied without any smoothness assumptions for propensity score function or the need of data splitting, relaxing certain existing stringent assumptions. The numerical performance of the proposed estimator is demonstrated by a simulation study and an application to the effect of a mother's smoking on a baby's birth weight conditioned on the mother's age. △ Less

Submitted 4 March, 2021; originally announced March 2021.

Comments: 19 pages, 2 figures

Defining and Estimating Subgroup Mediation Effects with Semi-Competing Risks Data

Authors: Fei Gao, Fan Xia, Kwun Chuen Gary Chan

Abstract: In many medical studies, an ultimate failure event such as death is likely to be affected by the occurrence and timing of other intermediate clinical events. Both event times are subject to censoring by loss-to-follow-up but the nonterminal event may further be censored by the occurrence of the primary outcome, but not vice versa. To study the effect of an intervention on both events, the intermed… ▽ More In many medical studies, an ultimate failure event such as death is likely to be affected by the occurrence and timing of other intermediate clinical events. Both event times are subject to censoring by loss-to-follow-up but the nonterminal event may further be censored by the occurrence of the primary outcome, but not vice versa. To study the effect of an intervention on both events, the intermediate event may be viewed as a mediator, but conventional definition of direct and indirect effects is not applicable due to semi-competing risks data structure. We define three principal strata based on whether the potential intermediate event occurs before the potential failure event, which allow proper definition of direct and indirect effects in one stratum whereas total effects are defined for all strata. We discuss the identification conditions for stratum-specific effects, and proposed a semiparametric estimator based on a multivariate logistic stratum membership model and within-stratum proportional hazards models for the event times. By treating the unobserved stratum membership as a latent variable, we propose an EM algorithm for computation. We study the asymptotic properties of the estimators by the modern empirical process theory and examine the performance of the estimators in numerical studies. △ Less

Submitted 15 January, 2021; v1 submitted 30 September, 2020; originally announced October 2020.

Quasi-conformal Geometry based Local Deformation Analysis of Lateral Cephalogram for Childhood OSA Classification

Authors: Hei-Long Chan, Hoi-Man Yuen, Chun-Ting Au, Kate Ching-Ching Chan, Albert Martin Li, Lok-Ming Lui

Abstract: Craniofacial profile is one of the anatomical causes of obstructive sleep apnea(OSA). By medical research, cephalometry provides information on patients' skeletal structures and soft tissues. In this work, a novel approach to cephalometric analysis using quasi-conformal geometry based local deformation information was proposed for OSA classification. Our study was a retrospective analysis based on… ▽ More Craniofacial profile is one of the anatomical causes of obstructive sleep apnea(OSA). By medical research, cephalometry provides information on patients' skeletal structures and soft tissues. In this work, a novel approach to cephalometric analysis using quasi-conformal geometry based local deformation information was proposed for OSA classification. Our study was a retrospective analysis based on 60 case-control pairs with accessible lateral cephalometry and polysomnography (PSG) data. By using the quasi-conformal geometry to study the local deformation around 15 landmark points, and combining the results with three linear distances between landmark points, a total of 1218 information features were obtained per subject. A L2 norm based classification model was built. Under experiments, our proposed model achieves 92.5% testing accuracy. △ Less

Submitted 31 May, 2020; originally announced June 2020.

Controlling the False Discovery Rate for Binary Feature Selection via Knockoff

Authors: Yuxiang Xie, Kwun Chuen Gary Chan

Abstract: Variable selection has been widely used in data analysis for the past decades, and it becomes increasingly important in the Big Data era as there are usually hundreds of variables available in a dataset. To enhance interpretability of a model, identifying potentially relevant features is often a step before fitting all the features into a regression model. A good variable selection method should e… ▽ More Variable selection has been widely used in data analysis for the past decades, and it becomes increasingly important in the Big Data era as there are usually hundreds of variables available in a dataset. To enhance interpretability of a model, identifying potentially relevant features is often a step before fitting all the features into a regression model. A good variable selection method should effectively control the fraction of false discoveries and ensure large enough power of its selection set. In a lot of contemporary data applications, a great portion of features are coded as binary variables. Binary features are widespread in many fields, from online controlled experiments to genome science to physical statistics. Although there has recently been a handful of literature for provable false discovery rate (FDR) control in variable selection, most of the theoretical analyses were based on some strong dependency assumption or Gaussian assumption among features. In this paper we propose a variable selection method in regression framework for selecting binary features. Under mild conditions, we show that FDR is controlled exactly under a target level in a finite sample if the underlying distribution of the binary features is known. We show in simulations that FDR control is still attained when feature distribution is estimated from data. We also provide theoretical results on the power of our variables selection method in a linear regression model or a logistic regression model. In the restricted settings where competitors exist, we show in simulations and real data application on a HIV antiretroviral therapy dataset that our method has higher power than the competitor. △ Less

Submitted 13 August, 2020; v1 submitted 2 July, 2018; originally announced July 2018.

MSC Class: 62

Efficient nonparametric estimation of causal mediation effects

Authors: K. C. G. Chan, K. Imai, S. C. P. Yam, Z. Zhang

Abstract: An essential goal of program evaluation and scientific research is the investigation of causal mechanisms. Over the past several decades, causal mediation analysis has been used in medical and social sciences to decompose the treatment effect into the natural direct and indirect effects. However, all of the existing mediation analysis methods rely on parametric modeling assumptions in one way or a… ▽ More An essential goal of program evaluation and scientific research is the investigation of causal mechanisms. Over the past several decades, causal mediation analysis has been used in medical and social sciences to decompose the treatment effect into the natural direct and indirect effects. However, all of the existing mediation analysis methods rely on parametric modeling assumptions in one way or another, typically requiring researchers to specify multiple regression models involving the treatment, mediator, outcome, and pre-treatment confounders. To overcome this limitation, we propose a novel nonparametric estimation method for causal mediation analysis that eliminates the need for applied researchers to model multiple conditional distributions. The proposed method balances a certain set of empirical moments between the treatment and control groups by weighting each observation; in particular, we establish that the proposed estimator is globally semiparametric efficient. We also show how to consistently estimate the asymptotic variance of the proposed estimator without additional efforts. Finally, we extend the proposed method to other relevant settings including the causal mediation analysis with multiple mediators. △ Less

Submitted 14 January, 2016; originally announced January 2016.

Comments: Nonparametric Estimation, Natural direct effects, Natural indirect effects, Treatment effects, Semiparametric efficiency

MSC Class: 62G05

Oracle, Multiple Robust and Multipurpose Calibration in a Missing Response Problem

Authors: Kwun Chuen Gary Chan, Sheung Chi Phillip Yam

Abstract: In the presence of a missing response, reweighting the complete case subsample by the inverse of nonmissing probability is both intuitive and easy to implement. When the population totals of some auxiliary variables are known and when the inclusion probabilities are known by design, survey statisticians have developed calibration methods for improving efficiencies of the inverse probability weight… ▽ More In the presence of a missing response, reweighting the complete case subsample by the inverse of nonmissing probability is both intuitive and easy to implement. When the population totals of some auxiliary variables are known and when the inclusion probabilities are known by design, survey statisticians have developed calibration methods for improving efficiencies of the inverse probability weighting estimators and the methods can be applied to missing data analysis. Model-based calibration has been proposed in the survey sampling literature, where multidimensional auxiliary variables are first summarized into a predictor function from a working regression model. Usually, one working model is being proposed for each parameter of interest and results in different sets of calibration weights for estimating different parameters. This paper considers calibration using multiple working regression models for estimating a single or multiple parameters. Contrary to a common belief that overfitting hurts efficiency, we present three rather unexpected results. First, when the missing probability is correctly specified and multiple working regression models for the conditional mean are posited, calibration enjoys an oracle property: the same semiparametric efficiency bound is attained as if the true outcome model is known in advance. Second, when the missing data mechanism is misspecified, calibration can still be a consistent estimator when any one of the outcome regression models is correctly specified. Third, a common set of calibration weights can be used to improve efficiency in estimating multiple parameters of interest and can simultaneously attain semiparametric efficiency bounds for all parameters of interest. We provide connections of a wide class of calibration estimators, constructed based on generalized empirical likelihood, to many existing estimators in biostatistics, econometrics and survey sampling and perform simulation studies to show that the finite sample properties of calibration estimators conform well with the theoretical results being studied. △ Less

Submitted 15 October, 2014; originally announced October 2014.

Comments: Published in at http://dx.doi.org/10.1214/13-STS461 the Statistical Science (http://www.imstat.org/sts/) by the Institute of Mathematical Statistics (http://www.imstat.org)

Report number: IMS-STS-STS461

Journal ref: Statistical Science 2014, Vol. 29, No. 3, 380-396

Marginalizable conditional model for clustered ordinal data

Authors: Rui Zhang, Kwun Chuen Gary Chan

Abstract: We introduce a flexible parametric mixed effects model for correlated binary data, with parameters that can be directly interpreted as marginal odds ratios. This leads to a robust estimation equation with an optimal weighting matrix being the inverse of a genuine model-based covariance matrix. Flexible correlation structures can be imposed by correlated random effects, and correlation parameters c… ▽ More We introduce a flexible parametric mixed effects model for correlated binary data, with parameters that can be directly interpreted as marginal odds ratios. This leads to a robust estimation equation with an optimal weighting matrix being the inverse of a genuine model-based covariance matrix. Flexible correlation structures can be imposed by correlated random effects, and correlation parameters can be estimated by solving a composite likelihood score function. Marginal parameters are consistently estimated even when the conditional parametric model is misspecified, and the robust estimation procedure has low estimation efficiency loss compared to the maximum likelihood estimation under a correct model specification. Simulations, analyses of the Madras longitudinal schizophrenia study and British social attributes panel survey were carried out to demonstrate our method. △ Less

Submitted 30 March, 2014; originally announced March 2014.

Comments: 23 pages

A marginalizable frailty model for correlated right-censored data

Authors: Rui Zhang Kwun Chuen Gary Chan

Abstract: We introduce a flexible individual frailty model for clustered right-censored data, in which covariate effects can be marginally interpreted as log failure odds ratios. Flexible correlation structures can be imposed by introducing multivariate exponential distributed frailties, constructed from a set of multivariate Gaussian random variables. Finite and infinite dimensional parameters are consiste… ▽ More We introduce a flexible individual frailty model for clustered right-censored data, in which covariate effects can be marginally interpreted as log failure odds ratios. Flexible correlation structures can be imposed by introducing multivariate exponential distributed frailties, constructed from a set of multivariate Gaussian random variables. Finite and infinite dimensional parameters are consistently estimated by maximizing a composite contributing marginal likelihood and a consistent estimate for their asymptotic covariance is proposed. Parameter estimation is implemented through a hybrid expectation-maximum algorithm. Simulations and an analysis of the Rats study were carried out to demonstrate our method. △ Less

Submitted 26 March, 2014; originally announced March 2014.

Comments: 46 pages, 3 tables

Backward estimation of stochastic processes with failure events as time origins

Authors: Kwun Chuen Gary Chan, Mei-Cheng Wang

Abstract: Stochastic processes often exhibit sudden systematic changes in pattern a short time before certain failure events. Examples include increase in medical costs before death and decrease in CD4 counts before AIDS diagnosis. To study such terminal behavior of stochastic processes, a natural and direct way is to align the processes using failure events as time origins. This paper studies backward stoc… ▽ More Stochastic processes often exhibit sudden systematic changes in pattern a short time before certain failure events. Examples include increase in medical costs before death and decrease in CD4 counts before AIDS diagnosis. To study such terminal behavior of stochastic processes, a natural and direct way is to align the processes using failure events as time origins. This paper studies backward stochastic processes counting time backward from failure events, and proposes one-sample nonparametric estimation of the mean of backward processes when follow-up is subject to left truncation and right censoring. We will discuss benefits of including prevalent cohort data to enlarge the identifiable region and large sample properties of the proposed estimator with related extensions. A SEER--Medicare linked data set is used to illustrate the proposed methodologies. △ Less

Submitted 16 November, 2010; originally announced November 2010.

Comments: Published in at http://dx.doi.org/10.1214/09-AOAS319 the Annals of Applied Statistics (http://www.imstat.org/aoas/) by the Institute of Mathematical Statistics (http://www.imstat.org)

Report number: IMS-AOAS-AOAS319

Journal ref: Annals of Applied Statistics 2010, Vol. 4, No. 3, 1602-1620