-
Error Controlled Feature Selection for Ultrahigh Dimensional and Highly Correlated Feature Space Using Deep Learning
Authors:
Arkaprabha Ganguli,
David Todem,
Tapabrata Maiti
Abstract:
In recent years, deep learning has been at the center of analytics due to its impressive empirical success in analyzing complex data objects. Despite this success, most of the existing tools behave like black-box machines, thus the increasing interest in interpretable, reliable, and robust deep learning models applicable to a broad class of applications. Feature-selected deep learning has emerged…
▽ More
In recent years, deep learning has been at the center of analytics due to its impressive empirical success in analyzing complex data objects. Despite this success, most of the existing tools behave like black-box machines, thus the increasing interest in interpretable, reliable, and robust deep learning models applicable to a broad class of applications. Feature-selected deep learning has emerged as a promising tool in this realm. However, the recent developments do not accommodate ultra-high dimensional and highly correlated features, in addition to the high noise level. In this article, we propose a novel screening and cleaning method with the aid of deep learning for a data-adaptive multi-resolutional discovery of highly correlated predictors with a controlled error rate. Extensive empirical evaluations over a wide range of simulated scenarios and several real datasets demonstrate the effectiveness of the proposed method in achieving high power while kee** the false discovery rate at a minimum.
△ Less
Submitted 31 October, 2022; v1 submitted 14 September, 2022;
originally announced September 2022.
-
Nonparametric Scanning For Nonrandom Missing Data With Continuous Instrumental Variables
Authors:
Arkaprabha Ganguli,
David Todem
Abstract:
This article introduces a new instrumental variable approach for estimating unknown population parameters with data having nonrandom missing values. With coarse and discrete instruments, Shao and Wang (2016) proposed a semiparametric method that uses the added information to identify the tilting parameter from the missing data propensity model. A naive application of this idea to continuous instru…
▽ More
This article introduces a new instrumental variable approach for estimating unknown population parameters with data having nonrandom missing values. With coarse and discrete instruments, Shao and Wang (2016) proposed a semiparametric method that uses the added information to identify the tilting parameter from the missing data propensity model. A naive application of this idea to continuous instruments through arbitrary discretizations is apt to be inefficient, and maybe questionable in some settings. We propose a nonparametric method not requiring arbitrary discretizations but involves scanning over continuous dichotomizations of the instrument; and combining scan statistics to estimate the unknown parameters via weighted integration.
We establish the asymptotic normality of the proposed integrated estimator and that of the underlying scan processes uniformly across the instrument sample space. Simulation studies and the analysis of a real data set demonstrate the gains of the methodology over procedures that rely either on arbitrary discretizations or moments of the instrument.
△ Less
Submitted 17 November, 2021;
originally announced November 2021.
-
A Millennium Bug Still Bites Public Health - An Illustration Using Cancer Mortality
Authors:
Martina Fu,
David Todem,
Wenjiang J. Fu,
Shuangge Ma
Abstract:
Accurate estimation of cancer mortality rates and the comparison across cancer sites, populations or time periods is crucial to public health, as identification of vulnerable groups who suffer the most from these diseases may lead to efficient cancer care and control with timely treatment. Because cancer mortality rate varies with age, comparisons require age-standardization using a reference popu…
▽ More
Accurate estimation of cancer mortality rates and the comparison across cancer sites, populations or time periods is crucial to public health, as identification of vulnerable groups who suffer the most from these diseases may lead to efficient cancer care and control with timely treatment. Because cancer mortality rate varies with age, comparisons require age-standardization using a reference population. The current method of using the Year 2000 Population Standard is standard practice, but serious concerns have been raised about its lack of justification. We have found that using the US Year 2000 Population Standard as reference overestimates prostate cancer mortality rates by 12-91% during the period 1970-2009 across all six sampled U.S. states, and also underestimates case fatality rates by 9-78% across six cancer sites, including female breast, cervix, prostate, lung, leukemia and colon-rectum. We develop a mean reference population method to minimize the bias using mathematical optimization theory and statistical modeling. The method corrects the bias to the largest extent in terms of squared loss and can be applied broadly to studies of many diseases.
△ Less
Submitted 29 January, 2014;
originally announced January 2014.
