Skip to main content

Showing 1–18 of 18 results for author: Zhang, K W

.
  1. arXiv:2406.13127  [pdf, other

    cs.AI

    Oralytics Reinforcement Learning Algorithm

    Authors: Anna L. Trella, Kelly W. Zhang, Stephanie M. Carpenter, David Elashoff, Zara M. Greer, Inbal Nahum-Shani, Dennis Ruenger, Vivek Shetty, Susan A. Murphy

    Abstract: Dental disease is still one of the most common chronic diseases in the United States. While dental disease is preventable through healthy oral self-care behaviors (OSCB), this basic behavior is not consistently practiced. We have developed Oralytics, an online, reinforcement learning (RL) algorithm that optimizes the delivery of personalized intervention prompts to improve OSCB. In this paper, we… ▽ More

    Submitted 18 June, 2024; originally announced June 2024.

  2. arXiv:2405.19466  [pdf, other

    cs.LG stat.ML

    Posterior Sampling via Autoregressive Generation

    Authors: Kelly W Zhang, Tiffany, Cai, Hongseok Namkoong, Daniel Russo

    Abstract: Real-world decision-making requires grappling with a perpetual lack of data as environments change; intelligent agents must comprehend uncertainty and actively gather information to resolve it. We propose a new framework for learning bandit algorithms from massive historical data, which we demonstrate in a cold-start recommendation problem. First, we use historical data to pretrain an autoregressi… ▽ More

    Submitted 29 May, 2024; originally announced May 2024.

  3. arXiv:2403.10946  [pdf, other

    stat.ML cs.LG

    The Fallacy of Minimizing Local Regret in the Sequential Task Setting

    Authors: Zi** Xu, Kelly W. Zhang, Susan A. Murphy

    Abstract: In the realm of Reinforcement Learning (RL), online RL is often conceptualized as an optimization problem, where an algorithm interacts with an unknown environment to minimize cumulative regret. In a stationary setting, strong theoretical guarantees, like a sublinear ($\sqrt{T}$) regret bound, can be obtained, which typically implies the convergence to an optimal policy and the cessation of explor… ▽ More

    Submitted 16 March, 2024; originally announced March 2024.

  4. arXiv:2402.17003  [pdf, other

    cs.LG cs.AI cs.CY

    Monitoring Fidelity of Online Reinforcement Learning Algorithms in Clinical Trials

    Authors: Anna L. Trella, Kelly W. Zhang, Inbal Nahum-Shani, Vivek Shetty, Iris Yan, Finale Doshi-Velez, Susan A. Murphy

    Abstract: Online reinforcement learning (RL) algorithms offer great potential for personalizing treatment for participants in clinical trials. However, deploying an online, autonomous algorithm in the high-stakes healthcare setting makes quality control and data quality especially difficult to achieve. This paper proposes algorithm fidelity as a critical requirement for deploying online RL algorithms in cli… ▽ More

    Submitted 26 February, 2024; originally announced February 2024.

  5. arXiv:2208.07406  [pdf, other

    cs.AI cs.LG

    Reward Design For An Online Reinforcement Learning Algorithm Supporting Oral Self-Care

    Authors: Anna L. Trella, Kelly W. Zhang, Inbal Nahum-Shani, Vivek Shetty, Finale Doshi-Velez, Susan A. Murphy

    Abstract: Dental disease is one of the most common chronic diseases despite being largely preventable. However, professional advice on optimal oral hygiene practices is often forgotten or abandoned by patients. Therefore patients may benefit from timely and personalized encouragement to engage in oral self-care behaviors. In this paper, we develop an online reinforcement learning (RL) algorithm for use in o… ▽ More

    Submitted 14 September, 2022; v1 submitted 15 August, 2022; originally announced August 2022.

  6. arXiv:2208.00250  [pdf, other

    cs.LG cs.AI

    A Bayesian Approach to Learning Bandit Structure in Markov Decision Processes

    Authors: Kelly W. Zhang, Omer Gottesman, Finale Doshi-Velez

    Abstract: In the reinforcement learning literature, there are many algorithms developed for either Contextual Bandit (CB) or Markov Decision Processes (MDP) environments. However, when deploying reinforcement learning algorithms in the real world, even with domain expertise, it is often difficult to know whether it is appropriate to treat a sequential decision making problem as a CB or an MDP. In other word… ▽ More

    Submitted 30 July, 2022; originally announced August 2022.

    Comments: Challenges of Real-World Reinforcement Learning 2020 (NeurIPS Workshop)

  7. Designing Reinforcement Learning Algorithms for Digital Interventions: Pre-implementation Guidelines

    Authors: Anna L. Trella, Kelly W. Zhang, Inbal Nahum-Shani, Vivek Shetty, Finale Doshi-Velez, Susan A. Murphy

    Abstract: Online reinforcement learning (RL) algorithms are increasingly used to personalize digital interventions in the fields of mobile health and online education. Common challenges in designing and testing an RL algorithm in these settings include ensuring the RL algorithm can learn and run stably under real-time constraints, and accounting for the complexity of the environment, e.g., a lack of accurat… ▽ More

    Submitted 18 August, 2022; v1 submitted 8 June, 2022; originally announced June 2022.

  8. arXiv:2202.07098  [pdf, ps, other

    cs.LG stat.ME

    Statistical Inference After Adaptive Sampling for Longitudinal Data

    Authors: Kelly W. Zhang, Lucas Janson, Susan A. Murphy

    Abstract: Online reinforcement learning and other adaptive sampling algorithms are increasingly used in digital intervention experiments to optimize treatment delivery for users over time. In this work, we focus on longitudinal user data collected by a large class of adaptive sampling algorithms that are designed to optimize treatment decisions online using accruing data from multiple users. Combining or "p… ▽ More

    Submitted 19 April, 2023; v1 submitted 14 February, 2022; originally announced February 2022.

    Comments: Fixing typos

  9. arXiv:2104.14074  [pdf, other

    cs.LG

    Statistical Inference with M-Estimators on Adaptively Collected Data

    Authors: Kelly W. Zhang, Lucas Janson, Susan A. Murphy

    Abstract: Bandit algorithms are increasingly used in real-world sequential decision-making problems. Associated with this is an increased desire to be able to use the resulting datasets to answer scientific questions like: Did one type of ad lead to more purchases? In which contexts is a mobile health intervention effective? However, classical statistical approaches fail to provide valid confidence interval… ▽ More

    Submitted 19 November, 2021; v1 submitted 28 April, 2021; originally announced April 2021.

    Journal ref: Advances in Neural Information Processing Systems, 2021

  10. arXiv:2002.03217  [pdf, other

    cs.LG stat.ML

    Inference for Batched Bandits

    Authors: Kelly W. Zhang, Lucas Janson, Susan A. Murphy

    Abstract: As bandit algorithms are increasingly utilized in scientific studies and industrial applications, there is an associated increasing need for reliable inference methods based on the resulting adaptively-collected data. In this work, we develop methods for inference on data collected in batches using a bandit algorithm. We first prove that the ordinary least squares estimator (OLS), which is asympto… ▽ More

    Submitted 8 January, 2021; v1 submitted 8 February, 2020; originally announced February 2020.

    Journal ref: NeurIPS 2020

  11. arXiv:1809.10040  [pdf, other

    cs.CL

    Language Modeling Teaches You More Syntax than Translation Does: Lessons Learned Through Auxiliary Task Analysis

    Authors: Kelly W. Zhang, Samuel R. Bowman

    Abstract: Recent work using auxiliary prediction task classifiers to investigate the properties of LSTM representations has begun to shed light on why pretrained representations, like ELMo (Peters et al., 2018) and CoVe (McCann et al., 2017), are so beneficial for neural language understanding models. We still, though, do not yet have a clear understanding of how the choice of pretraining objective affects… ▽ More

    Submitted 7 January, 2019; v1 submitted 26 September, 2018; originally announced September 2018.

    Journal ref: Blackbox NLP Workshop, EMNLP 2018

  12. arXiv:1510.00168  [pdf, ps, other

    cond-mat.mtrl-sci

    Two-Dimensional PN Monolayer Sheets with Fantastic Structures and Properties

    Authors: ShuangYing Ma, Chaoyu He, L. Z. Sun, Hai** Lin, Youyong Li, K. W. Zhang

    Abstract: Three two-dimensional phosphorus nitride (PN) monolayer sheets (named as $α$-, $β$-, and $γ$-PN, respectively) with fantastic structures and properties are predicted based on first-principles calculations. The $α$-PN and $γ$-PN are buckled structure, whereas $β$-PN shows puckered characteristics. Their unique structures endows these atomic PN sheets with high dynamic stabilities and anisotropic me… ▽ More

    Submitted 1 October, 2015; originally announced October 2015.

    Comments: 8 pages, 7 figures

  13. arXiv:1307.6324  [pdf, ps, other

    cond-mat.mtrl-sci

    Novel Two-dimensional SiC2 Sheet with Full Pentagon Network

    Authors: J. Liu, C. Y. He, N. Jiao, H. P. Xiao, K. W. Zhang, R. Z. Wang, L. Z. Sun

    Abstract: We propose a promising two-dimensional nano-sheet of SiC2 (SiC2-pentagon) consisting of tetrahedral silicon atoms and triple-linked carbon atoms in a fully-pentagon network. The SiC2-pentagon with buckled configuration is more favorable than its planar counterpart and previously proposed SiC2-silagraphene with tetra-coordinate silicon atoms; and its dynamical stability is confirmed through phonon… ▽ More

    Submitted 24 July, 2013; originally announced July 2013.

    Comments: 6 pages, 6 figures

  14. arXiv:1305.1791  [pdf, ps, other

    cond-mat.mtrl-sci

    Magnetic Exchange Coupling and Anisotropy of 3d Transition-Metal Nanowire on the Surface of Graphyne Sheet

    Authors: Junjie He, Pan Zhou, N. Jiao, S. Y. Ma, K. W. Zhang, R. Z. Wang, L. Z. Sun

    Abstract: Using density functional theory plus Hubbard-U (DFT+U) approach, we find that quasi one-dementation(1D) 3d transition metal(TM) zigzag nanowire can be constructed by TM adsorbed on the surface of graphyne sheet. The results show that the TM exchange coupling of the zigzag nanowire mediated by sp hybridized carbon atoms gives rise to long range ferromagnetic order except for Cr with anti-ferromagne… ▽ More

    Submitted 21 May, 2013; v1 submitted 8 May, 2013; originally announced May 2013.

    Comments: 8 pages, 7 figures

  15. arXiv:1204.6621  [pdf, ps, other

    cond-mat.mtrl-sci

    Structure, stability and electronic properties of tricycle type graphane

    Authors: Chaoyu He, L. Z. Sun, C. X. Zhang, N. Jiao, K. W. Zhang, Jianxin Zhong

    Abstract: We propose a new allotrope of graphane, named as tricycle graphane,with a 4up/2down UUUDUD hydrogenation in each hexagonal carbon ring,which is different from previously proposed allotropes with UUDUUD(boat-1) and UUUUDD (boat-2) types of hydrogenation. Its stability and electronic structures are systematically studied using first-principles method. We find that the tricycle graphane is a stable p… ▽ More

    Submitted 30 April, 2012; originally announced April 2012.

    Comments: 5 pages, 3 figures

  16. arXiv:1204.2188  [pdf, ps, other

    cond-mat.mtrl-sci

    First-principles study of a novel superhard boron nitride phase

    Authors: Chaoyu He, L. Z. Sun, C. X. Zhang, Xiangyang Peng, K. W. Zhang, Jianxin Zhong

    Abstract: A superhard boron nitride phase dubbed as Z-BN is proposed as possible intermediate phase between h-BN and zinc blende BN (c-BN), and investigated using first-principles calculations within the framework of the density functional theory. Although the structure of Z-BN is similar to that of bct-BN containing four-eight BN rings, it is more energy favorable than bct-BN. Our study reveals that Z-BN,… ▽ More

    Submitted 10 June, 2012; v1 submitted 10 April, 2012; originally announced April 2012.

    Comments: 5 pages, 5 figures

  17. arXiv:1203.5879  [pdf, ps, other

    cond-mat.mtrl-sci

    Four superhard carbon allotropes: First-principle study

    Authors: Chaoyu He, L. Z. Sun, C. X. Zhang, K. W. Zhang, Xiangyang Peng, Jianxin Zhong

    Abstract: Using a generalized genetic algorithm, we propose four new sp3 carbon allotropes with 5-6-7 (5-6-7-type Z-ACA and Z-CACB) or 4-6-8(4-6-8-type Z4-A3B1 and A4-A2B2) carbon rings. Their stability, mechanical and electronic properties are systematically studied using first-principles method. We find that all these four carbon allotropes show amazing stability in comparison with recently proposed carbo… ▽ More

    Submitted 19 April, 2012; v1 submitted 27 March, 2012; originally announced March 2012.

    Comments: 6 pages, 4 figures, submit to PCCP on 21-Feb-2012

  18. arXiv:1203.5509  [pdf, ps, other

    cond-mat.mtrl-sci

    New Superhard Carbon Phases Between Graphite and Diamond

    Authors: Chaoyu He, L. Z. Sun, C. X. Zhang, K. W. Zhang, Xiangyang Peng, Jianxin Zhong

    Abstract: Two new carbon allotropes (H-carbon and S-carbon) are proposed, as possible candidates for the intermediate superhard phases between graphite and diamond obtained in the process of cold compressing graphite, based on the results of first-principles calculations. Both H-carbon and S-carbon are more stable than previously proposed M-carbon and W-carbon and their bulk modulus are comparable to that o… ▽ More

    Submitted 11 June, 2012; v1 submitted 25 March, 2012; originally announced March 2012.

    Comments: 5pages,4figures,submitted to Phys.Rev.Lett on 18Jan12, transfer to Phys.Rev.B on 25Mar12; Solid State Communications(2012), http://dx.doi.org/10.1016/j.ssc.2012.05.022