Showing 1–8 of 8 results for author: Ahmad, M A

Creating Trustworthy LLMs: Dealing with Hallucinations in Healthcare AI

Authors: Muhammad Aurangzeb Ahmad, Ilker Yaramis, Taposh Dutta Roy

Abstract: Large language models have proliferated across multiple domains in as short period of time. There is however hesitation in the medical and healthcare domain towards their adoption because of issues like factuality, coherence, and hallucinations. Give the high stakes nature of healthcare, many researchers have even cautioned against its usage until these issues are resolved. The key to the implemen… ▽ More Large language models have proliferated across multiple domains in as short period of time. There is however hesitation in the medical and healthcare domain towards their adoption because of issues like factuality, coherence, and hallucinations. Give the high stakes nature of healthcare, many researchers have even cautioned against its usage until these issues are resolved. The key to the implementation and deployment of LLMs in healthcare is to make these models trustworthy, transparent (as much possible) and explainable. In this paper we describe the key elements in creating reliable, trustworthy, and unbiased models as a necessary condition for their adoption in healthcare. Specifically we focus on the quantification, validation, and mitigation of hallucinations in the context in healthcare. Lastly, we discuss how the future of LLMs in healthcare may look like. △ Less

Submitted 26 September, 2023; originally announced November 2023.

Validation of a Hospital Digital Twin with Machine Learning

Authors: Muhammad Aurangzeb Ahmad, Vijay Chickarmane, Farinaz Sabz Ali Pour, Nima Shariari, Taposh Dutta Roy

Abstract: Recently there has been a surge of interest in develo** Digital Twins of process flows in healthcare to better understand bottlenecks and areas of improvement. A key challenge is in the validation process. We describe a work in progress for a digital twin using an agent based simulation model for determining bed turnaround time for patients in hospitals. We employ a strategy using machine learni… ▽ More Recently there has been a surge of interest in develo** Digital Twins of process flows in healthcare to better understand bottlenecks and areas of improvement. A key challenge is in the validation process. We describe a work in progress for a digital twin using an agent based simulation model for determining bed turnaround time for patients in hospitals. We employ a strategy using machine learning for validating the model and implementing sensitivity analysis. △ Less

Submitted 8 March, 2023; v1 submitted 7 March, 2023; originally announced March 2023.

Machine Learning for Deferral of Care Prediction

Authors: Muhammad Aurangzeb Ahmad, Raafia Ahmed, Dr. Steve Overman, Patrick Campbell, Corinne Stroum, Bipin Karunakaran

Abstract: Care deferral is the phenomenon where patients defer or are unable to receive healthcare services, such as seeing doctors, medications or planned surgery. Care deferral can be the result of patient decisions, service availability, service limitations, or restrictions due to cost. Continual care deferral in populations may lead to a decline in population health and compound health issues leading to… ▽ More Care deferral is the phenomenon where patients defer or are unable to receive healthcare services, such as seeing doctors, medications or planned surgery. Care deferral can be the result of patient decisions, service availability, service limitations, or restrictions due to cost. Continual care deferral in populations may lead to a decline in population health and compound health issues leading to higher social and financial costs in the long term. Consequently, identification of patients who may be at risk of deferring care is important towards improving population health and reducing care total costs. Additionally, minority and vulnerable populations are at a greater risk of care deferral due to socioeconomic factors. In this paper, we (a) address the problem of predicting care deferral for well-care visits; (b) observe that social determinants of health are relevant explanatory factors towards predicting care deferral, and (c) compute how fair the models are with respect to demographics, socioeconomic factors and selected comorbidities. Many health systems currently use rules-based techniques to retroactively identify patients who previously deferred care. The objective of this model is to identify patients at risk of deferring care and allow the health system to prevent care deferrals through direct outreach or social determinant mediation. △ Less

Submitted 8 June, 2022; originally announced July 2022.

Machine Learning Approaches for Type 2 Diabetes Prediction and Care Management

Authors: Aloysius Lim, Ashish Singh, Jody Chiam, Carly Eckert, Vikas Kumar, Muhammad Aurangzeb Ahmad, Ankur Teredesai

Abstract: Prediction of diabetes and its various complications has been studied in a number of settings, but a comprehensive overview of problem setting for diabetes prediction and care management has not been addressed in the literature. In this document we seek to remedy this omission in literature with an encompassing overview of diabetes complication prediction as well as situating this problem in the c… ▽ More Prediction of diabetes and its various complications has been studied in a number of settings, but a comprehensive overview of problem setting for diabetes prediction and care management has not been addressed in the literature. In this document we seek to remedy this omission in literature with an encompassing overview of diabetes complication prediction as well as situating this problem in the context of real world healthcare management. We illustrate various problems encountered in real world clinical scenarios via our own experience with building and deploying such models. In this manuscript we illustrate a Machine Learning (ML) framework for addressing the problem of predicting Type 2 Diabetes Mellitus (T2DM) together with a solution for risk stratification, intervention and management. These ML models align with how physicians think about disease management and mitigation, which comprises these four steps: Identify, Stratify, Engage, Measure. △ Less

Submitted 28 April, 2021; v1 submitted 15 April, 2021; originally announced April 2021.

Assessing Fairness in Classification Parity of Machine Learning Models in Healthcare

Authors: Ming Yuan, Vikas Kumar, Muhammad Aurangzeb Ahmad, Ankur Teredesai

Abstract: Fairness in AI and machine learning systems has become a fundamental problem in the accountability of AI systems. While the need for accountability of AI models is near ubiquitous, healthcare in particular is a challenging field where accountability of such systems takes upon additional importance, as decisions in healthcare can have life altering consequences. In this paper we present preliminary… ▽ More Fairness in AI and machine learning systems has become a fundamental problem in the accountability of AI systems. While the need for accountability of AI models is near ubiquitous, healthcare in particular is a challenging field where accountability of such systems takes upon additional importance, as decisions in healthcare can have life altering consequences. In this paper we present preliminary results on fairness in the context of classification parity in healthcare. We also present some exploratory methods to improve fairness and choosing appropriate classification algorithms in the context of healthcare. △ Less

Submitted 6 February, 2021; originally announced February 2021.

Emergency Department Optimization and Load Prediction in Hospitals

Authors: Karthik K. Padthe, Vikas Kumar, Carly M. Eckert, Nicholas M. Mark, Anam Zahid, Muhammad Aurangzeb Ahmad, Ankur Teredesai

Abstract: Over the past several years, across the globe, there has been an increase in people seeking care in emergency departments (EDs). ED resources, including nurse staffing, are strained by such increases in patient volume. Accurate forecasting of incoming patient volume in emergency departments (ED) is crucial for efficient utilization and allocation of ED resources. Working with a suburban ED in the… ▽ More Over the past several years, across the globe, there has been an increase in people seeking care in emergency departments (EDs). ED resources, including nurse staffing, are strained by such increases in patient volume. Accurate forecasting of incoming patient volume in emergency departments (ED) is crucial for efficient utilization and allocation of ED resources. Working with a suburban ED in the Pacific Northwest, we developed a tool powered by machine learning models, to forecast ED arrivals and ED patient volume to assist end-users, such as ED nurses, in resource allocation. In this paper, we discuss the results from our predictive models, the challenges, and the learnings from users' experiences with the tool in active clinical deployment in a real world setting. △ Less

Submitted 6 February, 2021; originally announced February 2021.

Comments: 7 pages, 3 figures, 4 tables

Survey of explainable machine learning with visual and granular methods beyond quasi-explanations

Authors: Boris Kovalerchuk, Muhammad Aurangzeb Ahmad, Ankur Teredesai

Abstract: This paper surveys visual methods of explainability of Machine Learning (ML) with focus on moving from quasi-explanations that dominate in ML to domain-specific explanation supported by granular visuals. ML interpretation is fundamentally a human activity and visual methods are more readily interpretable. While efficient visual representations of high-dimensional data exist, the loss of interpreta… ▽ More This paper surveys visual methods of explainability of Machine Learning (ML) with focus on moving from quasi-explanations that dominate in ML to domain-specific explanation supported by granular visuals. ML interpretation is fundamentally a human activity and visual methods are more readily interpretable. While efficient visual representations of high-dimensional data exist, the loss of interpretable information, occlusion, and clutter continue to be a challenge, which lead to quasi-explanations. We start with the motivation and the different definitions of explainability. The paper focuses on a clear distinction between quasi-explanations and domain specific explanations, and between explainable and an actually explained ML model that are critically important for the explainability domain. We discuss foundations of interpretability, overview visual interpretability and present several types of methods to visualize the ML models. Next, we present methods of visual discovery of ML models, with the focus on interpretable models, based on the recently introduced concept of General Line Coordinates (GLC). These methods take the critical step of creating visual explanations that are not merely quasi-explanations but are also domain specific visual explanations while these methods themselves are domain-agnostic. The paper includes results on theoretical limits to preserve n-D distances in lower dimensions, based on the Johnson-Lindenstrauss lemma, point-to-point and point-to-graph GLC approaches, and real-world case studies. The paper also covers traditional visual methods for understanding ML models, which include deep learning and time series models. We show that many of these methods are quasi-explanations and need further enhancement to become domain specific explanations. We conclude with outlining open problems and current research frontiers. △ Less

Submitted 21 September, 2020; originally announced September 2020.

Comments: 45 pages, 34 figures

Journal ref: In: Interpretable Artificial Intelligence: A Perspective of Granular Computing (Eds. W. Pedrycz, S.M.Chen), Springer, 2021, pp. 217-267

The Challenge of Imputation in Explainable Artificial Intelligence Models

Authors: Muhammad Aurangzeb Ahmad, Carly Eckert, Ankur Teredesai

Abstract: Explainable models in Artificial Intelligence are often employed to ensure transparency and accountability of AI systems. The fidelity of the explanations are dependent upon the algorithms used as well as on the fidelity of the data. Many real world datasets have missing values that can greatly influence explanation fidelity. The standard way to deal with such scenarios is imputation. This can, ho… ▽ More Explainable models in Artificial Intelligence are often employed to ensure transparency and accountability of AI systems. The fidelity of the explanations are dependent upon the algorithms used as well as on the fidelity of the data. Many real world datasets have missing values that can greatly influence explanation fidelity. The standard way to deal with such scenarios is imputation. This can, however, lead to situations where the imputed values may correspond to a setting which refer to counterfactuals. Acting on explanations from AI models with imputed values may lead to unsafe outcomes. In this paper, we explore different settings where AI models with imputation can be problematic and describe ways to address such scenarios. △ Less

Submitted 29 July, 2019; originally announced July 2019.

Comments: The IJCAI-19 Workshop on Artificial Intelligence Safety