Standing on FURM ground -- A framework for evaluating Fair, Useful, and Reliable AI Models in healthcare systems
Authors:
Alison Callahan,
Duncan McElfresh,
Juan M. Banda,
Gabrielle Bunney,
Danton Char,
Jonathan Chen,
Conor K. Corbin,
Debadutta Dash,
Norman L. Downing,
Sneha S. Jain,
Nikesh Kotecha,
Jonathan Masterson,
Michelle M. Mello,
Keith Morse,
Srikar Nallan,
Abby Pandya,
Anurang Revri,
Aditya Sharma,
Christopher Sharp,
Rahul Thapa,
Michael Wornow,
Alaa Youssef,
Michael A. Pfeffer,
Nigam H. Shah
Abstract:
The impact of using artificial intelligence (AI) to guide patient care or operational processes is an interplay of the AI model's output, the decision-making protocol based on that output, and the capacity of the stakeholders involved to take the necessary subsequent action. Estimating the effects of this interplay before deployment, and studying it in real time afterwards, are essential to bridge…
▽ More
The impact of using artificial intelligence (AI) to guide patient care or operational processes is an interplay of the AI model's output, the decision-making protocol based on that output, and the capacity of the stakeholders involved to take the necessary subsequent action. Estimating the effects of this interplay before deployment, and studying it in real time afterwards, are essential to bridge the chasm between AI model development and achievable benefit. To accomplish this, the Data Science team at Stanford Health Care has developed a Testing and Evaluation (T&E) mechanism to identify fair, useful and reliable AI models (FURM) by conducting an ethical review to identify potential value mismatches, simulations to estimate usefulness, financial projections to assess sustainability, as well as analyses to determine IT feasibility, design a deployment strategy, and recommend a prospective monitoring and evaluation plan. We report on FURM assessments done to evaluate six AI guided solutions for potential adoption, spanning clinical and operational settings, each with the potential to impact from several dozen to tens of thousands of patients each year. We describe the assessment process, summarize the six assessments, and share our framework to enable others to conduct similar assessments. Of the six solutions we assessed, two have moved into a planning and implementation phase. Our novel contributions - usefulness estimates by simulation, financial projections to quantify sustainability, and a process to do ethical assessments - as well as their underlying methods and open source tools, are available for other healthcare systems to conduct actionable evaluations of candidate AI solutions.
△ Less
Submitted 14 March, 2024; v1 submitted 26 February, 2024;
originally announced March 2024.
Charged cosmological black holes: a thorough study of a family of solutions
Authors:
Daniel C. Guariento,
Alan Maciel,
Marina M. C. Mello,
Vilson T. Zanchin
Abstract:
We study a class of charged cosmological black holes defined by the Shah-Vaidya solution, which is similar to the McVittie solution but for a central object of nonzero electric charge. We show that the Shah-Vaidya metric is a solution of Einstein's equations with a cuscuton and a Maxwell fields as sources, as well as a mass parameter. We then analyze the possible causal structures of the solution…
▽ More
We study a class of charged cosmological black holes defined by the Shah-Vaidya solution, which is similar to the McVittie solution but for a central object of nonzero electric charge. We show that the Shah-Vaidya metric is a solution of Einstein's equations with a cuscuton and a Maxwell fields as sources, as well as a mass parameter. We then analyze the possible causal structures of the solution under some few physically reasonable assumptions, and determine the regions in the parameter space corresponding to well behaved charged cosmological black holes and those corresponding to naked singularities. The asymptotic behavior of the Hubble factor $H(t)$ is also determinant to the causal properties of the spacetime and a theorem explaining its effect is stated. Examples of causal diagrams covering all the possibles types of spacetimes allowed by our initial assumptions are drawn and discussed.
△ Less
Submitted 15 August, 2019; v1 submitted 14 August, 2019;
originally announced August 2019.
Evolving black holes from conformal transformations of static solutions
Authors:
Marina M. C. Mello,
Alan Maciel,
Vilson T. Zanchin
Abstract:
A class of nonstationary spacetimes is obtained by means of a conformal transformation of the Schwarzschild metric, where the conformal factor $a(t)$ is an arbitrary function of the time coordinate only. We investigate several situations including some where the final state is a central object with constant mass. The metric is such that there is an initial big-bang type singularity and the final s…
▽ More
A class of nonstationary spacetimes is obtained by means of a conformal transformation of the Schwarzschild metric, where the conformal factor $a(t)$ is an arbitrary function of the time coordinate only. We investigate several situations including some where the final state is a central object with constant mass. The metric is such that there is an initial big-bang type singularity and the final state depends on the chosen conformal factor. The Misner-Sharp mass is computed and a localized central object may be identified. The trap** horizons, geodesic and causal structure of the resulting spacetimes are investigated in detail. When $a(t)$ asymptotes to a constant in a short enough time scale, the spacetime presents an event horizon and its analytical extension reveals black-hole or white-hole regions. On the other hand, when $a(t)$ is unbounded from above as in cosmological models, the spacetime presents no event horizons and may present null singularities in the future. The energy-momentum content and other properties of the respective spacetimes are also investigated.
△ Less
Submitted 4 May, 2017; v1 submitted 15 November, 2016;
originally announced November 2016.
