A Generalist Learner for Multifaceted Medical Image Interpretation
Authors:
Hong-Yu Zhou,
Subathra Adithan,
Julián Nicolás Acosta,
Eric J. Topol,
Pranav Rajpurkar
Abstract:
Current medical artificial intelligence systems are often limited to narrow applications, hindering their widespread adoption in clinical practice. To address this limitation, we propose MedVersa, a generalist learner that enables flexible learning and tasking for medical image interpretation. By leveraging a large language model as a learnable orchestrator, MedVersa can learn from both visual and…
▽ More
Current medical artificial intelligence systems are often limited to narrow applications, hindering their widespread adoption in clinical practice. To address this limitation, we propose MedVersa, a generalist learner that enables flexible learning and tasking for medical image interpretation. By leveraging a large language model as a learnable orchestrator, MedVersa can learn from both visual and linguistic supervision, support multimodal inputs, and perform real-time task specification. This versatility allows MedVersa to adapt to various clinical scenarios and perform multifaceted medical image analysis. We introduce MedInterp, the largest multimodal dataset to date for medical image interpretation, consisting of over 13 million annotated instances spanning 11 tasks across 3 modalities, to support the development of MedVersa. Our experiments demonstrate that MedVersa achieves state-of-the-art performance in 9 tasks, sometimes outperforming specialist counterparts by over 10%. MedVersa is the first to showcase the viability of multimodal generative medical AI in implementing multimodal outputs, inputs, and dynamic task specification, highlighting its potential as a multifunctional system for comprehensive medical image analysis. This generalist approach to medical image interpretation paves the way for more adaptable and efficient AI-assisted clinical decision-making.
△ Less
Submitted 13 May, 2024;
originally announced May 2024.
Generative Artificial Intelligence in Healthcare: Ethical Considerations and Assessment Checklist
Authors:
Yilin Ning,
Salinelat Teixayavong,
Yuqing Shang,
Julian Savulescu,
Vaishaanth Nagaraj,
Di Miao,
Mayli Mertens,
Daniel Shu Wei Ting,
Jasmine Chiat Ling Ong,
Mingxuan Liu,
Jiuwen Cao,
Michael Dunn,
Roger Vaughan,
Marcus Eng Hock Ong,
Joseph Jao-Yiu Sung,
Eric J Topol,
Nan Liu
Abstract:
The widespread use of ChatGPT and other emerging technology powered by generative artificial intelligence (GenAI) has drawn much attention to potential ethical issues, especially in high-stakes applications such as healthcare, but ethical discussions are yet to translate into operationalisable solutions. Furthermore, ongoing ethical discussions often neglect other types of GenAI that have been use…
▽ More
The widespread use of ChatGPT and other emerging technology powered by generative artificial intelligence (GenAI) has drawn much attention to potential ethical issues, especially in high-stakes applications such as healthcare, but ethical discussions are yet to translate into operationalisable solutions. Furthermore, ongoing ethical discussions often neglect other types of GenAI that have been used to synthesise data (e.g., images) for research and practical purposes, which resolved some ethical issues and exposed others. We conduct a sco** review of ethical discussions on GenAI in healthcare to comprehensively analyse gaps in the current research, and further propose to reduce the gaps by develo** a checklist for comprehensive assessment and transparent documentation of ethical discussions in GenAI research. The checklist can be readily integrated into the current peer review and publication system to enhance GenAI research, and may be used for ethics-related disclosures for GenAI-powered products, healthcare applications of such products and beyond.
△ Less
Submitted 23 February, 2024; v1 submitted 2 November, 2023;
originally announced November 2023.
