-
Interpretable Reinforcement Learning for Robotics and Continuous Control
Authors:
Rohan Paleja,
Letian Chen,
Yaru Niu,
Andrew Silva,
Zhaoxin Li,
Songan Zhang,
Chace Ritchie,
Sugju Choi,
Kimberlee Chestnut Chang,
Hongtei Eric Tseng,
Yan Wang,
Subramanya Nageshrao,
Matthew Gombolay
Abstract:
Interpretability in machine learning is critical for the safe deployment of learned policies across legally-regulated and safety-critical domains. While gradient-based approaches in reinforcement learning have achieved tremendous success in learning policies for continuous control problems such as robotics and autonomous driving, the lack of interpretability is a fundamental barrier to adoption. W…
▽ More
Interpretability in machine learning is critical for the safe deployment of learned policies across legally-regulated and safety-critical domains. While gradient-based approaches in reinforcement learning have achieved tremendous success in learning policies for continuous control problems such as robotics and autonomous driving, the lack of interpretability is a fundamental barrier to adoption. We propose Interpretable Continuous Control Trees (ICCTs), a tree-based model that can be optimized via modern, gradient-based, reinforcement learning approaches to produce high-performing, interpretable policies. The key to our approach is a procedure for allowing direct optimization in a sparse decision-tree-like representation. We validate ICCTs against baselines across six domains, showing that ICCTs are capable of learning policies that parity or outperform baselines by up to 33% in autonomous driving scenarios while achieving a 300x-600x reduction in the number of parameters against deep learning baselines. We prove that ICCTs can serve as universal function approximators and display analytically that ICCTs can be verified in linear time. Furthermore, we deploy ICCTs in two realistic driving domains, based on interstate Highway-94 and 280 in the US. Finally, we verify ICCT's utility with end-users and find that ICCTs are rated easier to simulate, quicker to validate, and more interpretable than neural networks.
△ Less
Submitted 16 November, 2023;
originally announced November 2023.
-
A method for quantifying sectoral optic disc pallor in fundus photographs and its association with peripapillary RNFL thickness
Authors:
Samuel Gibbon,
Graciela Muniz-Terrera,
Fabian SL Yii,
Charlene Hamid,
Simon Cox,
Ian JC Maccormick,
Andrew J Tatham,
Craig Ritchie,
Emanuele Trucco,
Baljean Dhillon,
Thomas J MacGillivray
Abstract:
Purpose: To develop an automatic method of quantifying optic disc pallor in fundus photographs and determine associations with peripapillary retinal nerve fibre layer (pRNFL) thickness.
Methods: We used deep learning to segment the optic disc, fovea, and vessels in fundus photographs, and measured pallor. We assessed the relationship between pallor and pRNFL thickness derived from optical cohere…
▽ More
Purpose: To develop an automatic method of quantifying optic disc pallor in fundus photographs and determine associations with peripapillary retinal nerve fibre layer (pRNFL) thickness.
Methods: We used deep learning to segment the optic disc, fovea, and vessels in fundus photographs, and measured pallor. We assessed the relationship between pallor and pRNFL thickness derived from optical coherence tomography scans in 118 participants. Separately, we used images diagnosed by clinical inspection as pale (N=45) and assessed how measurements compared to healthy controls (N=46). We also developed automatic rejection thresholds, and tested the software for robustness to camera type, image format, and resolution.
Results: We developed software that automatically quantified disc pallor across several zones in fundus photographs. Pallor was associated with pRNFL thickness globally (\b{eta} = -9.81 (SE = 3.16), p < 0.05), in the temporal inferior zone (\b{eta} = -29.78 (SE = 8.32), p < 0.01), with the nasal/temporal ratio (\b{eta} = 0.88 (SE = 0.34), p < 0.05), and in the whole disc (\b{eta} = -8.22 (SE = 2.92), p < 0.05). Furthermore, pallor was significantly higher in the patient group. Lastly, we demonstrate the analysis to be robust to camera type, image format, and resolution.
Conclusions: We developed software that automatically locates and quantifies disc pallor in fundus photographs and found associations between pallor measurements and pRNFL thickness.
Translational relevance: We think our method will be useful for the identification, monitoring and progression of diseases characterized by disc pallor/optic atrophy, including glaucoma, compression, and potentially in neurodegenerative disorders.
△ Less
Submitted 13 November, 2023;
originally announced November 2023.
-
The James Webb Space Telescope Mission
Authors:
Jonathan P. Gardner,
John C. Mather,
Randy Abbott,
James S. Abell,
Mark Abernathy,
Faith E. Abney,
John G. Abraham,
Roberto Abraham,
Yasin M. Abul-Huda,
Scott Acton,
Cynthia K. Adams,
Evan Adams,
David S. Adler,
Maarten Adriaensen,
Jonathan Albert Aguilar,
Mansoor Ahmed,
Nasif S. Ahmed,
Tanjira Ahmed,
Rüdeger Albat,
Loïc Albert,
Stacey Alberts,
David Aldridge,
Mary Marsha Allen,
Shaune S. Allen,
Martin Altenburg
, et al. (983 additional authors not shown)
Abstract:
Twenty-six years ago a small committee report, building on earlier studies, expounded a compelling and poetic vision for the future of astronomy, calling for an infrared-optimized space telescope with an aperture of at least $4m$. With the support of their governments in the US, Europe, and Canada, 20,000 people realized that vision as the $6.5m$ James Webb Space Telescope. A generation of astrono…
▽ More
Twenty-six years ago a small committee report, building on earlier studies, expounded a compelling and poetic vision for the future of astronomy, calling for an infrared-optimized space telescope with an aperture of at least $4m$. With the support of their governments in the US, Europe, and Canada, 20,000 people realized that vision as the $6.5m$ James Webb Space Telescope. A generation of astronomers will celebrate their accomplishments for the life of the mission, potentially as long as 20 years, and beyond. This report and the scientific discoveries that follow are extended thank-you notes to the 20,000 team members. The telescope is working perfectly, with much better image quality than expected. In this and accompanying papers, we give a brief history, describe the observatory, outline its objectives and current observing program, and discuss the inventions and people who made it possible. We cite detailed reports on the design and the measured performance on orbit.
△ Less
Submitted 10 April, 2023;
originally announced April 2023.
-
The Science Performance of JWST as Characterized in Commissioning
Authors:
Jane Rigby,
Marshall Perrin,
Michael McElwain,
Randy Kimble,
Scott Friedman,
Matt Lallo,
René Doyon,
Lee Feinberg,
Pierre Ferruit,
Alistair Glasse,
Marcia Rieke,
George Rieke,
Gillian Wright,
Chris Willott,
Knicole Colon,
Stefanie Milam,
Susan Neff,
Christopher Stark,
Jeff Valenti,
Jim Abell,
Faith Abney,
Yasin Abul-Huda,
D. Scott Acton,
Evan Adams,
David Adler
, et al. (601 additional authors not shown)
Abstract:
This paper characterizes the actual science performance of the James Webb Space Telescope (JWST), as determined from the six month commissioning period. We summarize the performance of the spacecraft, telescope, science instruments, and ground system, with an emphasis on differences from pre-launch expectations. Commissioning has made clear that JWST is fully capable of achieving the discoveries f…
▽ More
This paper characterizes the actual science performance of the James Webb Space Telescope (JWST), as determined from the six month commissioning period. We summarize the performance of the spacecraft, telescope, science instruments, and ground system, with an emphasis on differences from pre-launch expectations. Commissioning has made clear that JWST is fully capable of achieving the discoveries for which it was built. Moreover, almost across the board, the science performance of JWST is better than expected; in most cases, JWST will go deeper faster than expected. The telescope and instrument suite have demonstrated the sensitivity, stability, image quality, and spectral range that are necessary to transform our understanding of the cosmos through observations spanning from near-earth asteroids to the most distant galaxies.
△ Less
Submitted 10 April, 2023; v1 submitted 12 July, 2022;
originally announced July 2022.
-
Learning Interpretable, High-Performing Policies for Autonomous Driving
Authors:
Rohan Paleja,
Yaru Niu,
Andrew Silva,
Chace Ritchie,
Sugju Choi,
Matthew Gombolay
Abstract:
Gradient-based approaches in reinforcement learning (RL) have achieved tremendous success in learning policies for autonomous vehicles. While the performance of these approaches warrants real-world adoption, these policies lack interpretability, limiting deployability in the safety-critical and legally-regulated domain of autonomous driving (AD). AD requires interpretable and verifiable control po…
▽ More
Gradient-based approaches in reinforcement learning (RL) have achieved tremendous success in learning policies for autonomous vehicles. While the performance of these approaches warrants real-world adoption, these policies lack interpretability, limiting deployability in the safety-critical and legally-regulated domain of autonomous driving (AD). AD requires interpretable and verifiable control policies that maintain high performance. We propose Interpretable Continuous Control Trees (ICCTs), a tree-based model that can be optimized via modern, gradient-based, RL approaches to produce high-performing, interpretable policies. The key to our approach is a procedure for allowing direct optimization in a sparse decision-tree-like representation. We validate ICCTs against baselines across six domains, showing that ICCTs are capable of learning interpretable policy representations that parity or outperform baselines by up to 33% in AD scenarios while achieving a 300x-600x reduction in the number of policy parameters against deep learning baselines. Furthermore, we demonstrate the interpretability and utility of our ICCTs through a 14-car physical robot demonstration.
△ Less
Submitted 31 July, 2023; v1 submitted 4 February, 2022;
originally announced February 2022.
-
Artificial Intelligence, speech and language processing approaches to monitoring Alzheimer's Disease: a systematic review
Authors:
Sofia de la Fuente Garcia,
Craig Ritchie,
Saturnino Luz
Abstract:
Language is a valuable source of clinical information in Alzheimer's Disease, as it declines concurrently with neurodegeneration. Consequently, speech and language data have been extensively studied in connection with its diagnosis. This paper summarises current findings on the use of artificial intelligence, speech and language processing to predict cognitive decline in the context of Alzheimer's…
▽ More
Language is a valuable source of clinical information in Alzheimer's Disease, as it declines concurrently with neurodegeneration. Consequently, speech and language data have been extensively studied in connection with its diagnosis. This paper summarises current findings on the use of artificial intelligence, speech and language processing to predict cognitive decline in the context of Alzheimer's Disease, detailing current research procedures, highlighting their limitations and suggesting strategies to address them. We conducted a systematic review of original research between 2000 and 2019, registered in PROSPERO (reference CRD42018116606). An interdisciplinary search covered six databases on engineering (ACM and IEEE), psychology (PsycINFO), medicine (PubMed and Embase) and Web of Science. Bibliographies of relevant papers were screened until December 2019. From 3,654 search results 51 articles were selected against the eligibility criteria. Four tables summarise their findings: study details (aim, population, interventions, comparisons, methods and outcomes), data details (size, type, modalities, annotation, balance, availability and language of study), methodology (pre-processing, feature generation, machine learning, evaluation and results) and clinical applicability (research implications, clinical potential, risk of bias and strengths/limitations). While promising results are reported across nearly all 51 studies, very few have been implemented in clinical research or practice. We concluded that the main limitations of the field are poor standardisation, limited comparability of results, and a degree of disconnect between study aims and clinical applications. Attempts to close these gaps should support translation of future research into clinical practice.
△ Less
Submitted 12 October, 2020;
originally announced October 2020.
-
The influence of diversity on the measurement of functional impairment: An international validation of the Amsterdam IADL Questionnaire in 8 countries
Authors:
Mark A. Dubbelman,
Merike Verrijp,
David Facal,
Gonzalo Sánchez-Benavides,
Laura J. E. Brown,
Wiesje M. van der Flier,
Hanna Jokinen,
Athene Lee,
Iracema Leroi,
Cristina Lojo-Seoane,
Vuk Milosevic,
José Luís Molinuevo,
Arturo X. Pereiro Rozas,
Craig Ritchie,
Stephen Salloway,
Gemma Stringer,
Stelios Zygouris,
Bruno Dubois,
Stéphane Epelbaum,
Philip Scheltens,
Sietske A. M. Sikkes
Abstract:
INTRODUCTION: To understand the potential influence of diversity on the measurement of functional impairment in dementia, we aimed to investigate possible bias caused by age, gender, education, and cultural differences. METHODS: 3,571 individuals (67.1 {\pm} 9.5 years old, 44.7% female) from the Netherlands, Spain, France, United States, United Kingdom, Greece, Serbia and Finland were included. Fu…
▽ More
INTRODUCTION: To understand the potential influence of diversity on the measurement of functional impairment in dementia, we aimed to investigate possible bias caused by age, gender, education, and cultural differences. METHODS: 3,571 individuals (67.1 {\pm} 9.5 years old, 44.7% female) from the Netherlands, Spain, France, United States, United Kingdom, Greece, Serbia and Finland were included. Functional impairment was measured using the Amsterdam IADL Questionnaire. Item bias was assessed using differential item functioning (DIF) analysis. RESULTS: There were some differences in activity endorsement. A few items showed statistically significant DIF. However, there was no evidence of meaningful item bias: effect sizes were low (ΔR2 range 0-0.03). Impact on total scores was minimal. DISCUSSION: The results imply a limited bias for age, gender, education and culture in the measurement of functional impairment. This study provides an important step in recognizing the potential influence of diversity on primary outcomes in dementia research.
△ Less
Submitted 29 January, 2020; v1 submitted 4 September, 2019;
originally announced September 2019.
-
Scanning nano-spin ensemble microscope for nanoscale magnetic and thermal imaging
Authors:
Jean-Philippe Tetienne,
Alain Lombard,
David A. Simpson,
Cameron Ritchie,
Jianing Lu,
Paul Mulvaney,
Lloyd C. L. Hollenberg
Abstract:
Quantum sensors based on solid-state spins provide tremendous opportunities in a wide range of fields from basic physics and chemistry to biomedical imaging. However, integrating them into a scanning probe microscope to enable practical, nanoscale quantum imaging is a highly challenging task. Recently, the use of single spins in diamond in conjunction with atomic force microscopy techniques has al…
▽ More
Quantum sensors based on solid-state spins provide tremendous opportunities in a wide range of fields from basic physics and chemistry to biomedical imaging. However, integrating them into a scanning probe microscope to enable practical, nanoscale quantum imaging is a highly challenging task. Recently, the use of single spins in diamond in conjunction with atomic force microscopy techniques has allowed significant progress towards this goal, but generalisation of this approach has so far been impeded by long acquisition times or by the absence of simultaneous topographic information. Here we report on a scanning quantum probe microscope which solves both issues, by employing a nano-spin ensemble hosted in a nanodiamond. This approach provides up to an order of magnitude gain in acquisition time, whilst preserving sub-100 nm spatial resolution both for the quantum sensor and topographic images. We demonstrate two applications of this microscope. We first image nanoscale clusters of maghemite particles through both spin resonance spectroscopy and spin relaxometry, under ambient conditions. Our images reveal fast magnetic field fluctuations in addition to a static component, indicating the presence of both superparamagnetic and ferromagnetic particles. We next demonstrate a new imaging modality where the nano-spin ensemble is used as a thermometer. We use this technique to map the photo-induced heating generated by laser irradiation of a single gold nanoparticle in a fluid environment. This work paves the way towards new applications of quantum probe microscopy such as thermal/magnetic imaging of operating microelectronic devices and magnetic detection of ion channels in cell membranes.
△ Less
Submitted 30 December, 2015; v1 submitted 2 September, 2015;
originally announced September 2015.
