-
Trust, distrust, and appropriate reliance in (X)AI: a survey of empirical evaluation of user trust
Authors:
Roel Visser,
Tobias M. Peters,
Ingrid Scharlau,
Barbara Hammer
Abstract:
A current concern in the field of Artificial Intelligence (AI) is to ensure the trustworthiness of AI systems. The development of explainability methods is one prominent way to address this, which has often resulted in the assumption that the use of explainability will lead to an increase in the trust of users and wider society. However, the dynamics between explainability and trust are not well e…
▽ More
A current concern in the field of Artificial Intelligence (AI) is to ensure the trustworthiness of AI systems. The development of explainability methods is one prominent way to address this, which has often resulted in the assumption that the use of explainability will lead to an increase in the trust of users and wider society. However, the dynamics between explainability and trust are not well established and empirical investigations of their relation remain mixed or inconclusive. In this paper we provide a detailed description of the concepts of user trust and distrust in AI and their relation to appropriate reliance. For that we draw from the fields of machine learning, human-computer interaction, and the social sciences. Furthermore, we have created a survey of existing empirical studies that investigate the effects of AI systems and XAI methods on user (dis)trust. With clarifying the concepts and summarizing the empirical investigations, we aim to provide researchers, who examine user trust in AI, with an improved starting point for develo** user studies to measure and evaluate the user's attitude towards and reliance on AI systems.
△ Less
Submitted 4 December, 2023;
originally announced December 2023.
-
The Importance of Distrust in AI
Authors:
Tobias M. Peters,
Roel W. Visser
Abstract:
In recent years the use of Artificial Intelligence (AI) has become increasingly prevalent in a growing number of fields. As AI systems are being adopted in more high-stakes areas such as medicine and finance, ensuring that they are trustworthy is of increasing importance. A concern that is prominently addressed by the development and application of explainability methods, which are purported to in…
▽ More
In recent years the use of Artificial Intelligence (AI) has become increasingly prevalent in a growing number of fields. As AI systems are being adopted in more high-stakes areas such as medicine and finance, ensuring that they are trustworthy is of increasing importance. A concern that is prominently addressed by the development and application of explainability methods, which are purported to increase trust from its users and wider society. While an increase in trust may be desirable, an analysis of literature from different research fields shows that an exclusive focus on increasing trust may not be warranted. Something which is well exemplified by the recent development in AI chatbots, which while highly coherent tend to make up facts. In this contribution, we investigate the concepts of trust, trustworthiness, and user reliance.
In order to foster appropriate reliance on AI we need to prevent both disuse of these systems as well as overtrust. From our analysis of research on interpersonal trust, trust in automation, and trust in (X)AI, we identify the potential merit of the distinction between trust and distrust (in AI). We propose that alongside trust a healthy amount of distrust is of additional value for mitigating disuse and overtrust. We argue that by considering and evaluating both trust and distrust, we can ensure that users can rely appropriately on trustworthy AI, which can both be useful as well as fallible.
△ Less
Submitted 2 November, 2023; v1 submitted 25 July, 2023;
originally announced July 2023.
-
Image guidance in deep brain stimulation surgery to treat Parkinson's disease: a review
Authors:
Yiming Xiao,
Jonathan C. Lau,
Dimuthu Hemachandra,
Greydon Gilmore,
Ali R. Khan,
Terry M. Peters
Abstract:
Deep brain stimulation (DBS) is an effective therapy as an alternative to pharmaceutical treatments for Parkinson's disease (PD). Aside from factors such as instrumentation, treatment plans, and surgical protocols, the success of the procedure depends heavily on the accurate placement of the electrode within the optimal therapeutic targets while avoiding vital structures that can cause surgical co…
▽ More
Deep brain stimulation (DBS) is an effective therapy as an alternative to pharmaceutical treatments for Parkinson's disease (PD). Aside from factors such as instrumentation, treatment plans, and surgical protocols, the success of the procedure depends heavily on the accurate placement of the electrode within the optimal therapeutic targets while avoiding vital structures that can cause surgical complications and adverse neurologic effects. While specific surgical techniques for DBS can vary, interventional guidance with medical imaging has greatly contributed to the development, outcomes, and safety of the procedure. With rapid development in novel imaging techniques, computational methods, and surgical navigation software, as well as growing insights into the disease and mechanism of action of DBS, modern image guidance is expected to further enhance the capacity and efficacy of the procedure in treating PD. This article surveys the state-of-the-art techniques in image-guided DBS surgery to treat PD, and discuss their benefits and drawbacks, as well as future directions on the topic.
△ Less
Submitted 10 March, 2020;
originally announced March 2020.
-
Interactive-Automatic Segmentation and Modelling of the Mitral Valve
Authors:
Patrick Carnahan,
Olivia Ginty,
John Moore,
Andras Lasso,
Matthew A. Jolley,
Christian Herz,
Mehdi Eskandari,
Daniel Bainbridge,
Terry M. Peters
Abstract:
Mitral valve regurgitation is the most common valvular disease, affecting 10% of the population over 75 years old. Left untreated, patients with mitral valve regurgitation can suffer declining cardiac health until cardiac failure and death. Mitral valve repair is generally preferred over valve replacement. However, there is a direct correlation between the volume of cases performed and surgical ou…
▽ More
Mitral valve regurgitation is the most common valvular disease, affecting 10% of the population over 75 years old. Left untreated, patients with mitral valve regurgitation can suffer declining cardiac health until cardiac failure and death. Mitral valve repair is generally preferred over valve replacement. However, there is a direct correlation between the volume of cases performed and surgical outcomes, therefore there is a demand for the ability of surgeons to practice repairs on patient specific models in advance of surgery. This work demonstrates a semi-automated segmentation method to enable fast and accurate modelling of the mitral valve that captures patient-specific valve geometry. This modelling approach utilizes 3D active contours in a user-in-the-loop system which segments first the atrial blood pool, then the mitral leaflets. In a group of 15 mitral valve repair patients, valve segmentation and modelling attains an overall accuracy (mean absolute surface distance) of 1.40+-0.26 mm, and an accuracy of 1.01+-0.13 mm when only comparing the extracted leaflet surface proximal to the ultrasound probe. Thus this image-based segmentation tool has the potential to improve the workflow for extracting patient-specific mitral valve geometry for 3D modelling of the valve.
△ Less
Submitted 3 May, 2019;
originally announced May 2019.
-
A Continuous Max-Flow Approach to Cyclic Field Reconstruction
Authors:
John S. H. Baxter,
Jonathan McLeod,
Terry M. Peters
Abstract:
Reconstruction of an image from noisy data using Markov Random Field theory has been explored by both the graph-cuts and continuous max-flow community in the form of the Potts and Ishikawa models. However, neither model takes into account the particular cyclic topology of specific intensity types such as the hue in natural colour images, or the phase in complex valued MRI. This paper presents \tex…
▽ More
Reconstruction of an image from noisy data using Markov Random Field theory has been explored by both the graph-cuts and continuous max-flow community in the form of the Potts and Ishikawa models. However, neither model takes into account the particular cyclic topology of specific intensity types such as the hue in natural colour images, or the phase in complex valued MRI. This paper presents \textit{cyclic continuous max-flow} image reconstruction which models the intensity being reconstructed as having a fundamentally cyclic topology. This model complements the Ishikawa model in that it is designed with image reconstruction in mind, having the topology of the intensity space inherent in the model while being readily extendable to an arbitrary intensity resolution.
△ Less
Submitted 11 November, 2015;
originally announced November 2015.
-
Shape Complexes in Continuous Max-Flow Hierarchical Multi-Labeling Problems
Authors:
John S. H. Baxter,
**g Yuan,
Terry M. Peters
Abstract:
Although topological considerations amongst multiple labels have been previously investigated in the context of continuous max-flow image segmentation, similar investigations have yet to be made about shape considerations in a general and extendable manner. This paper presents shape complexes for segmentation, which capture more complex shapes by combining multiple labels and super-labels constrai…
▽ More
Although topological considerations amongst multiple labels have been previously investigated in the context of continuous max-flow image segmentation, similar investigations have yet to be made about shape considerations in a general and extendable manner. This paper presents shape complexes for segmentation, which capture more complex shapes by combining multiple labels and super-labels constrained by geodesic star convexity. Shape complexes combine geodesic star convexity constraints with hierarchical label organization, which together allow for more complex shapes to be represented. This framework avoids the use of co-ordinate system war** techniques to convert shape constraints into topological constraints, which may be ambiguous or ill-defined for certain segmentation problems.
△ Less
Submitted 15 October, 2015;
originally announced October 2015.
-
A Proximal Bregman Projection Approach to Continuous Max-Flow Problems Using Entropic Distances
Authors:
John S. H. Baxter,
Martin Rajchl,
**g Yuan,
Terry M. Peters
Abstract:
One issue limiting the adaption of large-scale multi-region segmentation is the sometimes prohibitive memory requirements. This is especially troubling considering advances in massively parallel computing and commercial graphics processing units because of their already limited memory compared to the current random access memory used in more traditional computation. To address this issue in the fi…
▽ More
One issue limiting the adaption of large-scale multi-region segmentation is the sometimes prohibitive memory requirements. This is especially troubling considering advances in massively parallel computing and commercial graphics processing units because of their already limited memory compared to the current random access memory used in more traditional computation. To address this issue in the field of continuous max-flow segmentation, we have developed a \textit{pseudo-flow} framework using the theory of Bregman proximal projections and entropic distances which implicitly represents flow variables between labels and designated source and sink nodes. This reduces the memory requirements for max-flow segmentation by approximately 20\% for Potts models and approximately 30\% for hierarchical max-flow (HMF) and directed acyclic graph max-flow (DAGMF) models. This represents a great improvement in the state-of-the-art in max-flow segmentation, allowing for much larger problems to be addressed and accelerated using commercially available graphics processing hardware.
△ Less
Submitted 30 January, 2015;
originally announced January 2015.
-
A Continuous Max-Flow Approach to Multi-Labeling Problems under Arbitrary Region Regularization
Authors:
John S. H. Baxter,
Martin Rajchl,
**g Yuan,
Terry M. Peters
Abstract:
The incorporation of region regularization into max-flow segmentation has traditionally focused on ordering and part-whole relationships. A side effect of the development of such models is that it constrained regularization only to those cases, rather than allowing for arbitrary region regularization. Directed Acyclic Graphical Max-Flow (DAGMF) segmentation overcomes these limitations by allowing…
▽ More
The incorporation of region regularization into max-flow segmentation has traditionally focused on ordering and part-whole relationships. A side effect of the development of such models is that it constrained regularization only to those cases, rather than allowing for arbitrary region regularization. Directed Acyclic Graphical Max-Flow (DAGMF) segmentation overcomes these limitations by allowing for the algorithm designer to specify an arbitrary directed acyclic graph to structure a max-flow segmentation. This allows for individual 'parts' to be a member of multiple distinct 'wholes.'
△ Less
Submitted 5 June, 2014; v1 submitted 5 May, 2014;
originally announced May 2014.
-
RANCOR: Non-Linear Image Registration with Total Variation Regularization
Authors:
Martin Rajchl,
John S. H. Baxter,
Wu Qiu,
Ali R. Khan,
Aaron Fenster,
Terry M. Peters,
**g Yuan
Abstract:
Optimization techniques have been widely used in deformable registration, allowing for the incorporation of similarity metrics with regularization mechanisms. These regularization mechanisms are designed to mitigate the effects of trivial solutions to ill-posed registration problems and to otherwise ensure the resulting deformation fields are well-behaved. This paper introduces a novel deformable…
▽ More
Optimization techniques have been widely used in deformable registration, allowing for the incorporation of similarity metrics with regularization mechanisms. These regularization mechanisms are designed to mitigate the effects of trivial solutions to ill-posed registration problems and to otherwise ensure the resulting deformation fields are well-behaved. This paper introduces a novel deformable registration algorithm, RANCOR, which uses iterative convexification to address deformable registration problems under total-variation regularization. Initial comparative results against four state-of-the-art registration algorithms are presented using the Internet Brain Segmentation Repository (IBSR) database.
△ Less
Submitted 9 April, 2014;
originally announced April 2014.
-
A Continuous Max-Flow Approach to General Hierarchical Multi-Labeling Problems
Authors:
John S. H. Baxter,
Martin Rajchl,
**g Yuan,
Terry M. Peters
Abstract:
Multi-region segmentation algorithms often have the onus of incorporating complex anatomical knowledge representing spatial or geometric relationships between objects, and general-purpose methods of addressing this knowledge in an optimization-based manner have thus been lacking. This paper presents Generalized Hierarchical Max-Flow (GHMF) segmentation, which captures simple anatomical part-whole…
▽ More
Multi-region segmentation algorithms often have the onus of incorporating complex anatomical knowledge representing spatial or geometric relationships between objects, and general-purpose methods of addressing this knowledge in an optimization-based manner have thus been lacking. This paper presents Generalized Hierarchical Max-Flow (GHMF) segmentation, which captures simple anatomical part-whole relationships in the form of an unconstrained hierarchy. Regularization can then be applied to both parts and wholes independently, allowing for spatial grou** and clustering of labels in a globally optimal convex optimization framework. For the purposes of ready integration into a variety of segmentation tasks, the hierarchies can be presented in run-time, allowing for the segmentation problem to be readily specified and alternatives explored without undue programming effort or recompilation.
△ Less
Submitted 5 June, 2014; v1 submitted 1 April, 2014;
originally announced April 2014.