-
Trust, distrust, and appropriate reliance in (X)AI: a survey of empirical evaluation of user trust
Authors:
Roel Visser,
Tobias M. Peters,
Ingrid Scharlau,
Barbara Hammer
Abstract:
A current concern in the field of Artificial Intelligence (AI) is to ensure the trustworthiness of AI systems. The development of explainability methods is one prominent way to address this, which has often resulted in the assumption that the use of explainability will lead to an increase in the trust of users and wider society. However, the dynamics between explainability and trust are not well e…
▽ More
A current concern in the field of Artificial Intelligence (AI) is to ensure the trustworthiness of AI systems. The development of explainability methods is one prominent way to address this, which has often resulted in the assumption that the use of explainability will lead to an increase in the trust of users and wider society. However, the dynamics between explainability and trust are not well established and empirical investigations of their relation remain mixed or inconclusive. In this paper we provide a detailed description of the concepts of user trust and distrust in AI and their relation to appropriate reliance. For that we draw from the fields of machine learning, human-computer interaction, and the social sciences. Furthermore, we have created a survey of existing empirical studies that investigate the effects of AI systems and XAI methods on user (dis)trust. With clarifying the concepts and summarizing the empirical investigations, we aim to provide researchers, who examine user trust in AI, with an improved starting point for develo** user studies to measure and evaluate the user's attitude towards and reliance on AI systems.
△ Less
Submitted 4 December, 2023;
originally announced December 2023.
-
The Importance of Distrust in AI
Authors:
Tobias M. Peters,
Roel W. Visser
Abstract:
In recent years the use of Artificial Intelligence (AI) has become increasingly prevalent in a growing number of fields. As AI systems are being adopted in more high-stakes areas such as medicine and finance, ensuring that they are trustworthy is of increasing importance. A concern that is prominently addressed by the development and application of explainability methods, which are purported to in…
▽ More
In recent years the use of Artificial Intelligence (AI) has become increasingly prevalent in a growing number of fields. As AI systems are being adopted in more high-stakes areas such as medicine and finance, ensuring that they are trustworthy is of increasing importance. A concern that is prominently addressed by the development and application of explainability methods, which are purported to increase trust from its users and wider society. While an increase in trust may be desirable, an analysis of literature from different research fields shows that an exclusive focus on increasing trust may not be warranted. Something which is well exemplified by the recent development in AI chatbots, which while highly coherent tend to make up facts. In this contribution, we investigate the concepts of trust, trustworthiness, and user reliance.
In order to foster appropriate reliance on AI we need to prevent both disuse of these systems as well as overtrust. From our analysis of research on interpersonal trust, trust in automation, and trust in (X)AI, we identify the potential merit of the distinction between trust and distrust (in AI). We propose that alongside trust a healthy amount of distrust is of additional value for mitigating disuse and overtrust. We argue that by considering and evaluating both trust and distrust, we can ensure that users can rely appropriately on trustworthy AI, which can both be useful as well as fallible.
△ Less
Submitted 2 November, 2023; v1 submitted 25 July, 2023;
originally announced July 2023.
-
A Continuous Max-Flow Approach to Cyclic Field Reconstruction
Authors:
John S. H. Baxter,
Jonathan McLeod,
Terry M. Peters
Abstract:
Reconstruction of an image from noisy data using Markov Random Field theory has been explored by both the graph-cuts and continuous max-flow community in the form of the Potts and Ishikawa models. However, neither model takes into account the particular cyclic topology of specific intensity types such as the hue in natural colour images, or the phase in complex valued MRI. This paper presents \tex…
▽ More
Reconstruction of an image from noisy data using Markov Random Field theory has been explored by both the graph-cuts and continuous max-flow community in the form of the Potts and Ishikawa models. However, neither model takes into account the particular cyclic topology of specific intensity types such as the hue in natural colour images, or the phase in complex valued MRI. This paper presents \textit{cyclic continuous max-flow} image reconstruction which models the intensity being reconstructed as having a fundamentally cyclic topology. This model complements the Ishikawa model in that it is designed with image reconstruction in mind, having the topology of the intensity space inherent in the model while being readily extendable to an arbitrary intensity resolution.
△ Less
Submitted 11 November, 2015;
originally announced November 2015.
-
Shape Complexes in Continuous Max-Flow Hierarchical Multi-Labeling Problems
Authors:
John S. H. Baxter,
**g Yuan,
Terry M. Peters
Abstract:
Although topological considerations amongst multiple labels have been previously investigated in the context of continuous max-flow image segmentation, similar investigations have yet to be made about shape considerations in a general and extendable manner. This paper presents shape complexes for segmentation, which capture more complex shapes by combining multiple labels and super-labels constrai…
▽ More
Although topological considerations amongst multiple labels have been previously investigated in the context of continuous max-flow image segmentation, similar investigations have yet to be made about shape considerations in a general and extendable manner. This paper presents shape complexes for segmentation, which capture more complex shapes by combining multiple labels and super-labels constrained by geodesic star convexity. Shape complexes combine geodesic star convexity constraints with hierarchical label organization, which together allow for more complex shapes to be represented. This framework avoids the use of co-ordinate system war** techniques to convert shape constraints into topological constraints, which may be ambiguous or ill-defined for certain segmentation problems.
△ Less
Submitted 15 October, 2015;
originally announced October 2015.
-
A Proximal Bregman Projection Approach to Continuous Max-Flow Problems Using Entropic Distances
Authors:
John S. H. Baxter,
Martin Rajchl,
**g Yuan,
Terry M. Peters
Abstract:
One issue limiting the adaption of large-scale multi-region segmentation is the sometimes prohibitive memory requirements. This is especially troubling considering advances in massively parallel computing and commercial graphics processing units because of their already limited memory compared to the current random access memory used in more traditional computation. To address this issue in the fi…
▽ More
One issue limiting the adaption of large-scale multi-region segmentation is the sometimes prohibitive memory requirements. This is especially troubling considering advances in massively parallel computing and commercial graphics processing units because of their already limited memory compared to the current random access memory used in more traditional computation. To address this issue in the field of continuous max-flow segmentation, we have developed a \textit{pseudo-flow} framework using the theory of Bregman proximal projections and entropic distances which implicitly represents flow variables between labels and designated source and sink nodes. This reduces the memory requirements for max-flow segmentation by approximately 20\% for Potts models and approximately 30\% for hierarchical max-flow (HMF) and directed acyclic graph max-flow (DAGMF) models. This represents a great improvement in the state-of-the-art in max-flow segmentation, allowing for much larger problems to be addressed and accelerated using commercially available graphics processing hardware.
△ Less
Submitted 30 January, 2015;
originally announced January 2015.
-
A Continuous Max-Flow Approach to Multi-Labeling Problems under Arbitrary Region Regularization
Authors:
John S. H. Baxter,
Martin Rajchl,
**g Yuan,
Terry M. Peters
Abstract:
The incorporation of region regularization into max-flow segmentation has traditionally focused on ordering and part-whole relationships. A side effect of the development of such models is that it constrained regularization only to those cases, rather than allowing for arbitrary region regularization. Directed Acyclic Graphical Max-Flow (DAGMF) segmentation overcomes these limitations by allowing…
▽ More
The incorporation of region regularization into max-flow segmentation has traditionally focused on ordering and part-whole relationships. A side effect of the development of such models is that it constrained regularization only to those cases, rather than allowing for arbitrary region regularization. Directed Acyclic Graphical Max-Flow (DAGMF) segmentation overcomes these limitations by allowing for the algorithm designer to specify an arbitrary directed acyclic graph to structure a max-flow segmentation. This allows for individual 'parts' to be a member of multiple distinct 'wholes.'
△ Less
Submitted 5 June, 2014; v1 submitted 5 May, 2014;
originally announced May 2014.
-
RANCOR: Non-Linear Image Registration with Total Variation Regularization
Authors:
Martin Rajchl,
John S. H. Baxter,
Wu Qiu,
Ali R. Khan,
Aaron Fenster,
Terry M. Peters,
**g Yuan
Abstract:
Optimization techniques have been widely used in deformable registration, allowing for the incorporation of similarity metrics with regularization mechanisms. These regularization mechanisms are designed to mitigate the effects of trivial solutions to ill-posed registration problems and to otherwise ensure the resulting deformation fields are well-behaved. This paper introduces a novel deformable…
▽ More
Optimization techniques have been widely used in deformable registration, allowing for the incorporation of similarity metrics with regularization mechanisms. These regularization mechanisms are designed to mitigate the effects of trivial solutions to ill-posed registration problems and to otherwise ensure the resulting deformation fields are well-behaved. This paper introduces a novel deformable registration algorithm, RANCOR, which uses iterative convexification to address deformable registration problems under total-variation regularization. Initial comparative results against four state-of-the-art registration algorithms are presented using the Internet Brain Segmentation Repository (IBSR) database.
△ Less
Submitted 9 April, 2014;
originally announced April 2014.
-
A Continuous Max-Flow Approach to General Hierarchical Multi-Labeling Problems
Authors:
John S. H. Baxter,
Martin Rajchl,
**g Yuan,
Terry M. Peters
Abstract:
Multi-region segmentation algorithms often have the onus of incorporating complex anatomical knowledge representing spatial or geometric relationships between objects, and general-purpose methods of addressing this knowledge in an optimization-based manner have thus been lacking. This paper presents Generalized Hierarchical Max-Flow (GHMF) segmentation, which captures simple anatomical part-whole…
▽ More
Multi-region segmentation algorithms often have the onus of incorporating complex anatomical knowledge representing spatial or geometric relationships between objects, and general-purpose methods of addressing this knowledge in an optimization-based manner have thus been lacking. This paper presents Generalized Hierarchical Max-Flow (GHMF) segmentation, which captures simple anatomical part-whole relationships in the form of an unconstrained hierarchy. Regularization can then be applied to both parts and wholes independently, allowing for spatial grou** and clustering of labels in a globally optimal convex optimization framework. For the purposes of ready integration into a variety of segmentation tasks, the hierarchies can be presented in run-time, allowing for the segmentation problem to be readily specified and alternatives explored without undue programming effort or recompilation.
△ Less
Submitted 5 June, 2014; v1 submitted 1 April, 2014;
originally announced April 2014.
