-
Explainable Artificial Intelligence (XAI) 2.0: A Manifesto of Open Challenges and Interdisciplinary Research Directions
Authors:
Luca Longo,
Mario Brcic,
Federico Cabitza,
Jaesik Choi,
Roberto Confalonieri,
Javier Del Ser,
Riccardo Guidotti,
Yoichi Hayashi,
Francisco Herrera,
Andreas Holzinger,
Richard Jiang,
Hassan Khosravi,
Freddy Lecue,
Gianclaudio Malgieri,
Andrés Páez,
Wojciech Samek,
Johannes Schneider,
Timo Speith,
Simone Stumpf
Abstract:
As systems based on opaque Artificial Intelligence (AI) continue to flourish in diverse real-world applications, understanding these black box models has become paramount. In response, Explainable AI (XAI) has emerged as a field of research with practical and ethical benefits across various domains. This paper not only highlights the advancements in XAI and its application in real-world scenarios…
▽ More
As systems based on opaque Artificial Intelligence (AI) continue to flourish in diverse real-world applications, understanding these black box models has become paramount. In response, Explainable AI (XAI) has emerged as a field of research with practical and ethical benefits across various domains. This paper not only highlights the advancements in XAI and its application in real-world scenarios but also addresses the ongoing challenges within XAI, emphasizing the need for broader perspectives and collaborative efforts. We bring together experts from diverse fields to identify open problems, striving to synchronize research agendas and accelerate XAI in practical applications. By fostering collaborative discussion and interdisciplinary cooperation, we aim to propel XAI forward, contributing to its continued success. Our goal is to put forward a comprehensive proposal for advancing XAI. To achieve this goal, we present a manifesto of 27 open problems categorized into nine categories. These challenges encapsulate the complexities and nuances of XAI and offer a road map for future research. For each problem, we provide promising research directions in the hope of harnessing the collective intelligence of interested stakeholders.
△ Less
Submitted 30 October, 2023;
originally announced October 2023.
-
Mask-Mediator-Wrapper architecture as a Data Mesh driver
Authors:
Juraj Dončević,
Krešimir Fertalj,
Mario Brčić,
Mihael Kovač
Abstract:
The data mesh is a novel data management concept that emphasises the importance of a domain before technology. The concept is still in the early stages of development and many efforts to implement and use it are expected to have negative consequences for organizations due to a lack of technological guidelines and best practices. To mitigate the risk of negative outcomes this paper proposes the use…
▽ More
The data mesh is a novel data management concept that emphasises the importance of a domain before technology. The concept is still in the early stages of development and many efforts to implement and use it are expected to have negative consequences for organizations due to a lack of technological guidelines and best practices. To mitigate the risk of negative outcomes this paper proposes the use of the mask-mediator-wrapper architecture as a data mesh driver. The mask-mediator-wrapper architecture provides a set of prefabricated configurable components that provide basic functionalities which a data mesh requires. This paper shows how the two concepts are compatible in terms of functionality, data modelling, evolvability and aligned capabilities. A mask-mediator-wrapper driven data mesh facilitates: low-risk adoption trials, rapid prototy**, standardization, and a guarantee of evolvability.
△ Less
Submitted 10 September, 2022;
originally announced September 2022.
-
Mask-Mediator-Wrapper: A revised mediator-wrapper architecture for heterogeneous data source integration
Authors:
Juraj Dončević,
Krešimir Fertalj,
Mario Brčić,
Agneza Krajna
Abstract:
This paper deals with the mediator-wrapper architecture. It is an important architectural pattern that enables a more flexible and modular architecture in opposition to monolithic architectures for data source integration systems. This paper identifies certain realistic and concrete scenarios where the mediator-wrapper architecture underperforms. These issues are addressed with the extension of th…
▽ More
This paper deals with the mediator-wrapper architecture. It is an important architectural pattern that enables a more flexible and modular architecture in opposition to monolithic architectures for data source integration systems. This paper identifies certain realistic and concrete scenarios where the mediator-wrapper architecture underperforms. These issues are addressed with the extension of the architecture via the mask component type. The mask component is detailed so it can be reasoned about without prescribing a concrete programming language or paradigm. The benefits of the new mask-mediator-wrapper architecture are analytically proven in relevant scenarios. One of the applications of the new architecture is envisioned for modern data sources integration systems backing Big data processing.
△ Less
Submitted 25 August, 2022;
originally announced August 2022.
-
Prismal view of ethics
Authors:
Sarah Isufi,
Kristijan Poje,
Igor Vukobratovic,
Mario Brcic
Abstract:
We shall have a hard look at ethics and try to extract insights in the form of abstract properties that might become tools. We want to connect ethics to games, talk about the performance of ethics, introduce curiosity into the interplay between competing and coordinating in well-performing ethics, and offer a view of possible developments that could unify increasing aggregates of entities. All thi…
▽ More
We shall have a hard look at ethics and try to extract insights in the form of abstract properties that might become tools. We want to connect ethics to games, talk about the performance of ethics, introduce curiosity into the interplay between competing and coordinating in well-performing ethics, and offer a view of possible developments that could unify increasing aggregates of entities. All this is under a long shadow cast by computational complexity that is quite negative about games. This analysis is the first step toward finding modeling aspects that might be used in AI ethics for integrating modern AI systems into human society.
△ Less
Submitted 10 September, 2022; v1 submitted 26 May, 2022;
originally announced May 2022.
-
Explainability in reinforcement learning: perspective and position
Authors:
Agneza Krajna,
Mario Brcic,
Tomislav Lipic,
Juraj Doncevic
Abstract:
Artificial intelligence (AI) has been embedded into many aspects of people's daily lives and it has become normal for people to have AI make decisions for them. Reinforcement learning (RL) models increase the space of solvable problems with respect to other machine learning paradigms. Some of the most interesting applications are in situations with non-differentiable expected reward function, oper…
▽ More
Artificial intelligence (AI) has been embedded into many aspects of people's daily lives and it has become normal for people to have AI make decisions for them. Reinforcement learning (RL) models increase the space of solvable problems with respect to other machine learning paradigms. Some of the most interesting applications are in situations with non-differentiable expected reward function, operating in unknown or underdefined environment, as well as for algorithmic discovery that surpasses performance of any teacher, whereby agent learns from experimental experience through simple feedback. The range of applications and their social impact is vast, just to name a few: genomics, game-playing (chess, Go, etc.), general optimization, financial investment, governmental policies, self-driving cars, recommendation systems, etc. It is therefore essential to improve the trust and transparency of RL-based systems through explanations. Most articles dealing with explainability in artificial intelligence provide methods that concern supervised learning and there are very few articles dealing with this in the area of RL. The reasons for this are the credit assignment problem, delayed rewards, and the inability to assume that data is independently and identically distributed (i.i.d.). This position paper attempts to give a systematic overview of existing methods in the explainable RL area and propose a novel unified taxonomy, building and expanding on the existing ones. The position section describes pragmatic aspects of how explainability can be observed. The gap between the parties receiving and generating the explanation is especially emphasized. To reduce the gap and achieve honesty and truthfulness of explanations, we set up three pillars: proactivity, risk attitudes, and epistemological constraints. To this end, we illustrate our proposal on simple variants of the shortest path problem.
△ Less
Submitted 22 March, 2022;
originally announced March 2022.
-
Impossibility Results in AI: A Survey
Authors:
Mario Brcic,
Roman V. Yampolskiy
Abstract:
An impossibility theorem demonstrates that a particular problem or set of problems cannot be solved as described in the claim. Such theorems put limits on what is possible to do concerning artificial intelligence, especially the super-intelligent one. As such, these results serve as guidelines, reminders, and warnings to AI safety, AI policy, and governance researchers. These might enable solution…
▽ More
An impossibility theorem demonstrates that a particular problem or set of problems cannot be solved as described in the claim. Such theorems put limits on what is possible to do concerning artificial intelligence, especially the super-intelligent one. As such, these results serve as guidelines, reminders, and warnings to AI safety, AI policy, and governance researchers. These might enable solutions to some long-standing questions in the form of formalizing theories in the framework of constraint satisfaction without committing to one option. We strongly believe this to be the most prudent approach to long-term AI safety initiatives. In this paper, we have categorized impossibility theorems applicable to AI into five mechanism-based categories: deduction, indistinguishability, induction, tradeoffs, and intractability. We found that certain theorems are too specific or have implicit assumptions that limit application. Also, we added new results (theorems) such as the unfairness of explainability, the first explainability-related result in the induction category. The remaining results deal with misalignment between the clones and put a limit to the self-awareness of agents. We concluded that deductive impossibilities deny 100%-guarantees for security. In the end, we give some ideas that hold potential in explainability, controllability, value alignment, ethics, and group decision-making. They can be deepened by further investigation.
△ Less
Submitted 19 February, 2022; v1 submitted 1 September, 2021;
originally announced September 2021.
-
AI safety: state of the field through quantitative lens
Authors:
Mislav Juric,
Agneza Sandic,
Mario Brcic
Abstract:
Last decade has seen major improvements in the performance of artificial intelligence which has driven wide-spread applications. Unforeseen effects of such mass-adoption has put the notion of AI safety into the public eye. AI safety is a relatively new field of research focused on techniques for building AI beneficial for humans. While there exist survey papers for the field of AI safety, there is…
▽ More
Last decade has seen major improvements in the performance of artificial intelligence which has driven wide-spread applications. Unforeseen effects of such mass-adoption has put the notion of AI safety into the public eye. AI safety is a relatively new field of research focused on techniques for building AI beneficial for humans. While there exist survey papers for the field of AI safety, there is a lack of a quantitative look at the research being conducted. The quantitative aspect gives a data-driven insight about the emerging trends, knowledge gaps and potential areas for future research. In this paper, bibliometric analysis of the literature finds significant increase in research activity since 2015. Also, the field is so new that most of the technical issues are open, including: explainability with its long-term utility, and value alignment which we have identified as the most important long-term research topic. Equally, there is a severe lack of research into concrete policies regarding AI. As we expect AI to be the one of the main driving forces of changes in society, AI safety is the field under which we need to decide the direction of humanity's future.
△ Less
Submitted 9 July, 2020; v1 submitted 12 February, 2020;
originally announced February 2020.
