-
Hacking a surrogate model approach to XAI
Authors:
Alexander Wilhelm,
Katharina A. Zweig
Abstract:
In recent years, the number of new applications for highly complex AI systems has risen significantly. Algorithmic decision-making systems (ADMs) are one of such applications, where an AI system replaces the decision-making process of a human expert. As one approach to ensure fairness and transparency of such systems, explainable AI (XAI) has become more important. One variant to achieve explainab…
▽ More
In recent years, the number of new applications for highly complex AI systems has risen significantly. Algorithmic decision-making systems (ADMs) are one of such applications, where an AI system replaces the decision-making process of a human expert. As one approach to ensure fairness and transparency of such systems, explainable AI (XAI) has become more important. One variant to achieve explainability are surrogate models, i.e., the idea to train a new simpler machine learning model based on the input-output-relationship of a black box model. The simpler machine learning model could, for example, be a decision tree, which is thought to be intuitively understandable by humans. However, there is not much insight into how well the surrogate model approximates the black box.
Our main assumption is that a good surrogate model approach should be able to bring such a discriminating behavior to the attention of humans; prior to our research we assumed that a surrogate decision tree would identify such a pattern on one of its first levels. However, in this article we show that even if the discriminated subgroup - while otherwise being the same in all categories - does not get a single positive decision from the black box ADM system, the corresponding question of group membership can be pushed down onto a level as low as wanted by the operator of the system.
We then generalize this finding to pinpoint the exact level of the tree on which the discriminating question is asked and show that in a more realistic scenario, where discrimination only occurs to some fraction of the disadvantaged group, it is even more feasible to hide such discrimination.
Our approach can be generalized easily to other surrogate models.
△ Less
Submitted 24 June, 2024;
originally announced June 2024.
-
Quantitative study about the estimated impact of the AI Act
Authors:
Marc P. Hauer,
Tobias D Krafft,
Dr. Andreas Sesing-Wagenpfeil,
Prof. Katharina Zweig
Abstract:
With the Proposal for a Regulation laying down harmonised rules on Artificial Intelligence (AI Act) the European Union provides the first regulatory document that applies to the entire complex of AI systems. While some fear that the regulation leaves too much room for interpretation and thus bring little benefit to society, others expect that the regulation is too restrictive and, thus, blocks pro…
▽ More
With the Proposal for a Regulation laying down harmonised rules on Artificial Intelligence (AI Act) the European Union provides the first regulatory document that applies to the entire complex of AI systems. While some fear that the regulation leaves too much room for interpretation and thus bring little benefit to society, others expect that the regulation is too restrictive and, thus, blocks progress and innovation, as well as hinders the economic success of companies within the EU. Without a systematic approach, it is difficult to assess how it will actually impact the AI landscape. In this paper, we suggest a systematic approach that we applied on the initial draft of the AI Act that has been released in April 2021. We went through several iterations of compiling the list of AI products and projects in and from Germany, which the Lernende Systeme platform lists, and then classified them according to the AI Act together with experts from the fields of computer science and law. Our study shows a need for more concrete formulation, since for some provisions it is often unclear whether they are applicable in a specific case or not. Apart from that, it turns out that only about 30\% of the AI systems considered would be regulated by the AI Act, the rest would be classified as low-risk. However, as the database is not representative, the results only provide a first assessment. The process presented can be applied to any collections, and also repeated when regulations are about to change. This allows fears of over- or under-regulation to be investigated before the regulations comes into effect.
△ Less
Submitted 29 March, 2023;
originally announced April 2023.
-
Diversity in News Recommendations
Authors:
Abraham Bernstein,
Claes de Vreese,
Natali Helberger,
Wolfgang Schulz,
Katharina Zweig,
Christian Baden,
Michael A. Beam,
Marc P. Hauer,
Lucien Heitz,
Pascal Jürgens,
Christian Katzenbach,
Benjamin Kille,
Beate Klimkiewicz,
Wiebke Loosen,
Judith Moeller,
Goran Radanovic,
Guy Shani,
Nava Tintarev,
Suzanne Tolmeijer,
Wouter van Atteveldt,
Sanne Vrijenhoek,
Theresa Zueger
Abstract:
News diversity in the media has for a long time been a foundational and uncontested basis for ensuring that the communicative needs of individuals and society at large are met. Today, people increasingly rely on online content and recommender systems to consume information challenging the traditional concept of news diversity. In addition, the very concept of diversity, which differs between disci…
▽ More
News diversity in the media has for a long time been a foundational and uncontested basis for ensuring that the communicative needs of individuals and society at large are met. Today, people increasingly rely on online content and recommender systems to consume information challenging the traditional concept of news diversity. In addition, the very concept of diversity, which differs between disciplines, will need to be re-evaluated requiring a interdisciplinary investigation, which requires a new level of mutual cooperation between computer scientists, social scientists, and legal scholars. Based on the outcome of a multidisciplinary workshop, we have the following recommendations, directed at researchers, funders, legislators, regulators, and the media industry: 1. Do more research on news recommenders and diversity. 2. Create a safe harbor for academic research with industry data. 3. Optimize the role of public values in news recommenders. 4. Create a meaningful governance framework. 5. Fund a joint lab to spearhead the needed interdisciplinary research, boost practical innovation, develop. reference solutions, and transfer insights into practice.
△ Less
Submitted 25 May, 2021; v1 submitted 19 May, 2020;
originally announced May 2020.
-
Collaborative Interactive Learning -- A clarification of terms and a differentiation from other research fields
Authors:
Tom Hanika,
Marek Herde,
Jochen Kuhn,
Jan Marco Leimeister,
Paul Lukowicz,
Sarah Oeste-Reiß,
Albrecht Schmidt,
Bernhard Sick,
Gerd Stumme,
Sven Tomforde,
Katharina Anna Zweig
Abstract:
The field of collaborative interactive learning (CIL) aims at develo** and investigating the technological foundations for a new generation of smart systems that support humans in their everyday life. While the concept of CIL has already been carved out in detail (including the fields of dedicated CIL and opportunistic CIL) and many research objectives have been stated, there is still the need t…
▽ More
The field of collaborative interactive learning (CIL) aims at develo** and investigating the technological foundations for a new generation of smart systems that support humans in their everyday life. While the concept of CIL has already been carved out in detail (including the fields of dedicated CIL and opportunistic CIL) and many research objectives have been stated, there is still the need to clarify some terms such as information, knowledge, and experience in the context of CIL and to differentiate CIL from recent and ongoing research in related fields such as active learning, collaborative learning, and others. Both aspects are addressed in this paper.
△ Less
Submitted 16 May, 2019;
originally announced May 2019.
-
What did you see? Personalization, regionalization and the question of the filter bubble in Google's search engine
Authors:
Tobias D. Krafft,
Michael Gamer,
Katharina A. Zweig
Abstract:
This report analyzes the Google search results from more than 1,500 volunteer data donors who, in the five weeks leading up to the federal election on September 24th, 2017, automatically searched Google for 16 predefined names of political parties and politicians every four hours. It is based on an adjusted database consisting of more than 8,000,000 data records, which were generated in the contex…
▽ More
This report analyzes the Google search results from more than 1,500 volunteer data donors who, in the five weeks leading up to the federal election on September 24th, 2017, automatically searched Google for 16 predefined names of political parties and politicians every four hours. It is based on an adjusted database consisting of more than 8,000,000 data records, which were generated in the context of the research project "#Datenspende: Google und die Bundestagswahl 2017" and sent to us for evaluation. The #Datenspende project was commissioned by six state media authorities. Spiegel Online acted as a media partner. Focal points of the present study are, i.a., the question of the degree of personalization of search results, the proportion of regionalization and the risk of algorithm-based filter bubble formation or reinforcement by the leader in the search engine market.
△ Less
Submitted 28 December, 2018;
originally announced December 2018.
-
Link Classification and Tie Strength Ranking in Online Social Networks with Exogenous Interaction Networks
Authors:
Mohammed Abufouda,
Katharina A. Zweig
Abstract:
Online social networks (OSNs) have become the main medium for connecting people, sharing knowledge and information, and for communication. The social connections between people using these OSNs are formed as virtual links (e.g., friendship and following connections) that connect people. These links are the heart of today's OSNs as they facilitate all of the activities that the members of a social…
▽ More
Online social networks (OSNs) have become the main medium for connecting people, sharing knowledge and information, and for communication. The social connections between people using these OSNs are formed as virtual links (e.g., friendship and following connections) that connect people. These links are the heart of today's OSNs as they facilitate all of the activities that the members of a social network can do. However, many of these networks suffer from noisy links, i.e., links that do not reflect a real relationship or links that have a low intensity, that change the structure of the network and prevent accurate analysis of these networks. Hence, a process for assessing and ranking the links in a social network is crucial in order to sustain a healthy and real network. Here, we define link assessment as the process of identifying noisy and non-noisy links in a network. In this paper, we address the problem of link assessment and link ranking in social networks using external interaction networks. In addition to a friendship social network, additional exogenous interaction networks are utilized to make the assessment process more meaningful. We employed machine learning classifiers for assessing and ranking the links in the social network of interest using the data from exogenous interaction networks. The method was tested with two different datasets, each containing the social network of interest, with the ground truth, along with the exogenous interaction networks. The results show that it is possible to effectively assess the links of a social network using only the structure of a single network of the exogenous interaction networks, and also using the structure of the whole set of exogenous interaction networks. The experiments showed that some classifiers do better than others regarding both link classification and link ranking.
△ Less
Submitted 14 August, 2017;
originally announced August 2017.
-
Most central or least central? How much modeling decisions influence a node's centrality ranking in multiplex networks
Authors:
Sude Tavassoli,
Katharina Anna Zweig
Abstract:
To understand a node's centrality in a multiplex network, its centrality values in all the layers of the network can be aggregated. This requires a normalization of the values, to allow their meaningful comparison and aggregation over networks with different sizes and orders. The concrete choices of such preprocessing steps like normalization and aggregation are almost never discussed in network a…
▽ More
To understand a node's centrality in a multiplex network, its centrality values in all the layers of the network can be aggregated. This requires a normalization of the values, to allow their meaningful comparison and aggregation over networks with different sizes and orders. The concrete choices of such preprocessing steps like normalization and aggregation are almost never discussed in network analytic papers. In this paper, we show that even sticking to the most simple centrality index (the degree) but using different, classic choices of normalization and aggregation strategies, can turn a node from being among the most central to being among the least central. We present our results by using an aggregation operator which scales between different, classic aggregation strategies based on three multiplex networks. We also introduce a new visualization and characterization of a node's sensitivity to the choice of a normalization and aggregation strategy in multiplex networks. The observed high sensitivity of single nodes to the specific choice of aggregation and normalization strategies is of strong importance, especially for all kinds of intelligence-analytic software as it questions the interpretations of the findings.
△ Less
Submitted 17 June, 2016;
originally announced June 2016.
-
Analyzing the activity of a person in a chat by combining network analysis and fuzzy logic
Authors:
Sude Tavassoli,
Katharina Anna Zweig
Abstract:
Chat-log data that contains information about sender and receiver of the statements sent around in the chat can be readily turned into a directed temporal multi-network representation. In the resulting network, the activity of a chat member can, for example, be operationalized as his degree (number of distinct interaction partners) or his strength (total number of interactions). However, the data…
▽ More
Chat-log data that contains information about sender and receiver of the statements sent around in the chat can be readily turned into a directed temporal multi-network representation. In the resulting network, the activity of a chat member can, for example, be operationalized as his degree (number of distinct interaction partners) or his strength (total number of interactions). However, the data itself contains more information that is not readily representable in the network, e.g., the total number of words used by a member or the reaction time to what the members said. As degree and strength, these values can be seen as a way to operationalize the idea of activity of a chat-log member. This paper deals with the question of how the overall activity of a member can be assessed, given multiple and probably opposing criteria by using a fuzzy operator. We then present a new way of visualizing the results and show how to apply it to the network representation of chat-log data. Finally, we discuss how this approach can be used to deal with other conflicting situations, like the different rankings produced by different centrality indices.
△ Less
Submitted 16 July, 2015; v1 submitted 14 July, 2015;
originally announced July 2015.
-
What makes a phase transition? Analysis of the random satisfiability problem
Authors:
K. A. Zweig,
G. Palla,
T. Vicsek
Abstract:
In the last 30 years it was found that many combinatorial systems undergo phase transitions. One of the most important examples of these can be found among the random k-satisfiability problems (often referred to as k-SAT), asking whether there exists an assignment of Boolean values satisfying a Boolean formula composed of clauses with k random variables each. The random 3-SAT problem is reported…
▽ More
In the last 30 years it was found that many combinatorial systems undergo phase transitions. One of the most important examples of these can be found among the random k-satisfiability problems (often referred to as k-SAT), asking whether there exists an assignment of Boolean values satisfying a Boolean formula composed of clauses with k random variables each. The random 3-SAT problem is reported to show various phase transitions at different critical values of the ratio of the number of clauses to the number of variables. The most famous of these occurs when the probability of finding a satisfiable instance suddenly drops from 1 to 0. This transition is associated with a rise in the hardness of the problem, but until now the correlation between any of the proposed phase transitions and the hardness is not totally clear. In this paper we will first show numerically that the number of solutions universally follows a lognormal distribution, thereby explaining the puzzling question of why the number of solutions is still exponential at the critical point. Moreover we provide evidence that the hardness of the closely related problem of counting the total number of solutions does not show any phase transition-like behavior. This raises the question of whether the probability of finding a satisfiable instance is really an order parameter of a phase transition or whether it is more likely to just show a simple sharp threshold phenomenon. More generally, this paper aims at starting a discussion where a simple sharp threshold phenomenon turns into a genuine phase transition.
△ Less
Submitted 1 February, 2010;
originally announced February 2010.
