-
MRxaI: Black-Box Explainability for Image Classifiers in a Medical Setting
Authors:
Nathan Blake,
Hana Chockler,
David A. Kelly,
Santiago Calderon Pena,
Akchunya Chanchal
Abstract:
Existing tools for explaining the output of image classifiers can be divided into white-box, which rely on access to the model internals, and black-box, agnostic to the model. As the usage of AI in the medical domain grows, so too does the usage of explainability tools. Existing work on medical image explanations focuses on white-box tools, such as gradcam. However, there are clear advantages to s…
▽ More
Existing tools for explaining the output of image classifiers can be divided into white-box, which rely on access to the model internals, and black-box, agnostic to the model. As the usage of AI in the medical domain grows, so too does the usage of explainability tools. Existing work on medical image explanations focuses on white-box tools, such as gradcam. However, there are clear advantages to switching to a black-box tool, including the ability to use it with any classifier and the wide selection of black-box tools available. On standard images, black-box tools are as precise as white-box. In this paper we compare the performance of several black-box methods against gradcam on a brain cancer MRI dataset. We demonstrate that most black-box tools are not suitable for explaining medical image classifications and present a detailed analysis of the reasons for their shortcomings. We also show that one black-box tool, a causal explainability-based rex, performs as well as \gradcam.
△ Less
Submitted 24 November, 2023;
originally announced November 2023.
-
Finding polarised communities and tracking information diffusion on Twitter: The Irish Abortion Referendum
Authors:
Caroline Pena,
Pádraig MacCarron,
David J. P. O'Sullivan
Abstract:
The analysis of social networks enables the understanding of social interactions, polarisation of ideas, and the spread of information and therefore plays an important role in society. We use Twitter data - as it is a popular venue for the expression of opinion and dissemination of information - to identify opposing sides of a debate and, importantly, to observe how information spreads between the…
▽ More
The analysis of social networks enables the understanding of social interactions, polarisation of ideas, and the spread of information and therefore plays an important role in society. We use Twitter data - as it is a popular venue for the expression of opinion and dissemination of information - to identify opposing sides of a debate and, importantly, to observe how information spreads between these groups in our current polarised climate.
To achieve this, we collected over 688,000 Tweets from the Irish Abortion Referendum of 2018 to build a conversation network from users mentions with sentiment-based homophily. From this network, community detection methods allow us to isolate yes- or no-aligned supporters with high accuracy (90.9%). We supplement this by tracking how information cascades spread via over 31,000 retweet-cascades. We found that very little information spread between polarised communities. This provides a valuable methodology for extracting and studying information diffusion on large networks by isolating ideologically polarised groups and exploring the propagation of information within and between these groups.
△ Less
Submitted 15 November, 2023;
originally announced November 2023.
-
Grey Literature in Software Engineering: A Critical Review
Authors:
Fernando Kamei,
Igor Wiese,
Crescencio Lima,
Ivanilton Polato,
Vilmar Nepomuceno,
Waldemar Ferreira,
Márcio Ribeiro,
Carolline Pena,
Bruno Cartaxo,
Gustavo Pinto,
Sérgio Soares
Abstract:
Context: Grey Literature (GL) recently has grown in Software Engineering (SE) research since the increased use of online communication channels by software engineers. However, there is still a limited understanding of how SE research is taking advantage of GL. Objective: This research aimed to understand how SE researchers use GL in their secondary studies. Method: We conducted a tertiary study of…
▽ More
Context: Grey Literature (GL) recently has grown in Software Engineering (SE) research since the increased use of online communication channels by software engineers. However, there is still a limited understanding of how SE research is taking advantage of GL. Objective: This research aimed to understand how SE researchers use GL in their secondary studies. Method: We conducted a tertiary study of studies published between 2011 and 2018 in high-quality software engineering conferences and journals. We then applied qualitative and quantitative analysis to investigate 446 potential studies. Results: From the 446 selected studies, 126 studies cited GL but only 95 of those used GL to answer a specific research question representing almost 21% of all the 446 secondary studies. Interestingly, we identified that few studies employed specific search mechanisms and used additional criteria for assessing GL. Moreover, by the time we conducted this research, 49% of the GL URLs are not working anymore. Based on our findings, we discuss some challenges in using GL and potential mitigation plans. Conclusion: In this paper, we summarized the last 10 years of software engineering research that uses GL, showing that GL has been essential for bringing practical new perspectives that are scarce in traditional literature. By drawing the current landscape of use, we also raise some awareness of related challenges (and strategies to deal with them).
△ Less
Submitted 12 May, 2021; v1 submitted 27 April, 2021;
originally announced April 2021.
-
An analysis of Reinforcement Learning applied to Coach task in IEEE Very Small Size Soccer
Authors:
Carlos H. C. Pena,
Mateus G. Machado,
Mariana S. Barros,
José D. P. Silva,
Lucas D. Maciel,
Tsang Ing Ren,
Edna N. S. Barros,
Pedro H. M. Braga,
Hansenclever F. Bassani
Abstract:
The IEEE Very Small Size Soccer (VSSS) is a robot soccer competition in which two teams of three small robots play against each other. Traditionally, a deterministic coach agent will choose the most suitable strategy and formation for each adversary's strategy. Therefore, the role of a coach is of great importance to the game. In this sense, this paper proposes an end-to-end approach for the coach…
▽ More
The IEEE Very Small Size Soccer (VSSS) is a robot soccer competition in which two teams of three small robots play against each other. Traditionally, a deterministic coach agent will choose the most suitable strategy and formation for each adversary's strategy. Therefore, the role of a coach is of great importance to the game. In this sense, this paper proposes an end-to-end approach for the coaching task based on Reinforcement Learning (RL). The proposed system processes the information during the simulated matches to learn an optimal policy that chooses the current formation, depending on the opponent and game conditions. We trained two RL policies against three different teams (balanced, offensive, and heavily offensive) in a simulated environment. Our results were assessed against one of the top teams of the VSSS league, showing promising results after achieving a win/loss ratio of approximately 2.0.
△ Less
Submitted 23 November, 2020;
originally announced November 2020.
-
Deep Learning on Retina Images as Screening Tool for Diagnostic Decision Support
Authors:
Maria Camila Alvarez Trivino,
Jeremie Despraz,
Jesus Alfonso Lopez Sotelo,
Carlos Andres Pena
Abstract:
In this project, we developed a deep learning system applied to human retina images for medical diagnostic decision support. The retina images were provided by EyePACS. These images were used in the framework of a Kaggle contest, whose purpose to identify diabetic retinopathy signs through an automatic detection system. Using as inspiration one of the solutions proposed in the contest, we implemen…
▽ More
In this project, we developed a deep learning system applied to human retina images for medical diagnostic decision support. The retina images were provided by EyePACS. These images were used in the framework of a Kaggle contest, whose purpose to identify diabetic retinopathy signs through an automatic detection system. Using as inspiration one of the solutions proposed in the contest, we implemented a model that successfully detects diabetic retinopathy from retina images. After a carefully designed preprocessing, the images were used as input to a deep convolutional neural network (CNN). The CNN performed a feature extraction process followed by a classification stage, which allowed the system to differentiate between healthy and ill patients using five categories. Our model was able to identify diabetic retinopathy in the patients with an agreement rate of 76.73% with respect to the medical expert's labels for the test data.
△ Less
Submitted 24 July, 2018;
originally announced July 2018.
-
A Lexicalist Approach to the Translation of Colloquial Text
Authors:
Fred Popowich,
Davide Turcato,
Olivier Laurens,
Paul McFetridge,
J. Devlan Nicholson,
Patrick McGivern,
Maricela Corzo Pena,
Lisa Pidruchney,
Scott MacDonald
Abstract:
Colloquial English (CE) as found in television programs or typical conversations is different than text found in technical manuals, newspapers and books. Phrases tend to be shorter and less sophisticated. In this paper, we look at some of the theoretical and implementational issues involved in translating CE. We present a fully automatic large-scale multilingual natural language processing syste…
▽ More
Colloquial English (CE) as found in television programs or typical conversations is different than text found in technical manuals, newspapers and books. Phrases tend to be shorter and less sophisticated. In this paper, we look at some of the theoretical and implementational issues involved in translating CE. We present a fully automatic large-scale multilingual natural language processing system for translation of CE input text, as found in the commercially transmitted closed-caption television signal, into simple target sentences. Our approach is based on the Whitelock's Shake and Bake machine translation paradigm, which relies heavily on lexical resources. The system currently translates from English to Spanish with the translation modules for Brazilian Portuguese under development.
△ Less
Submitted 18 June, 1997;
originally announced June 1997.
