-
Exploring the Trade-off Between Model Performance and Explanation Plausibility of Text Classifiers Using Human Rationales
Authors:
Lucas E. Resck,
Marcos M. Raimundo,
Jorge Poco
Abstract:
Saliency post-hoc explainability methods are important tools for understanding increasingly complex NLP models. While these methods can reflect the model's reasoning, they may not align with human intuition, making the explanations not plausible. In this work, we present a methodology for incorporating rationales, which are text annotations explaining human decisions, into text classification mode…
▽ More
Saliency post-hoc explainability methods are important tools for understanding increasingly complex NLP models. While these methods can reflect the model's reasoning, they may not align with human intuition, making the explanations not plausible. In this work, we present a methodology for incorporating rationales, which are text annotations explaining human decisions, into text classification models. This incorporation enhances the plausibility of post-hoc explanations while preserving their faithfulness. Our approach is agnostic to model architectures and explainability methods. We introduce the rationales during model training by augmenting the standard cross-entropy loss with a novel loss function inspired by contrastive learning. By leveraging a multi-objective optimization algorithm, we explore the trade-off between the two loss functions and generate a Pareto-optimal frontier of models that balance performance and plausibility. Through extensive experiments involving diverse models, datasets, and explainability methods, we demonstrate that our approach significantly enhances the quality of model explanations without causing substantial (sometimes negligible) degradation in the original model's performance.
△ Less
Submitted 3 April, 2024;
originally announced April 2024.
-
Granularity at Scale: Estimating Neighborhood Socioeconomic Indicators from High-Resolution Orthographic Imagery and Hybrid Learning
Authors:
Ethan Brewer,
Giovani Valdrighi,
Parikshit Solunke,
Joao Rulff,
Yurii Piadyk,
Zhonghui Lv,
Jorge Poco,
Claudio Silva
Abstract:
Many areas of the world are without basic information on the socioeconomic well-being of the residing population due to limitations in existing data collection methods. Overhead images obtained remotely, such as from satellite or aircraft, can help serve as windows into the state of life on the ground and help "fill in the gaps" where community information is sparse, with estimates at smaller geog…
▽ More
Many areas of the world are without basic information on the socioeconomic well-being of the residing population due to limitations in existing data collection methods. Overhead images obtained remotely, such as from satellite or aircraft, can help serve as windows into the state of life on the ground and help "fill in the gaps" where community information is sparse, with estimates at smaller geographic scales requiring higher resolution sensors. Concurrent with improved sensor resolutions, recent advancements in machine learning and computer vision have made it possible to quickly extract features from and detect patterns in image data, in the process correlating these features with other information. In this work, we explore how well two approaches, a supervised convolutional neural network and semi-supervised clustering based on bag-of-visual-words, estimate population density, median household income, and educational attainment of individual neighborhoods from publicly available high-resolution imagery of cities throughout the United States. Results and analyses indicate that features extracted from the imagery can accurately estimate the density (R$^2$ up to 0.81) of neighborhoods, with the supervised approach able to explain about half the variation in a population's income and education. In addition to the presented approaches serving as a basis for further geographic generalization, the novel semi-supervised approach provides a foundation for future work seeking to estimate fine-scale information from aerial imagery without the need for label data.
△ Less
Submitted 18 February, 2024; v1 submitted 28 September, 2023;
originally announced September 2023.
-
WSAM: Visual Explanations from Style Augmentation as Adversarial Attacker and Their Influence in Image Classification
Authors:
Felipe Moreno-Vera,
Edgar Medina,
Jorge Poco
Abstract:
Currently, style augmentation is capturing attention due to convolutional neural networks (CNN) being strongly biased toward recognizing textures rather than shapes. Most existing styling methods either perform a low-fidelity style transfer or a weak style representation in the embedding vector. This paper outlines a style augmentation algorithm using stochastic-based sampling with noise addition…
▽ More
Currently, style augmentation is capturing attention due to convolutional neural networks (CNN) being strongly biased toward recognizing textures rather than shapes. Most existing styling methods either perform a low-fidelity style transfer or a weak style representation in the embedding vector. This paper outlines a style augmentation algorithm using stochastic-based sampling with noise addition to improving randomization on a general linear transformation for style transfer. With our augmentation strategy, all models not only present incredible robustness against image stylizing but also outperform all previous methods and surpass the state-of-the-art performance for the STL-10 dataset. In addition, we present an analysis of the model interpretations under different style variations. At the same time, we compare comprehensive experiments demonstrating the performance when applied to deep neural architectures in training settings.
△ Less
Submitted 28 August, 2023;
originally announced August 2023.
-
TDANetVis: Suggesting temporal resolutions for graph visualization using zigzag persistent homology
Authors:
Raphaël Tinarrage,
Jean R. Ponciano,
Claudio D. G. Linhares,
Agma J. M. Traina,
Jorge Poco
Abstract:
Temporal graphs are commonly used to represent complex systems and track the evolution of their constituents over time. Visualizing these graphs is crucial as it allows one to quickly identify anomalies, trends, patterns, and other properties leading to better decision-making. In this context, the to-be-adopted temporal resolution is crucial in constructing and analyzing the layout visually. The c…
▽ More
Temporal graphs are commonly used to represent complex systems and track the evolution of their constituents over time. Visualizing these graphs is crucial as it allows one to quickly identify anomalies, trends, patterns, and other properties leading to better decision-making. In this context, the to-be-adopted temporal resolution is crucial in constructing and analyzing the layout visually. The choice of a resolution is critical, e.g., when dealing with temporally sparse graphs. In such cases, changing the temporal resolution by grou** events (i.e., edges) from consecutive timestamps, a technique known as timeslicing, can aid in the analysis and reveal patterns that might not be discernible otherwise. However, choosing a suitable temporal resolution is not trivial. In this paper, we propose TDANetVis, a methodology that suggests temporal resolutions potentially relevant for analyzing a given graph, i.e., resolutions that lead to substantial topological changes in the graph structure. To achieve this goal, TDANetVis leverages zigzag persistent homology, a well-established technique from Topological Data Analysis (TDA). To enhance visual graph analysis, TDANetVis also incorporates the colored barcode, a novel timeline-based visualization built on the persistence barcodes commonly used in TDA. We demonstrate the usefulness and effectiveness of TDANetVis through a usage scenario and a user study involving 27 participants.
△ Less
Submitted 7 April, 2023;
originally announced April 2023.
-
MoReVis: A Visual Summary for Spatiotemporal Moving Regions
Authors:
Giovani Valdrighi,
Nivan Ferreira,
Jorge Poco
Abstract:
Spatial and temporal interactions are central and fundamental in many activities in our world. A common problem faced when visualizing this type of data is how to provide an overview that helps users navigate efficiently. Traditional approaches use coordinated views or 3D metaphors like the Space-time cube to tackle this problem. However, they suffer from overplotting and often lack spatial contex…
▽ More
Spatial and temporal interactions are central and fundamental in many activities in our world. A common problem faced when visualizing this type of data is how to provide an overview that helps users navigate efficiently. Traditional approaches use coordinated views or 3D metaphors like the Space-time cube to tackle this problem. However, they suffer from overplotting and often lack spatial context, hindering data exploration. More recent techniques, such as MotionRugs, propose compact temporal summaries based on 1D projection. While powerful, these techniques do not support the situation for which the spatial extent of the objects and their intersections is relevant, such as the analysis of surveillance videos or tracking weather storms. In this paper, we propose MoReVis, a visual overview of spatiotemporal data that considers the objects' spatial extent and strives to show spatial interactions among these objects by displaying spatial intersections. Like previous techniques, our method involves projecting the spatial coordinates to 1D to produce compact summaries. However, our solution's core consists of performing a layout optimization step that sets the size and positions of the visual marks on the summary to resemble the actual values on the original space. We also provide multiple interactive mechanisms to make interpreting the results more straightforward for the user. We perform an extensive experimental evaluation and usage scenarios. Moreover, we evaluated the usefulness of MoReVis in a study with 9 participants. The results point out the effectiveness and suitability of our method in representing different datasets compared to traditional techniques.
△ Less
Submitted 25 February, 2023;
originally announced February 2023.
-
LargeNetVis: Visual Exploration of Large Temporal Networks Based on Community Taxonomies
Authors:
Claudio D. G. Linhares,
Jean R. Ponciano,
Diogenes S. Pedro,
Luis E. C. Rocha,
Agma J. M. Traina,
Jorge Poco
Abstract:
Temporal (or time-evolving) networks are commonly used to model complex systems and the evolution of their components throughout time. Although these networks can be analyzed by different means, visual analytics stands out as an effective way for a pre-analysis before doing quantitative/statistical analyses to identify patterns, anomalies, and other behaviors in the data, thus leading to new insig…
▽ More
Temporal (or time-evolving) networks are commonly used to model complex systems and the evolution of their components throughout time. Although these networks can be analyzed by different means, visual analytics stands out as an effective way for a pre-analysis before doing quantitative/statistical analyses to identify patterns, anomalies, and other behaviors in the data, thus leading to new insights and better decision-making. However, the large number of nodes, edges, and/or timestamps in many real-world networks may lead to polluted layouts that make the analysis inefficient or even infeasible. In this paper, we propose LargeNetVis, a web-based visual analytics system designed to assist in analyzing small and large temporal networks. It successfully achieves this goal by leveraging three taxonomies focused on network communities to guide the visual exploration process. The system is composed of four interactive visual components: the first (Taxonomy Matrix) presents a summary of the network characteristics, the second (Global View) gives an overview of the network evolution, the third (a node-link diagram) enables community- and node-level structural analysis, and the fourth (a Temporal Activity Map -- TAM) shows the community- and node-level activity under a temporal perspective.
△ Less
Submitted 8 August, 2022;
originally announced August 2022.
-
ClinicalPath: a Visualization tool to Improve the Evaluation of Electronic Health Records in Clinical Decision-Making
Authors:
Claudio D. G. Linhares,
Daniel M. Lima,
Jean R. Ponciano,
Mauro M. Olivatto,
Marco A. Gutierrez,
Jorge Poco,
Caetano Traina Jr.,
Agma J. M. Traina
Abstract:
Physicians work at a very tight schedule and need decision-making support tools to help on improving and doing their work in a timely and dependable manner. Examining piles of sheets with test results and using systems with little visualization support to provide diagnostics is daunting, but that is still the usual way for the physicians' daily procedure, especially in develo** countries. Electr…
▽ More
Physicians work at a very tight schedule and need decision-making support tools to help on improving and doing their work in a timely and dependable manner. Examining piles of sheets with test results and using systems with little visualization support to provide diagnostics is daunting, but that is still the usual way for the physicians' daily procedure, especially in develo** countries. Electronic Health Records systems have been designed to keep the patients' history and reduce the time spent analyzing the patient's data. However, better tools to support decision-making are still needed. In this paper, we propose ClinicalPath, a visualization tool for users to track a patient's clinical path through a series of tests and data, which can aid in treatments and diagnoses. Our proposal is focused on patient's data analysis, presenting the test results and clinical history longitudinally. Both the visualization design and the system functionality were developed in close collaboration with experts in the medical domain to ensure a right fit of the technical solutions and the real needs of the professionals. We validated the proposed visualization based on case studies and user assessments through tasks based on the physician's daily activities. Our results show that our proposed system improves the physicians' experience in decision-making tasks, made with more confidence and better usage of the physicians' time, allowing them to take other needed care for the patients.
△ Less
Submitted 26 May, 2022;
originally announced May 2022.
-
LegalVis: Exploring and Inferring Precedent Citations in Legal Documents
Authors:
Lucas E. Resck,
Jean R. Ponciano,
Luis Gustavo Nonato,
Jorge Poco
Abstract:
To reduce the number of pending cases and conflicting rulings in the Brazilian Judiciary, the National Congress amended the Constitution, allowing the Brazilian Supreme Court (STF) to create binding precedents (BPs), i.e., a set of understandings that both Executive and lower Judiciary branches must follow. The STF's justices frequently cite the 58 existing BPs in their decisions, and it is of pri…
▽ More
To reduce the number of pending cases and conflicting rulings in the Brazilian Judiciary, the National Congress amended the Constitution, allowing the Brazilian Supreme Court (STF) to create binding precedents (BPs), i.e., a set of understandings that both Executive and lower Judiciary branches must follow. The STF's justices frequently cite the 58 existing BPs in their decisions, and it is of primary relevance that judicial experts could identify and analyze such citations. To assist in this problem, we propose LegalVis, a web-based visual analytics system designed to support the analysis of legal documents that cite or could potentially cite a BP. We model the problem of identifying potential citations (i.e., non-explicit) as a classification problem. However, a simple score is not enough to explain the results; that is why we use an interpretability machine learning method to explain the reason behind each identified citation. For a compelling visual exploration of documents and BPs, LegalVis comprises three interactive visual components: the first presents an overview of the data showing temporal patterns, the second allows filtering and grou** relevant documents by topic, and the last one shows a document's text aiming to interpret the model's output by pointing out which paragraphs are likely to mention the BP, even if not explicitly specified. We evaluated our identification model and obtained an accuracy of 96%; we also made a quantitative and qualitative analysis of the results. The usefulness and effectiveness of LegalVis were evaluated through two usage scenarios and feedback from six domain experts.
△ Less
Submitted 3 March, 2022;
originally announced March 2022.
-
ChartText: Linking Text with Charts in Documents
Authors:
Joao Pinheiro,
Jorge Poco
Abstract:
Recent works show that interactive documents connecting text with visualizations facilitate reading comprehension. However, creating this type of content requires specialized knowledge. We present ChartText, a method that links text with visualizations in this work. Our approach supports documents that include bar charts, line charts, and scatter plots. ChartText receives the visual encoding of th…
▽ More
Recent works show that interactive documents connecting text with visualizations facilitate reading comprehension. However, creating this type of content requires specialized knowledge. We present ChartText, a method that links text with visualizations in this work. Our approach supports documents that include bar charts, line charts, and scatter plots. ChartText receives the visual encoding of the visualization and its associated text as input. It then performs the linking in two stages: The matching stage creates individual links relating simple phrases between the text and the chart. Then, it combines the individual links according to the visual channels in the grou** stage, building more meaningful connections. We use two datasets to design and evaluate our method; the first comes from web documents (24 bar charts and texts) and the second from academic documents (25 bar charts, 25 line charts, and 25 scatter plots with their texts). Our experiments show that our method obtains F1 scores of 0.50 and 0.66 on both datasets. We can also use a semi-automatic approach correcting individual links; in this case, the scores rise to 0.68 and 0.84, respectively. To show the usefulness of our technique, we implement two proofs of concept. We create interactive documents using graphic overlays in the first one, facilitating the reading experience. We use voice instead of text to annotate charts in real-time in the second. For example, in a videoconference, our technique can automatically annotate a chart following the presenter's description.
△ Less
Submitted 13 January, 2022;
originally announced January 2022.
