Showing 1–12 of 12 results for author: Harrison, L

History-enhanced ICT For Sustainability education: Learning together with Business Computing students

Authors: Ian Brooks, Laura Harrison, Mark Reeves, Martin Simpson, Rose Wallis

Abstract: This research explores the use of History to enhance education in the field of ICT For Sustainability ICT4S in response to a challenge from the ICT4S 2023 conference. No previous studies were found in ICT4S but the literature on History and Education for Sustainable Development is reviewed. An ICT4S lecturer collaborated with History lecturers to add an historic parallel to each weeks teaching on… ▽ More This research explores the use of History to enhance education in the field of ICT For Sustainability ICT4S in response to a challenge from the ICT4S 2023 conference. No previous studies were found in ICT4S but the literature on History and Education for Sustainable Development is reviewed. An ICT4S lecturer collaborated with History lecturers to add an historic parallel to each weeks teaching on a Sustainable Business and Computing unit for final year undergraduate BSc Business Computing students. A list of the topics and rationale is provided. Student perceptions were surveyed before and after the teaching and semi-structured interviews carried out. A majority of students saw relevance to their degree and career. There was an increase in the proportion of students with interest in History. The paper explores the lessons learned from the interdisciplinary collaboration, including topic choice, format and perceived value. The project has enhanced the way we approach our subjects as computing and history educators. We believe this is the first empirical, survey-based study of the use of history to enhance ICT4S education. The team will extend the research to a larger unit covering a wider range of computing degrees. △ Less

Submitted 16 June, 2024; originally announced June 2024.

Adaptive Assessment of Visualization Literacy

Authors: Yuan Cui, Lily W. Ge, Yiren Ding, Fumeng Yang, Lane Harrison, Matthew Kay

Abstract: Visualization literacy is an essential skill for accurately interpreting data to inform critical decisions. Consequently, it is vital to understand the evolution of this ability and devise targeted interventions to enhance it, requiring concise and repeatable assessments of visualization literacy for individuals. However, current assessments, such as the Visualization Literacy Assessment Test (VLA… ▽ More Visualization literacy is an essential skill for accurately interpreting data to inform critical decisions. Consequently, it is vital to understand the evolution of this ability and devise targeted interventions to enhance it, requiring concise and repeatable assessments of visualization literacy for individuals. However, current assessments, such as the Visualization Literacy Assessment Test (VLAT), are time-consuming due to their fixed, lengthy format. To address this limitation, we develop two streamlined computerized adaptive tests (CATs) for visualization literacy, A-VLAT and A-CALVI, which measure the same set of skills as their original versions in half the number of questions. Specifically, we (1) employ item response theory (IRT) and non-psychometric constraints to construct adaptive versions of the assessments, (2) finalize the configurations of adaptation through simulation, (3) refine the composition of test items of A-CALVI via a qualitative study, and (4) demonstrate the test-retest reliability (ICC: 0.98 and 0.98) and convergent validity (correlation: 0.81 and 0.66) of both CATs via four online studies. We discuss practical recommendations for using our CATs and opportunities for further customization to leverage the full potential of adaptive assessments. All supplemental materials are available at https://osf.io/a6258/. △ Less

Submitted 27 August, 2023; originally announced August 2023.

Help or Hinder? Evaluating the Impact of Fairness Metrics and Algorithms in Visualizations for Consensus Ranking

Authors: Hilson Shrestha, Kathleen Cachel, Mallak Alkhathlan, Elke Rundensteiner, Lane Harrison

Abstract: For applications where multiple stakeholders provide recommendations, a fair consensus ranking must not only ensure that the preferences of rankers are well represented, but must also mitigate disadvantages among socio-demographic groups in the final result. However, there is little empirical guidance on the value or challenges of visualizing and integrating fairness metrics and algorithms into hu… ▽ More For applications where multiple stakeholders provide recommendations, a fair consensus ranking must not only ensure that the preferences of rankers are well represented, but must also mitigate disadvantages among socio-demographic groups in the final result. However, there is little empirical guidance on the value or challenges of visualizing and integrating fairness metrics and algorithms into human-in-the-loop systems to aid decision-makers. In this work, we design a study to analyze the effectiveness of integrating such fairness metrics-based visualization and algorithms. We explore this through a task-based crowdsourced experiment comparing an interactive visualization system for constructing consensus rankings, ConsensusFuse, with a similar system that includes visual encodings of fairness metrics and fair-rank generation algorithms, FairFuse. We analyze the measure of fairness, agreement of rankers' decisions, and user interactions in constructing the fair consensus ranking across these two systems. In our study with 200 participants, results suggest that providing these fairness-oriented support features nudges users to align their decision with the fairness metrics while minimizing the tedious process of manually having to amend the consensus ranking. We discuss the implications of these results for the design of next-generation fairness oriented-systems and along with emerging directions for future research. △ Less

Submitted 11 August, 2023; originally announced August 2023.

Comments: 14 pages

Journal ref: Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency (pp. 1685-1698)

Taken By Surprise? Evaluating how Bayesian Weighting Influences Peoples' Takeaways in Map Visualizations

Authors: Akim Ndlovu, Hilson Shrestha, Lane T. Harrison

Abstract: Choropleth maps have been studied and extended in many ways to counteract the many biases that can occur when using them. Two recent techniques, Surprise metrics and Value Suppressing Uncertainty Palettes (VSUPs), offer promising solutions but have yet to be tested empirically with users of visualizations. In this paper, we explore how well people can make use of these techniques in map exploratio… ▽ More Choropleth maps have been studied and extended in many ways to counteract the many biases that can occur when using them. Two recent techniques, Surprise metrics and Value Suppressing Uncertainty Palettes (VSUPs), offer promising solutions but have yet to be tested empirically with users of visualizations. In this paper, we explore how well people can make use of these techniques in map exploration tasks. We report a crowdsourced experiment where n = 300 participants are assigned to one of Choropleth, Surprise (only), and VSUP conditions (depicting rates and Surprise in a suppressed palette). Results show clear differences in map analysis outcomes, e.g. with Surprise maps leading people to significantly higher areas of population, or VSUPs performing similar or better than Choropleths for rate selection. Qualitative analysis suggests that many participants may only consider a subset of the metrics presented to them during exploration and decision-making. We discuss how these results generally support the use of Surprise and VSUP techniques in practice, and opportunities for further technique development. The material for the study (data, study results and code) is publicly available on https://osf.io/exb95/. △ Less

Submitted 27 July, 2023; originally announced July 2023.

Comments: 9 pages

The Risks of Ranking: Revisiting Graphical Perception to Model Individual Differences in Visualization Performance

Authors: Russell Davis, Xiaoying Pu, Yiren Ding, Brian D. Hall, Karen Bonilla, Mi Feng, Matthew Kay, Lane Harrison

Abstract: Graphical perception studies typically measure visualization encoding effectiveness using the error of an "average observer", leading to canonical rankings of encodings for numerical attributes: e.g., position > area > angle > volume. Yet different people may vary in their ability to read different visualization types, leading to variance in this ranking across individuals not captured by populati… ▽ More Graphical perception studies typically measure visualization encoding effectiveness using the error of an "average observer", leading to canonical rankings of encodings for numerical attributes: e.g., position > area > angle > volume. Yet different people may vary in their ability to read different visualization types, leading to variance in this ranking across individuals not captured by population-level metrics using "average observer" models. One way we can bridge this gap is by recasting classic visual perception tasks as tools for assessing individual performance, in addition to overall visualization performance. In this paper we replicate and extend Cleveland and McGill's graphical comparison experiment using Bayesian multilevel regression, using these models to explore individual differences in visualization skill from multiple perspectives. The results from experiments and modeling indicate that some people show patterns of accuracy that credibly deviate from the canonical rankings of visualization effectiveness. We discuss implications of these findings, such as a need for new ways to communicate visualization effectiveness to designers, how patterns in individuals' responses may show systematic biases and strategies in visualization judgment, and how recasting classic visual perception tasks as tools for assessing individual performance may offer new ways to quantify aspects of visualization literacy. Experiment data, source code, and analysis scripts are available at the following repository: https://osf.io/8ub7t/?view\_only=9be4798797404a4397be3c6fc2a68cc0. △ Less

Submitted 21 December, 2022; v1 submitted 20 December, 2022; originally announced December 2022.

Comments: 16 pages, 9 figures

ACM Class: H.5.0

Journal ref: IEEE Transactions on Visualization and Computer Graphics 2022

MANI-Rank: Multiple Attribute and Intersectional Group Fairness for Consensus Ranking

Authors: Kathleen Cachel, Elke Rundensteiner, Lane Harrison

Abstract: Combining the preferences of many rankers into one single consensus ranking is critical for consequential applications from hiring and admissions to lending. While group fairness has been extensively studied for classification, group fairness in rankings and in particular rank aggregation remains in its infancy. Recent work introduced the concept of fair rank aggregation for combining rankings but… ▽ More Combining the preferences of many rankers into one single consensus ranking is critical for consequential applications from hiring and admissions to lending. While group fairness has been extensively studied for classification, group fairness in rankings and in particular rank aggregation remains in its infancy. Recent work introduced the concept of fair rank aggregation for combining rankings but restricted to the case when candidates have a single binary protected attribute, i.e., they fall into two groups only. Yet it remains an open problem how to create a consensus ranking that represents the preferences of all rankers while ensuring fair treatment for candidates with multiple protected attributes such as gender, race, and nationality. In this work, we are the first to define and solve this open Multi-attribute Fair Consensus Ranking (MFCR) problem. As a foundation, we design novel group fairness criteria for rankings, called MANI-RANK, ensuring fair treatment of groups defined by individual protected attributes and their intersection. Leveraging the MANI-RANK criteria, we develop a series of algorithms that for the first time tackle the MFCR problem. Our experimental study with a rich variety of consensus scenarios demonstrates our MFCR methodology is the only approach to achieve both intersectional and protected attribute fairness while also representing the preferences expressed through many base rankings. Our real-world case study on merit scholarships illustrates the effectiveness of our MFCR methods to mitigate bias across multiple protected attributes and their intersections. This is an extended version of "MANI-Rank: Multiple Attribute and Intersectional Group Fairness for Consensus Ranking", to appear in ICDE 2022. △ Less

Submitted 20 July, 2022; originally announced July 2022.

Comments: This paper has been accepted by IEEE ICDE 2022. 15 pages, and 7 figures

Probablement, Wahrscheinlich, Likely ? A Cross-Language Study of How People Verbalize Probabilities in Icon Array Visualizations

Authors: Noëlle Rakotondravony, Yiren Ding, Lane Harrison

Abstract: Visualizations today are used across a wide range of languages and cultures. Yet the extent to which language impacts how we reason about data and visualizations remains unclear. In this paper, we explore the intersection of visualization and language through a cross-language study on estimative probability tasks with icon-array visualizations. Across Arabic, English, French, German, and Mandarin,… ▽ More Visualizations today are used across a wide range of languages and cultures. Yet the extent to which language impacts how we reason about data and visualizations remains unclear. In this paper, we explore the intersection of visualization and language through a cross-language study on estimative probability tasks with icon-array visualizations. Across Arabic, English, French, German, and Mandarin, n = 50 participants per language both chose probability expressions - e.g. likely, probable - to describe icon-array visualizations (Vis-to-Expression), and drew icon-array visualizations to match a given expression (Expression-to-Vis). Results suggest that there is no clear one-to-one map** of probability expressions and associated visual ranges between languages. Several translated expressions fell significantly above or below the range of the corresponding English expressions. Compared to other languages, French and German respondents appear to exhibit high levels of consistency between the visualizations they drew and the words they chose. Participants across languages used similar words when describing scenarios above 80% chance, with more variance in expressions targeting mid-range and lower values. We discuss how these results suggest potential differences in the expressiveness of language as it relates to visualization interpretation and design goals, as well as practical implications for translation efforts and future studies at the intersection of languages, culture, and visualization. Experiment data, source code, and analysis scripts are available at the following repository: https://osf.io/g5d4r/. △ Less

Submitted 2 October, 2023; v1 submitted 19 July, 2022; originally announced July 2022.

Comments: 11 pages, 10 figures, conference paper

Journal ref: IEEE Transactions on Visualization and Computer Graphics 2023

VisQuiz: Exploring Feedback Mechanisms to Improve Graphical Perception

Authors: Ryan Birchfield, Maddison Caten, Errica Cheng, Madyson Kelly, Truman Larson, Hoan Phan Pham, Yiren Ding, Noëlle Rakotondravony, Lane Harrison

Abstract: Graphical perception studies are a key element of visualization research, forming the basis of design recommendations and contributing to our understanding of how people make sense of visualizations. However, graphical perception studies typically include only brief training sessions, and the impact of longer and more in-depth feedback remains unclear. In this paper, we explore the design and eval… ▽ More Graphical perception studies are a key element of visualization research, forming the basis of design recommendations and contributing to our understanding of how people make sense of visualizations. However, graphical perception studies typically include only brief training sessions, and the impact of longer and more in-depth feedback remains unclear. In this paper, we explore the design and evaluation of feedback for graphical perception tasks, called VisQuiz. Using a quiz-like metaphor, we design feedback for a typical visualization comparison experiment, showing participants their answer alongside the correct answer in an animated sequence in each trial. We extend this quiz metaphor to include summary feedback after each stage of the experiment, providing additional moments for participants to reflect on their performance. To evaluate VisQuiz, we conduct a between-subjects experiment, including three stages of 40 trials each with a control condition that included only summary feedback. Results from n = 80 participants show that once participants started receiving trial feedback (Stage 2) they performed significantly better with bubble charts than those in the control condition. This effect carried over when feedback was removed (Stage 3). Results also suggest an overall trend of improved performance due to feedback. We discuss these findings in the context of other visualization literacy efforts, and possible future work at the intersection of visualization, feedback, and learning. Experiment data and analysis scripts are available at the following repository https://osf.io/jys5d/ △ Less

Submitted 2 October, 2023; v1 submitted 19 July, 2022; originally announced July 2022.

Comments: 5 pages, 5 figures, short paper

Journal ref: Proceedings of IEEE Visualization conference 2023

FairFuse: Interactive Visual Support for Fair Consensus Ranking

Authors: Hilson Shrestha, Kathleen Cachel, Mallak Alkhathlan, Elke Rundensteiner, Lane Harrison

Abstract: Fair consensus building combines the preferences of multiple rankers into a single consensus ranking, while ensuring any group defined by a protected attribute (such as race or gender) is not disadvantaged compared to other groups. Manually generating a fair consensus ranking is time-consuming and impractical -- even for a fairly small number of candidates. While algorithmic approaches for auditin… ▽ More Fair consensus building combines the preferences of multiple rankers into a single consensus ranking, while ensuring any group defined by a protected attribute (such as race or gender) is not disadvantaged compared to other groups. Manually generating a fair consensus ranking is time-consuming and impractical -- even for a fairly small number of candidates. While algorithmic approaches for auditing and generating fair consensus rankings have been developed, these have not been operationalized in interactive systems. To bridge this gap, we introduce FairFuse, a visualization system for generating, analyzing, and auditing fair consensus rankings. We construct a data model which includes base rankings entered by rankers, augmented with measures of group fairness, and algorithms for generating consensus rankings with varying degrees of fairness. We design novel visualizations that encode these measures in a parallel-coordinates style rank visualization, with interactions for generating and exploring fair consensus rankings. We describe use cases in which FairFuse supports a decision-maker in ranking scenarios in which fairness is important, and discuss emerging challenges for future efforts supporting fairness-oriented rank analysis. Code and demo videos available at https://osf.io/hd639/. △ Less

Submitted 1 August, 2022; v1 submitted 15 July, 2022; originally announced July 2022.

Comments: 5 pages, 4 figures; supplement: 4 pages

Formal Semantics of Heterogeneous CUDA-C: A Modular Approach with Applications

Authors: Chris Hathhorn, Michela Becchi, William L. Harrison, Adam Procter

Abstract: We extend an off-the-shelf, executable formal semantics of C (Ellison and Rosu's K Framework semantics) with the core features of CUDA-C. The hybrid CPU/GPU computation model of CUDA-C presents challenges not just for programmers, but also for practitioners of formal methods. Our formal semantics helps expose and clarify these issues. We demonstrate the usefulness of our semantics by generating a… ▽ More We extend an off-the-shelf, executable formal semantics of C (Ellison and Rosu's K Framework semantics) with the core features of CUDA-C. The hybrid CPU/GPU computation model of CUDA-C presents challenges not just for programmers, but also for practitioners of formal methods. Our formal semantics helps expose and clarify these issues. We demonstrate the usefulness of our semantics by generating a tool from it capable of detecting some race conditions and deadlocks in CUDA-C programs. We discuss limitations of our model and argue that its extensibility can easily enable a wider range of verification tasks. △ Less

Submitted 26 November, 2012; originally announced November 2012.

Comments: In Proceedings SSV 2012, arXiv:1211.5873

Journal ref: EPTCS 102, 2012, pp. 115-124

mod_oai: An Apache Module for Metadata Harvesting

Authors: Michael L. Nelson, Herbert Van de Sompel, Xiaoming Liu, Terry L. Harrison, Nathan McFarland

Abstract: We describe mod_oai, an Apache 2.0 module that implements the Open Archives Initiative Protocol for Metadata Harvesting (OAI-PMH). OAIPMH is the de facto standard for metadata exchange in digital libraries and allows repositories to expose their contents in a structured, application-neutral format with semantics optimized for accurate incremental harvesting. Current implementations of OAI-PMH ar… ▽ More We describe mod_oai, an Apache 2.0 module that implements the Open Archives Initiative Protocol for Metadata Harvesting (OAI-PMH). OAIPMH is the de facto standard for metadata exchange in digital libraries and allows repositories to expose their contents in a structured, application-neutral format with semantics optimized for accurate incremental harvesting. Current implementations of OAI-PMH are either separate applications that access an existing repository, or are built-in to repository software packages. mod_oai is different in that it optimizes harvesting web content by building OAI-PMH capability into the Apache server. We discuss the implications of adding harvesting capability to an Apache server and describe our initial experimental results accessing a departmental web site using both web crawling and OAIPMH harvesting techniques. △ Less

Submitted 24 March, 2005; originally announced March 2005.

Initial Experiences Re-Exporting Duplicate and Similarity Computation with an OAI-PMH aggregator

Authors: Terry L. Harrison, Aravind Elango, Johan Bollen, Michael Nelson

Abstract: The proliferation of the Open Archive Initiative Protocol for Metadata Harvesting (OAI-PMH) has resulted in the creation of a large number of service providers, all harvesting from either data providers or aggregators. If data were available regarding the similarity of metadata records, service providers could track redundant records across harvests from multiple sources as well as provide addit… ▽ More The proliferation of the Open Archive Initiative Protocol for Metadata Harvesting (OAI-PMH) has resulted in the creation of a large number of service providers, all harvesting from either data providers or aggregators. If data were available regarding the similarity of metadata records, service providers could track redundant records across harvests from multiple sources as well as provide additional end-user services. Due to the large number of metadata formats and the diverse map** strategies employed by data providers, similarity calculation requirements necessitate the use of information retrieval strategies. We describe an OAI-PMH aggregator implementation that uses the optional ``<about>'' container to re-export the results of similarity calculations. Metadata records (3751) were harvested from a NASA data provider and similarities for the records were computed. The results were useful for detecting duplicates, similarities and metadata errors. △ Less

Submitted 5 January, 2004; originally announced January 2004.

Comments: 10 pages

ACM Class: H.3.7; H.3.3