SuperNOVA: Design Strategies and Opportunities for Interactive Visualization in Computational Notebooks

Abstract.

Computational notebooks, such as Jupyter Notebook, have become data scientists’ de facto programming environments. Many visualization researchers and practitioners have developed interactive visualization tools that support notebooks, yet little is known about the appropriate design of these tools. To address this critical research gap, we investigate the design strategies in this space by analyzing 163 notebook visualization tools. Our analysis encompasses 64 systems from academic papers and 105 systems sourced from a pool of 55k notebooks containing interactive visualizations that we obtain via scra** 8.6 million notebooks on GitHub. Through this study, we identify key design implications and trade-offs, such as leveraging multimodal data in notebooks as well as balancing the degree of visualization-notebook integration. Furthermore, we provide empirical evidence that tools compatible with more notebook platforms have a greater impact. Finally, we develop SuperNOVA, an open-source interactive browser to help researchers explore existing notebook visualization tools. SuperNOVA is publicly accessible at: https://poloclub.github.io/supernova/.

Fig. 1. SuperNOVA is a browser for exploring 163 notebook interactive visualization tools. Users can filter and search for tools with specific properties in the left panel. Clicking on a tool reveals details including paper metadata and GitHub repository.

1. Introduction

Computational notebooks, such as Jupyter Notebook (Kluyver and others, 2016) and Colab, are the most popular programming environments among data scientists (Kaggle, 2022). These notebooks seamlessly combine text, code, and visual outputs in a document that consists of an arbitrary number of cells—small text and code editors. Users can execute a code cell, and its output (e.g., text and visualizations) will be displayed below the cell. By providing a literate programming environment, notebooks enable users to perform exploratory data analysis, document their work, and share insights with collaborators (Rule et al., 2018).

To create easy-to-adopt tools, there is a trend in the VIS community to develop interactive visualization systems that can be used in notebooks (e.g., Ono et al., 2021; Xenopoulos et al., 2023; Wang et al., 2022e). Designing visualizations for notebook environments presents unique opportunities and considerations. On the one hand, notebook visualization tools allow direct modification of data through user interactions (Uber, 2016), and users can mix-and-match different visualization tools to create dashboards (Wang et al., 2022a). However, notebook users often write fragmentary code and execute it nonlinearly (Mcnutt et al., 2023; Weinman et al., 2021), which differs from traditional workflows for using interactive visualization systems (Chen and Golan, 2016).

Therefore, if researchers do not consider notebooks’ unique characteristics, their notebook visualization tools may not fully realize the potential of notebooks and, at worst, may impede the ability of notebook users to effectively use these tools. To shed light on the existing landscape of notebook visualization tools and help visualization researchers and practitioners harness the potential of notebook environments, we contribute:

• The first systematic review of 163 notebook visualization tools including 64 systems introduced in academic papers and 105 tools sourced from a pool of 55k notebooks containing interactive visualizations that we obtain via scra** 8.6 million notebooks on GitHub (Fig. 2). To inform the design of future tools, we discuss unique design implications (§ 4) and trade-offs (§ 5).

• Organizational framework to characterize notebook visualization tools in terms of their motivation for supporting notebooks (§ 4), targeted users (§ 4.1), and a four-dimensional design space based on user needs (§ 5). This framework facilitates a more comprehensive understanding of the landscape of notebook visualization tools. Based on this framework, we further analyze the effects of design factors on the impact of notebook visualization tools. We find tools supporting more notebook platforms have significantly more GitHub stars and paper citations (§ 6).

To broaden the public’s access to our collection, we develop SuperNOVA (Fig. 1), an interactive tool that helps researchers and designers explore existing notebook visualization tools and search for design inspiration and implementation references. Anyone can easily add new tools to this open-source explorer. SuperNOVA is publicly accessible at: https://poloclub.github.io/supernova/.

2. Related Work

Our work joins the research body of studying interactive tools for notebooks. To understand notebook users’ behaviors, researchers conduct interview studies (Kery et al., 2018) and analyze 1 million notebooks scraped from GitHub (Rule et al., 2018). Researchers present methods to help researchers develop notebook-compatible visualization tools (Piazentin Ono et al., 2021; Wang et al., 2022d). More recently, a design space analysis is conducted for AI-powered code assistants for notebooks (Mcnutt et al., 2023). In contrast, our work focuses on the design of visualization tools for notebooks by analyzing 163 tools identified from academic papers and 8.6 million notebooks. Additionally, inspired by the popular interactive survey browsers for text visualization (Kucher and Kerren, 2015), biological data visualization (Kerren et al., 2017), visualizations for trust in machine learning (Chatzimparmpas et al., 2024), and embedding visualization (Huang et al., 2023), we develop SuperNOVA, the first interactive explorer for notebook visualization tools.

3. Methodology

Systematic Review. To study how researchers and practitioners design notebook visualization tools, we collected and analyzed 64 academic papers and 105 tools in the wild. In this study, we define notebook visualization tools as systems that can display interactive visualizations in Python computational notebooks. (1) Literature collection: we searched Google Scholar for notebook visualization tools and performed forward and backward reference searches to snowball the results. (2) In-the-wild tool collection: we scraped 8.6 million notebooks from GitHub and filtered 55k notebooks containing interactive visualizations by matching notebook cell output types. We extracted 984 potential visualization packages by matching variable names and imported modules using abstract syntax trees, and we manually examined each package to keep 105 that were indeed notebook visualization tools (see ‡ B for details). (3) Coding: we conducted a multi-phase coding process to analyze the collected papers, documentation, and demo notebooks. First, three authors independently open coded (Braun and Clarke, 2006) the same 30 random tools regarding the motivations for using notebooks and design strategies using Google Sheets. After discussing the codebook and resolving disagreements, the three coders independently conducted open coding on the remaining tools, allocating an equal number of tools to each author. Following the analysis of the final codebook and themes, one author applied deductive coding (Merriam et al., 2002) to assign identified design patterns to each tool. We share all scra** code, codebook, and metadata of 163 tools in SuperNOVA’s repository.

Fig. 2. We present an organizational framework to characterize notebook visualization tools based on their design motivations and strategies through a review of 163 tools.

This figure has two sections, labeled ”Section 4 WHY” and ”Section 5 HOW,” each with a different background color, purple and teal respectively. The ”Section 4 WHY” section includes the title ”Motivation for notebook vis” and lists factors such as ”Fit into users’ workflow,” ”Access to rich data,” ”Portability & Sharing,” ”Easy implementation,” and ”Observed in 161 notebook vis tools.” The ”Section 5 HOW” section contains the title ”Design strategies” and lists items like ”Vis-notebook integration,” ”Data source type,” ”Sensemaking context,” and ”Modularity,” followed by ”64 papers + 103 tools in the wild.” At the bottom, there is an ”ORGANIZATIONAL FRAMEWORK” bar spanning both sections, depicting icons for charts, a building, code brackets, and an integration symbol, suggesting a structure for organizing the information.

Organizational Framework. Our large-scale systematic review resulted in an organizational framework characterizing notebook visualization tools in terms of motivations for supporting notebooks (§ 4), targeted users (§ 4.1), and design patterns based on user needs (§ 5). Using this framework, we develop SuperNOVA (Fig. 1), an interactive explorer that allows for easy filtering and searching for notebook visualization tools with desired properties. Based on our review, we distill 4 design implications and 4 design trade-offs to help future researchers design notebook visualization tools. Finally, we conduct a correlation analysis and two regression analyses to examine the effects of design patterns on the impacts of notebook interactive visualization tools (§ 6).

4. Why Notebook Visualization Tools

This section discusses the motivation for develo** interactive visualization tools for computational notebooks. We organize these motivations into four non-mutually exclusive groups.

4.1. Seamless Workflow Integration

Fig. 3. Many notebook visualization tools are developed for educators and students, such as GILP (Robbins et al., 2023) which offers interactive and easy-to-understand visualizations to help students learn about linear programming algorithms. Educators can directly integrate GILP into notebook-based assignments.

Screenshot of the GILP package.

Our study reveals that most of the surveyed visualization tools support notebooks as a means of aligning with the workflows of end-users. We observe that different user groups have distinct notebook usage patterns. Therefore, to ground our discussion on the notebook workflows of end-users, we categorize end-users into three user groups: data scientists, scientists, and educators and students.

Data Scientists. Notebooks are the most popular programming environment among data scientists (Kaggle, 2022). Consequently, many researchers have developed notebook visualization tools to promote adoption among data scientists. Data scientists use notebooks for conducting rapid experiments, collaborating with other stakeholders, and directly deploying notebooks within production pipelines (Chattopadhyay et al., 2020). Notebook visualization tools have covered almost every stage of data scientists’ workflow, from annotating data (Zhang et al., 2023b) and exploring data (Li et al., 2023b), to develo** ML models (Ono et al., 2021), documenting models (Bhat et al., 2023), evaluating models (Munechika et al., 2022), and communicating findings to stakeholders (Wang et al., 2023a).

Scientists. Notebooks are also popular among scientists, including biologists and physicists. Scientists use them as an interface for accessing remote clusters (Sbailò et al., 2022), and publishing notebooks with academic papers is considered good practice for reproducible research (Herwig et al., 2018). Thus, many notebook visualization tools are developed to facilitate scientific research workflows, such as designing experiments (Guo et al., 2021), simulating physical environments (Freeman et al., 2021), and analyzing molecules (Nguyen et al., 2018) and astronomical data (Araya et al., 2018).

Educators and Students. Notebooks are increasingly being used as interactive textbooks in computing education, as they enable students to easily interact with code and test their ideas (Smith et al., 2021). Educators also use notebooks for assigning and grading programming assignments (Hull et al., 2023). In this use case, notebooks serve as worksheets where students write and run their code in specific cells. We observe a growing trend of notebook visualization tools that are specifically developed for educators and students. For example, GILP (Robbins et al., 2023) visualizes simplex algorithms in notebooks, allowing educators to design interactive textbooks and assignments (Fig. 3). VizProg (Zhang et al., 2023a) helps instructors monitor students’ coding progress during in-class exercises through interactive visualizations.

Our findings highlight that computational notebooks are a popular medium among diverse user groups. In addition to data scientists, scientists, educators, and students also use notebooks in their workflows. This provides visualization researchers and designers with exciting opportunities to develop tools that can be easily adopted. However, we find different user groups have distinct notebook workflows. For example, scientists use notebooks for collaboration and reproducible research, while educators use them as textbooks and worksheets. Therefore, researchers should engage with targeted user groups in the early design process (Sedlmair et al., 2012) to investigate users’ notebook workflows and ground their designs.

Implication on domain-specific design: Designing notebook visualization tools requires researchers to engage with targeted user groups to develop tailored tools, as different user groups have distinct notebook usage patterns.

4.2. Easy Access to Read and Refine Artifacts

Fig. 4. Computational notebooks offer unique opportunities for visualization tools to read and refine users’ artifacts, such as code, data, and models. For example, (A) Lux (Lee et al., 2021) leverages a user’s data transformation code to recommend visualizations, while (B) GAM Changer (Wang et al., 2022b) enables users to interactively edit an ML model’s learned weights.

Screenshots of Lux and GAM Changer packages.

Notebook visualization tools not only benefit from easy adoption but also access to programming artifacts, including code, raw data, and models. These tools can be categorized into two groups based on their uses of artifacts.

Artifacts $\rightarrow$ Visualization Generation. To create visualizations in non-notebook environments, data scientists often need to manually specify chart types and input data. However, notebook tools have access to all artifacts needed to create visualizations. For example, B2 (Wu et al., 2020) uses dataframes and code queries in notebooks to automatically synthesize interactive visualizations. Similarly, Lux (Lee et al., 2021) and Solas (Epperson et al., 2022) provide automatic visualization recommendations based on a user’s dataframe and analysis history (Fig. 4A). Through accessing ML models that are being trained in notebooks, TensorBoard (Abadi et al., 2016) can visualize the model’s performance in real time.

Visualizations $\rightarrow$ Artifact Refinement. After gaining insights from visualizations, data scientists often manually refine their code, data, and models outside of notebooks. Notebooks can accelerate this process by directly updating artifacts. For example, Mage (Kery et al., 2020) automatically generates code to reflect the change caused by a user’s interaction with visualizations (e.g., deleting a column from a table). Similarly, GAM Changer (Wang et al., 2022b) enables users to modify ML model weights by direct manipulation on visualizations (Fig. 4B).

When designing notebook visualization tools, it is crucial to consider integrating the input and output in the visualization workflow (e.g., Chen and Golan, 2016; Cashman et al., 2019; Upson et al., 1989) into the notebook environment. Take Keim et al. (2008)’s visual analytics pipeline as an example, the input data can be notebook runtime artifacts, text, and usage logs (§ 5.2), and the output knowledge can be directly operationalized to synthesize code, transform data, and update ML models in the notebook (§ 5.1).

Implication on new opportunities enabled by easy artifact access: Computational notebooks provide unique opportunities for researchers to integrate the input of a visualization pipeline (e.g., notebook runtime artifacts and text) and operationalize its output (e.g., transforming data and updating ML models) within the users’ existing workflow.

4.3. Portability and Shareability

The notebook community has developed a vibrant ecosystem to convert notebooks into a wide range of mediums. This includes the ability for users to publish notebooks containing interactive visualizations as slides (Wang et al., 2023a), interactive books (Community, 2020), and dashboards (Bäuerle et al., 2022). Therefore, given the portability of notebooks, notebook visualization tools have the potential to reach a more diverse audience. For instance, InterpretML (Nori et al., 2019) leverages Jupyter Book (Community, 2020) to incorporate in-notebook visualizations into its documentation, providing readers with an engaging way to learn about ML model explanations (Fig. 5). However, different visualization modalities may present unique design challenges, such as potential accessibility concerns for interactive visualizations in presentation slides (Yip et al., 2021) and the need to consider social contexts for dashboard design (Sarikaya et al., 2019). Thus, it is crucial for researchers to carefully consider specific design constraints associated with different modalities if they decide to use notebooks as a bridge to other visualization mediums.

Implication on cross-modality design: The notebook ecosystem offers various options for distributing and sharing notebook visualization tools with diverse stakeholders through various modalities (e.g., interactive books, slides, dashboards). However, researchers need to consider unique design challenges associated with the targeted modalities.

Fig. 5. The vibrant notebook ecosystem enables developers to easily transfer their visualizations across various platforms. For example, (A) the Python library InterpretML (Wang et al., 2022c)’s notebook explainable ML visualizations are also used on (B) its documentation website via Jupyter Book (Community, 2020).

Screenshots of the package InterpretML and its documentation website.

4.4. Ease of Implementation

There exist multiple methods, varying in difficulty, for implementing notebook visualization tools. Some methods are simple and attract researchers to add notebook support for existing visualizations. For example, the ML library CatBoost (Prokhorenkova et al., 2019) uses Jupyter Notebook’s native ipywidgets to add checkboxes and sliders to help users customize simple loss function plots. Recent researchers have introduced NOVA workflow (Wang et al., 2022d), which enables easy conversion of web-based visualization apps into notebook widgets (e.g., Wang et al., 2022e; Munechika et al., 2022; Wang et al., 2022b). Moreover, we observe that some developers use notebooks as a platform for rapidly prototy** and deploying GUI applications. For instance, Pigeon (Germanidis, 2017) leverages ipywidgets to implement a simple visualization tool that allows annotators to label text and image data. Computational notebooks are web-based systems, and the low barrier to authoring notebook visualization tools reflects and contributes to the trend of web-based interactive visualizations (Battle et al., 2018, 2022). With the increasing ease of develo** notebook visualization tools, we anticipate a growing number of such tools catering to various notebook user groups (§ 4.1).

Implication on growing trend of notebook visualization tools: As the implementation is becoming increasingly accessible, the trend of using computational notebooks as a flexible platform for deploying and develo** web-based interactive visualization tools will continue.

Fig. 6. The integration level between notebook and visualization tools varies based on data communication channels. (A) Tools such as Argo Lite (Li et al., 2020) retrieve data from external servers instead of the notebook. (B) Visual Auditor (Munechika et al., 2022) visualizes different slices of the dataset that are sent from the notebook. (C) More integrated tools like pydeck (Uber, 2016) not only visualize data from the notebook but also send data back to the notebook, for example, information on a user’s selected map cells.

Screenshots of Argo Lite, Visual Auditor, and Pydeck.

5. How to Design Notebook Vis Tools

This section discusses the design patterns of existing notebook visualization tools. To organize these patterns, we construct a four-dimensional design space based on the tool users’ needs.

5.1. Notebook-Visulization Integration

The level of integration between notebook environments and visualization tools can vary widely. We characterize this integration continuum by the data communication channels between these two parties, where loosely integrated visualization tools have fewer communication channels than more tightly integrated tools.

No Direct Communication. A few notebook visualization tools do not directly receive data from the notebook environment, as their data source is not available within users’ notebooks. Nevertheless, notebooks allow these tools to retrieve data from external sources (§ C.2.1), thereby allowing users to enjoy these tools in their workflows. For example, TensorBoard reads log files from the file system, and StatCast (Lage et al., 2016) reads data from a separate database server. Argo Lite (Li et al., 2020) allows notebook users to view graph visualizations that are created from a separate website (Fig. 6A).

One-way Communication. Most notebook visualization tools have a one-way communication with the notebook environment: they receive input from the notebook but do not send data back to the notebook (§ C.2.2). (1) Users can explicitly specify the input. For example, users can write code to feed an ML model and data into Visual Auditor (Munechika et al., 2022), which generates interactive visualizations for auditing model biases (Fig. 6B). (2) Some tools also leverage implicit input. For instance, Solas provides situated visualization recommendations by analyzing a user’s historical analysis code. With a one-way communication, users can follow the familiar input-output notebook pattern (Kluyver and others, 2016) to customize visualization tools.

Bidirectional Communication. Tools with high notebook integration not only receive input from the notebook but also update its content (§ C.2.3). (1) These tools can add new code or text to the notebook. For example, B2 (Wu et al., 2020) adds a user’s interaction history to the notebook cells, and Mage (Kery et al., 2020) generates code that can lead to the same consequence as user interactions. (2) Some tools directly modify the runtime states in a notebook. For instance, the spatial visualization tool pydeck (Uber, 2016) stores the user’s selected data from the visualization in a runtime variable, which users can access in other code cells (Fig. 6C). Bidirectional communication in notebooks can be a powerful and unique feature that help interactive visualization users operationalize visualization insights (§ 4.2).

Notebooks enable researchers to integrate both input (  one-way communication) and output (  bidirectional communication) of a visualization pipeline into the users’ existing workflow (Easy Access to Read and Refine Artifacts). However, designing  bidirectional communication requires caution. Chattopadhyay et al. (2020) find that notebook users often struggle to keep track of the states in different cells. Therefore, automatically modifying notebook states through a visualization tool could cause further confusion. Similarly, in Wu et al. (2020)’s study, some participants found it ”annoying” when notebook content was populated from a visualization tool. Thus, it is crucial to offer users clear feedback and allow users to configure state-updating behaviors.

Trade-off on data communication: Designing data communication channels (e.g., one-way vs. bidirectional communication) in notebook visualization tools requires careful balance: while bidirectional communication enriches user workflow, it also risks confusion, highlighting the need for clear user feedback and configurable content update policies.

5.2. Data Source and Type

Notebook environments offer rich and multimodal data sources that a visualization tool can use to meet user needs.

Runtime Artifacts. The most common visualization data source is a notebook’s runtime artifacts. Visualization tools have access to any data specified by notebook users; existing notebook visualization tools support many data modalities, such as tables (Brugman, 2019), spatial data (Uber, 2016), and 3D images (Abraham et al., 2014). Some tools also leverage ML models in a notebook runtime, hel** users interpret transformers (Vig, 2019), curate decision trees (Wang et al., 2022e), calibrate generalized additive models (Xenopoulos et al., 2023), and explore counterfactual explanations (Wexler et al., 2019).

Code and Text. Notebooks combine code and text documentation, which visualization tools can exploit to enhance visualizations (§ C.3). For example, Anteater (Faust et al., 2022) leverages trace-based visualization to help notebook users debug their analysis code. Jigsaw (Kluyver and others, 2016) uses variable names in a notebook to validate and correct code generated by AI models. More recently, researchers also use code and text in notebooks to create interactive slides to communicate data insights (Wang et al., 2023a; Li et al., 2023a). Moreover, to help users write high-quality ML model documentation, DocML (Bhat et al., 2023) links a model card visualization to both code and text cells in a notebook.

External Data. Moreover, notebook visualization tools can access data beyond the notebook environment, such as the file system, networks, and hardware information. For example, TensorBoard and StatCast visualize data from a local directory and a database server, respectively. NVDashboard (NVIDIA, 2021) provides notebook users with an interactive dashboard to monitor real-time GPU usage.

Although notebooks provide unique and valuable data for designing interactive visualization tools, accessing various data types requires different implementation strategies. While it is relatively easy to read  runtime artifacts (§ 4.4), it requires more engineering effort to read  code and text or implement  bidirectional communication (see ‡ C for detailed discussion on implementation strategies). Certain strategies are only compatible with specific notebook platforms; for example, tools implemented with Jupyter extensions cannot be used in Google Colab. Thus, there is a trade-off between accessing powerful notebook features and ensuring compatibility with diverse notebook platforms.

Trade-off on compatibility: Notebooks provide access to unique data types, including runtime artifacts, code, text and external data. However, there is a trade-off between leveraging powerful, yet platform-specific features like reading code and text and bidirectional communication, and ensuring broader compatibility across various notebook platforms.

5.3. Display Style & Sensemaking Context

Fig. 7. Notebook visualization tools’ display styles vary based on the user’s sensemaking context. For example, (A) TimberTrek (Wang et al., 2022e) uses an on-demand display to visualize a large collection of decision trees next to the cells where the trees are created. (B) AutoProfiler (Epperson et al., 2023) leverages an always-on display to automatically and continuously highlight data distributions and summary statistics of the user’s datasets.

Screenshots of TimberTrek and AutoProfiler.

Notebook visualization tools’ display styles can vary based on the user’s sensemaking context (Liu and Stasko, 2010). On-demand displays can be used for situational contexts, while always-on displays are suitable for continuous contexts.

On-demand display. Most visualization tools show visualizations below a code cell (e.g., Wexler et al., 2019; Wang et al., 2022b; Xenopoulos et al., 2023). These visualizations are part of the cell flow—they move vertically with the cells when a user scrolls through the notebook. With this layout, users can easily create multiple instances of the same visualization tool with different input data. For example, users can create multiple instances of TimberTrek (Wang et al., 2022e) in different cells with different collections of decision trees and compare across these collections (Fig. 7A).

Always-on display. Notebook tools can also display visualizations outside of notebook cells (§ C.4), leading to an always-on display detached from the cell flow. For instance, AutoProfiler (Wu et al., 2020) continuously updates data distribution visualizations in a resizable dashboard pane to the right of the notebook UI, allowing users to view persistent data profiling information while exploring their datasets (Fig. 7B). Similarly, NVDashboard (NVIDIA, 2021) displays multiple charts outside of the notebook UI so that users can monitor their GPU usage in real time while interacting with the notebook.

The design choice of visualization display style in the notebook depends on the users’ needs and the sensemaking context. Based on Liu and Stasko (2010)’s sensemaking model, visualizations provide external anchoring, cognitive offloading and information foraging. They suggest that visualization designers should minimize the “semantic distance” (Hutchins et al., 1985) between the tasks users want to perform and the physical form of visualizations. In computational notebooks, an  on-demand display can assist users with situational sensemaking and temporary anchoring for comparisons. On the other hand, an  always-on display can be beneficial for ongoing monitoring and tasks that require continuous cognitive offloading.

Trade-off on display style: Researchers need to consider the trade-off between on-demand and always-on displays of interactive visualizations in notebooks based on the users’ needs. On-demand displays aid situational sensemaking and comparisons, while always-on displays support continuous monitoring and cognitive offloading.

5.4. Modularity

Modularity in notebook visualization tools is a critical consideration when catering to different analysis needs, such as exploratory and exploitative (Batch and Elmqvist, 2018), and user’s programming proficiency. This ensures a balance between the code and the graphical user interface.

Monolithic System. Most notebook visualization tools are monolithic, presenting the entire system all at once. For example, when a user calls ydata-profilling (Brugman, 2019) in a notebook cell, the tool displays a panel beneath the cell that contains all exploratory data analysis visualizations (Fig. 8A). These visualizations are organized into multiple tabs based on their tasks, such as variable interactions, correlations, and missing values.

Fig. 8. The modularity of notebook visualization tools varies. For example, (A) ydata-profiling (Brugman, 2019) shows all components in a notebook cell, where users navigate data profiling visualizations via tabs. (B) In contrast, Aequitas (Saleiro et al., 2019) modularizes different visualization components into distinct Python functions, enabling users to write code to show ML model fairness visualizations tailored to specific needs.

Screenshots of ydata-profiling and Aequitas.

Modular Components. Modular visualization tools accommodate the fragmentary nature of notebook code and allow users to easily customize and compose visualizations. For example, Aequitas (Saleiro et al., 2019), an ML fairness auditing toolkit, provides different interactive visualizations for different fairness metrics. These visualizations are modularized into separate functions, enabling users to write code to generate and compose visualizations that meet specific needs (Fig. 8B). For instance, with Aequitas, a user can create and inspect a fairness overview in a notebook cell and delve into specific fairness metrics in other separate cells.

Monolithic and modular architectures have been extensively discussed in the software engineering literature for decades (Aoyama, 1998). Within the visual analytics research community, there is a recent trend towards shifting from designing “over-complicated” monolithic systems to simpler and reusable modular components (Wu et al., 2022; Bertini, 2022). The use of  modular components aligns well with computational notebooks, as notebook users can easily display and customize different components in separate notebook cells. Additionally, users can take advantage of dashboard authoring tools (Wang et al., 2022a; Bäuerle et al., 2022) to compose different visualization components into a dashboard directly in their notebooks (§ 4.3). However,  modular components require the users to know their visualization goals (i.e., exploitative analysis) and know how to write code to display the appropriate components. In contrast, a  monolithic system is more friendly to beginner users and suitable for exploratory analysis, where it can guide users to uncover data patterns and insights.

Trade-off on modularity: Modular visualization tools are composable and reusable, particularly in notebooks where users can easily display and customize them. While modular components offer flexibility for users with clear analysis goals and coding skills, monolithic systems remain more beginner-friendly and ideal for exploratory analysis.

6. Analysis

Leveraging our organizational framework as a lens, we conduct a quantitative analysis to study the relationship between the design of notebook visualization tools and the impacts of these tools (e.g., GitHub star count and publication citation count). Our analysis offers additional insights into future design decisions.

Data Collection. We characterize all 163 notebook visualization tools using our framework (Table 1). Then, we collect the GitHub star count, first commit date, publication year, and citation count of

Fig. 9. We conduct two regression analyses to investigate the effects of various design factors on the impact of notebook visualization tools, as measured by (A) GitHub star count and (B) paper citation count. We encode categorical variables using dummy variables, with the baseline category labeled in the figure. The results reveal that, in addition to time, notebook interactive visualization tools that support more notebook platforms have significantly more GitHub stars and paper citations.

This is a comparative figure divided into two panels: A and B. Both panels analyze the impacts of various design features on the success of notebook platforms, with panel A focusing on GitHub Star Count and panel B on Paper Citation Count.

Panel A, titled ”Design Impacts on GitHub Star Count,” shows a horizontal axis measuring the estimated coefficients with 95% confidence intervals. Key variables such as ”Communication,” ”Data Source,” ”Display Style,” and ”Modularity” are compared against baselines. Notable is the ”Num Supported Platforms,” with a statistically significant p-value (p ¡ 0.0001). The graph indicates that bidirectional communication, integrating code with runtime as a data source, on-demand display style, and modular design have positive impacts on GitHub star counts.

Panel B, titled ”Design Impacts on Paper Citation Count,” mirrors the design of panel A but focuses on the number of citations a paper receives. Again, the ”Num Supported Platforms” is statistically significant (p ¡ 0.0001). Bidirectional communication, integration of code and runtime, on-demand display, and modularity are shown with their respective impacts on citation counts.

Both panels include a note indicating that the log of days since the first commit or years since publication is significantly related to the outcome. The bottom of each panel reports the number of observations, the intercept, the significance of the intercept, and the R-squared value, indicating the proportion of variance explained by the models. The star and quotation mark icons next to the titles of the panels symbolize GitHub stars and citations, respectively.

Fig. 10. Design dimension correlations via pair-wise $X^{2}$ tests.

This figure is a matrix-style figure titled ”Correlation Between Design Dimensions.” It presents a table with p-values that indicate the strength and significance of correlations between various design dimensions for notebook platforms, based on 161 observations using the Chi-squared independence test.

The design dimensions listed both horizontally and vertically include ”Num Platforms,” ”Communication,” ”Data Source,” ”Display Style,” ”Modularity,” and ”Implementation.” Each cell in the matrix provides the p-value of the correlation between the dimensions intersecting at that cell, with values ranging from 0.001 to 0.640. Lower p-values suggest stronger evidence against the null hypothesis of no association.

The color gradient scale on the right side of the figure ranges from light to dark grey, corresponding to p-values from 0.0 to -0.5, indicating that darker cells have a lower p-value and thus a stronger correlation. For instance, the correlation between ”Num Platforms” and ”Modularity” has a p-value of 0.006, indicating a significant correlation, while ”Data Source” and ”Display Style” have a p-value of 0.640, suggesting a weaker correlation.

all available tools via the GitHub API (GitHub, 2023) and Semantic Scholar API (Kinney et al., 2023). Among all 163 tools, 135 have GitHub repositories and 76 have Semantic Scholar entries.

Correlation Analysis. We analyze the correlations across different design dimensions by conducting pair-wise $X^{2}$ independence tests (Fig. 10). Unsurprisingly, our results highlight that implementation strategies correlate with many other design dimensions. For example, tools that support  always-on display are more likely to be implemented using extensions. Interestingly, data source is correlated with both communication and display styles. In particular, tools that access  code and text from the notebook are more likely to support  bidirectional communication and  always-on display. We hypothesize that this is because designers often use text and code from a notebook for generative tasks (e.g., automatic visualization generation), and they prefer always-on displays to provide notebook users with continuous feedback (Display Style & Sensemaking Context). For example, visualization recommendation tools B2 (Wu et al., 2020) and PI2 (Chen and Wu, 2022) leverage existing code and text in a notebook to generate new visualization code in the notebook and display synthesized visualizations on an always-on panel. Finally, we observe that tools with  bidirectional communication support much less notebook platforms than tools with  one-way communication. This empirical finding reflects the trade-off between notebook integration and platform dependency (Data Source and Type).

Regression Analysis. We conduct two regression analyses to examine the effects of design factors on the impact of notebook visualization tools, as measured by GitHub star and paper citation counts (Fig. 9). Since implementation strategies correlate with many other design dimensions, we do not include it in both regression models. We include time as an independent variable and use dummy variables to encode categorical variables. The results highlight that tools supporting more notebook platforms have significantly more GitHub stars and paper citations. Other design dimensions do not significantly affect the popularity and recognition of notebook visualization tools. This result implies that future researchers and developers should prioritize notebook platform compatibility to maximize the impact of their tools.

7. Discussion and Future Work

By analyzing 163 interactive notebook visualization tools identified from 8.6 million public notebooks and 64 academic papers (§ 3), we present an organization framework to characterize these tools (§ 4, § 5). We provide practice design implications and trade-offs as well as insights from statistical analyses (§ 6). Based on our findings, we discuss future research opportunities and limitations of our study.

Democratizing Notebook Visualization Tool Creation. We discover a spectrum of methods, varying in difficulty, for authoring notebook visualization tools (§ 4.4). In particular, accessing  code and text and supporting  bidirectional communication require significant engineering effort (§ 5.2). Furthermore, some implementation strategies are only compatible with specific notebook platforms (Data Source and Type). Therefore, we see research opportunities to lower the barrier to authoring notebook interactive visualization tools that harness the full potential of notebook platforms. First, practitioners often use libraries such as D3 (Bostock et al., 2011) and VegaLite (Satyanarayan et al., 2017) to develop web-based interactive visualizations. It would be valuable if these libraries integrated native support for notebook platforms or new libraries specifically targeted authoring notebook visualizations. On the other hand, researchers can also enhance notebook platforms to better support interactive visualizations. For example, similar to browser vendors sharing the same web standard, researchers can develop a universal notebook protocol that enables developers to access and communicate data using a standardized method across notebook platforms.

Fig. 11. StickyLand (Wang et al., 2022a) enables a non-linear notebook layout, allowing notebook users to easily switch between on-demand and always-on displays of notebook visualization tools. For example, while prototy** large language model-powered apps by writing prompts, (A) a user can create a sticky cell with HuggingFace Tokenizer (Moi and Patry, 2023) to continuously visualize the prompt’s tokenization patterns. (B) The user can also use an always-on display with Farsight (Wang et al., 2024) to alert to potential risks associated with the prompt. (C) Moreover, the user can use Farsight’s interactive tree visualization in an on-demand display for brainstorming use cases, stakeholders, and potential harms of their apps.

Screenshots of StickyLand, Farsight, and HuggingFace Tokenizer.

Enriching Fluid Notebook-Vis Integration. The design trade-offs regarding visualization display styles (Display Style & Sensemaking Context) and modularity (Modularity) partially arise from the rigid layout of the popular cell-based notebooks (Lau et al., 2020). For example, most notebook platforms present cells in a linear manner, thereby requiring designers to decide whether to display their visualization tools within the flow (  on-demand display) of the cell or detach them from the flow (  always-on display). To address this trade-off, researchers can explore alternative notebook layouts. For example, researchers have introduced sticky cells (Wang et al., 2022a) to break the linear presentation of notebook cells. These sticky cells provide visualization designers with the flexibility to seamlessly switch between  on-demand and  always-on displays (Fig. 11). Similarly, regarding the modularity of visualization tools, future researchers could develop intelligent notebook interfaces that automatically adapt a visualization tool between  modular and  monolithic modes based on the users’ current tasks and requirements.

Promoting Responsible AI through Notebook Workflows. We observe an interesting trend that researchers exploit notebooks as a means to promote responsible AI practices (e.g., Aequitas (Saleiro et al., 2019), Fairlearn (Dudík et al., 2020), Farsight (Wang et al., 2024), and MLDoc (Bhat et al., 2023)). We identify two motivations for this emerging trend. First, AI practitioners often lack incentives to adopt responsible AI practices (Rakova et al., 2021; Schiff et al., 2020), such as fairness assessment and model documentation. By integrating responsible AI practices directly into practitioners’ existing notebook workflows (§ 4.1), researchers aim to minimize adoption friction and “nudge” (Bhat et al., 2023) practitioners to follow these practices. For example, Farsight alerts users to potential harms of their large language model-powered apps while they are develo** prompts in a notebook (Fig. 11). Similarly, MLDoc automatically creates and shows an AI “model card” (Mitchell et al., 2019) using content from a notebook.

Secondly, responsible AI requires collaboration across disciplines and teams within an organization (Rakova et al., 2021; Wang et al., 2023b). Because AI practitioners have already been using notebooks to collaborate with diverse stakeholders (e.g., designers and managers) (Zhang et al., 2020), researchers leverage notebooks as a boundary object to facilitate responsible AI practices across teams. For example, in Deng et al. (2022)’s study on ML fairness toolkits, a participant highlighted “a simple notebook format and compelling visualizations are needed for [organizational] leadership to adopt the toolkits.” Thus, as the mitigation of AI harms has become increasingly crucial, we see exciting research opportunities for researchers to design, develop, and evaluate notebook visualization tools to promote responsible AI.

Limitations. In this study, to keep our review manageable and focused, we focus on computational notebooks designed for Python, the most commonly used programming language among data scientists (Kaggle, 2022). Future work can explore notebooks designed for other languages, such as R Markdown (Studio, 2016) for R and Observable (Observable, 2021) for JavaScript. As notebook visualization tools are still nascent, there are limited user studies evaluating the effectiveness of these tools. In addition, although there are many different notebook user groups (§ 4.1), the existing HCI notebook research focuses on data scientists (Lau et al., 2020). To broaden the understanding of notebook visualization tools, future research endeavors can involve engaging with diverse user groups, including scientists, educators, students, and users with accessibility needs.

8. Conclusion

We collect a total of 163 notebook visualization tools, including 64 from academic papers and 103 sourced from a pool of 55k notebooks containing interactive visualizations that we obtain by scra** 8.6 million notebooks on GitHub. Based on our review, we introduce a framework for characterizing these tools in terms of their motivation for supporting notebooks, targeted users, and design patterns. We further discuss key design implications and trade-offs as well as research opportunities for notebook visualization. Finally, we present SuperNOVA to help researchers and developers easily explore existing notebook visualization tools. We hope that our work contributes to a more comprehensive understanding of notebook visualization tools and helps researchers design and develop visualization tools that are easy to use and adopt.

Appendix A Characterizing Notebook Visualization Tools

Below we characterize 163 collected notebook visualization tools using our organizational framework (targeted users and a four-dimensional design space described in § 3), as well as their supported notebook platforms and implementation methods. See SuperNOVA for an interactive version with more details about each entry.

Table 1. This table describes the characterization of 163 notebook visualization tools using our organizational framework. The columns include notebook visualization tools’ names, intended users (data scientists  , scientist  , educators and students  ); visualization-notebook communication styles (no direct communication  , one-way  , bidirectional  ); data source (runtime  , text and code  , external  ); display style (on-demand  , always-on  ); modularity (monolithic  , modular  ); their supported notebook platforms (Jupyter Notebook only  , JupyterLab only  , Jupyter Notebook + JupyterLab  , all popular platforms  ); and their implementation methods (NOVA  , HTML display  , ipywidget  , Lab Extension  , custom servers  ).

Notebook Vis Tool	User	Com.	Data	Disp.	Mod.	Plat.	Imp.			Notebook Vis Tool	User	Com.	Data	Disp.	Mod.	Plat.	Imp.
Aequitas (Saleiro et al., 2019)										Altair (VanderPlas et al., 2018)
Anteater (Faust et al., 2022)										Apache Beam (Apache, 2019)
Argo Lite (Li et al., 2020)										Atria (Williams et al., 2019)
AutoProfiler (Epperson et al., 2023)										AutoViz (AutoViML, 2020)
Ax (Facebook, 2019)										B2 (Wu et al., 2020)
Bacata (Verano Merino et al., 2020)										Bamboolib (Krabel, 2019)
BERTopic (Grootendorst, 2022)										BertViz (Vig, 2019)
Bokeh (Bokeh Development Team, 2014)										bqplot (Bqplot, 2016)
Brax (Google, 2021)										Calibrate (Xenopoulos et al., 2023)
Calling Context Tree (Scully-Allison et al., 2022)										CatBoost (Prokhorenkova et al., 2019)
CausalVis (Guo et al., 2023)										ChartPy (Cuemacro, 2016)
Clustergrammar (Fernandez et al., 2017)										Cyberhubs (Herwig et al., 2018)
Cytoscapejs (Franz et al., 2022)										CZML3 (Poliastro, 2019)
d3fdgraph (Mining, 2019)										Data+Shift (Palmeiro et al., 2022)
Data-Purifier (Gupta, 2021)										datapane (Datapane, 2023)
DataPrep (Peng et al., 2021)										DEA Tools (Krause et al., 2021)
DocML (Bhat et al., 2023)										EDAssistant (Li et al., 2023b)
ELI5 (Korobov, 2016)										Emblaze (Sivaraman et al., 2022)
Errudite (Wu et al., 2019)										Escher (King, 2016)
Evidently (AI, 2022)										Farsight (Wang et al., 2024)
FiftyOne (Voxel51, 2020)										Firefly (Gurvich and Geller, 2023)
Flexx (Klein, 2016)										Folium (Fernandes, 2019)
GAM Changer (Wang et al., 2022b)										GateNLP (Petrak, 2020)
GeoPandas (Jordahl et al., 2022)										GeoViews (Rudiger, 2016)
GILP (Robbins et al., 2023)										Graphistry (Graphistry, 2016)
gravis (Haas, 2021)										Gwaihir (Simonne et al., 2022)
HCIplot (Gonzalez, 2019)										HiPlot (Facebook, 2020)
igv (Robinson, 2022)										imolecule (Fuller, 2013a)
InkSight (Lin et al., 2023)										Intake (Durant, 2018)
Interpret-Community (Microsoft, 2019)										InterpretML (Nori et al., 2019)
ipyannotate (Kukushkin, 2018)										ipydatagrid (Bloomberg, 2019)
ipysheet (QuantStack, 2017)										IPython Vega (Satyanarayan et al., 2017)
ipytree (QuantStack, 2022)										ipyvizzu (Vizzu, 2022)
itables (Wouts, 2019)										itkwidgets (McCormick et al., 2022)
ivpy (Crockett, 2021)										jgraph (Fuller, 2013b)
Jigsaw (Jain et al., 2022)										Jupyter Dash (Parmer, 2020)
JupyterPiDAQ (Lab, 2020)										jyquickhelper (dupré, 2016)
Kaleidoscope (QuSTaR, 2019)										KeplerGL (Keplergl, 2019)
Keras (Chollet, 2015)										LightGBM (Ke et al., 2017)
Lightkurve (Lightkurve Collaboration et al., 2018)										LIT Tool (Tenney et al., 2020)
Lux (Lee et al., 2021)										mage (Kery et al., 2020)
Matplotlib (Hunter, 2007)										Mayavi (Enthought, 2015)
MEGAnno (Zhang et al., 2023b)										MLProvLab (Kerzel et al., 2023)
ModelSketchBook (Lam et al., 2023)										mols2grid (Bouysset, 2021)
Moving Pandas (Graser and Dragaschnig, 2020)										NaaVRE (Zhao et al., 2022)
nbinteract (Lau and Hug, 2018)										Nbtutor (Logan, 2023)
Nengo (Nengo, 2019)										neo4jupyter (Maeztu, 2016)
Networkit (Angriman et al., 2022)										NGLview (Nguyen et al., 2018)
Nilearn (Abraham et al., 2014)										NL4DV (Narechania et al., 2021)
NoLiES (Sohns et al., 2022)										NOMAD (Sbailò et al., 2022)
Notable (Li et al., 2023a)										NVDashboard (NVIDIA, 2021)
Open3d (Org, 2019)										Pandas-Bokeh (Hlobil, 2018)
PandasGUI (Rose, 2020)										Panel (Rudiger, 2021)
Pathpy (Hackl, 2019)										Perspective (Stein, 2022)
PI2 (Chen and Wu, 2022)										Pigeon (Germanidis, 2017)
PipelineProfiler (Ono et al., 2021)										PixieDust (PixieDust, 2016)
Plotly (Sievert et al., 2017)										Plotly-Resampler (Van Der Donckt et al., 2022)
Plotting Agent (Wang et al., 2021)										Py2cytoscape (Boucas, 2015)
py3Dmol (Autodesk, 2016)										PyCaret (Baum, 2020)
pydeck (Uber, 2016)										pydgrid (Mauricio, 2017)
PyGeoHydro (Chegini et al., 2021)										pyLDAvis (Sievert and Shirley, 2014)
PyMove (Sampaio, 2018)										PyPathway (PyPathway, 2022)
PyPotree (Borelli, 2019)										Pytket (Sivarajah et al., 2020)
Pyvis (Perrone et al., 2020)										PyZX (Kissinger and van de Wetering, 2020)
Qgrid (Shawver, 2017)										Quantstats (Aroussi, 2019)
Quick-EDA (Venkatachalapathi, 2020)										RAI Widgets (Microsoft, 2020)
SHAP (Lundberg and Lee, 2017)										Slide4N (Wang et al., 2023a)
Smoothy (Araya et al., 2018)										Solas (Epperson et al., 2022)
Spatialtis (AaltoGIS, 2020)										ST-VISIONS (Tritsarolis et al., 2021)
StatCast Dashboard (Lage et al., 2016)										SweetViz (Bertrand, 2020)
Symphony (Bäuerle et al., 2022)										Taggle (Furmanova et al., 2020)
TensorBoard (Abadi et al., 2016)										TF Model Analysis (Google, 2018)
TimberTrek (Wang et al., 2022e)										TissUUmaps (Pielawski et al., 2022)
Toyplot (Laboratories, 2022)										Trimesh (Dawson-Haggerty et al., 2019)
ULCA (Fujiwara et al., 2022)										VAEX (Breddels, 2016)
VAINE (Guo et al., 2021)										visJS2jupyter (Rosenthal et al., 2018)
Visual Auditor (Munechika et al., 2022)										Vizic (Yu et al., 2017)
VizProg (Zhang et al., 2023a)										VizSeq (Wang et al., 2019)
VizSmith (Bavishi et al., 2021)										Wandb (Weights and Biases, 2021)
Weedle (Kwon et al., 2023)										What-if Tool (Wexler et al., 2019)
Whatlies (Warmerdam et al., 2020)										Wrex (Drosos et al., 2020)
ydata-profiling (Brugman, 2019)

Appendix B Data Collection Details

To study how researchers and practitioners design interactive visualization tools for computational notebooks, we collected and analyzed 64 academic papers and 105 systems in the wild. We define notebook visualization tools as systems that can display interactive visualizations in Python computational notebooks.

Literature Collection. We searched Google Scholar for notebook visualization tools and performed forward and backward reference searches to snowball the results. The venues of collected papers range from scientific journals (e.g., Bioinformatics andFrontiers in Neuroinformatics) to human-computer interaction and machine learning conferences (e.g., VIS, CHI, and NeurIPS).

Visualization Package Collection. We first scraped 8.6 million notebooks with .ipynb extension from GitHub. Each notebook file is a JSON file containing metadata about the notebook and the notebook cells. The notebook cells contain information about the cell type, the source code or text of the cell, and any output generated by the cell. We pruned the scraped notebooks to only those containing interactive components by searching for script tags in cell outputs of type text/html. If a cell was deemed a potential candidate, we extracted the associated source code for that cell. Next, for each candidate interactive notebook, we identified all modules in the notebook by parsing it as an abstract syntax tree and looking for import statements. Finally, we spliced the last line of the source code for the candidate cell into its individual variable components and checked if these matched any of the imported modules or their aliases.

Automating this procedure across 8.6 million notebooks, we built a comprehensive list of 984 Python packages that were potential visualization tools. Since this list of packages contained false positives (not all identified packages were interactive visualization tools), we manually examined each package to verify if it was an interactive visualization tool by looking at the source code and documentation for the package and its usage in notebooks. In total, we identified 105 packages that were also interactive visualization tools.

Appendix C Implementation Details

Depending on the need for a backend server, visualization-notebook communication, needed data types, and display styles, there are multiple methods with varying difficulties to implement notebook visualization tools. Note that some methods are only compatible with specific notebook platforms (e.g., JupyterLab, Colab, VSCode, and Kaggle Notebook).

C.1. With Backend Servers

To implement notebook visualization tools that require a backend server, the developer needs to configure the server to support notebooks and establish callback functions to share states with the notebook. The server can either be run directly from the notebook environment or externally. The front-end of the tool can then be displayed in the notebook using the notebook’s native HTML display. It is important to separate the server from the main thread if it is run directly from the notebook to avoid blocking the Python kernel. For example, Jupyter-Dash (Parmer, 2020) and LIT (Tenney et al., 2020) use this method with a Flask backend server and a direct WSGI server, respectively.

C.2. Without Backend Servers

If the tool does not require a server, several implementation methods depend on the visualization-notebook communications.

C.2.1. No Direct Communication.

If a web-based visualization tool does not communicate with the notebook environment, the developer can simply use the notebook’s native HTML display to show the tool a notebook cell. The HTML display internally uses iframe to embed any web documents.

C.2.2. One-way Communication.

To pass data from the notebook Python kernel to the visualization tool, one can use the Web standard’s postMessage method to send serialized Python objects as JSON text to the visualization tool’s iframe. See NOVA (Wang et al., 2022d) for more details and examples about this approach. Example tools include GAM Changer (Wang et al., 2022b) and TimberTrek (Wang et al., 2022e).

Alternatively, developers can use existing interactive visualization packages such as Plotly (Sievert et al., 2017), Bokeh (Bokeh Development Team, 2014), Altair (VanderPlas et al., 2018), and Panel (Rudiger, 2021) as building blocks to implement their visualization tools. Then, the developer can use these packages’ APIs to pass data from notebooks to the visualization tools. However, this approach is less customizable, and it is best suited for simpler tools. Example tools include InterpretML (Nori et al., 2019) and Nilearn (Abraham et al., 2014).

C.2.3. Bidirectional Communication.

To send data back from the visualization tool to the Python kernel, the developer needs to use platform-specific solutions, which vary across platforms because notebook platforms have different security protocols. For Jupyter Notebook and JupyterLab, one can use ipywidget with the comm protocol to synchronize states between the visualization tool and the notebook. Example tools include Mage (Kery et al., 2020) and pydec (Uber, 2016).

C.3. Access and Modify Code and Text

To access and modify notebook content outside of the Python kernel, such as raw code and text (§ 5.2), visualization tool developers need to use platform-specific APIs. For Jupyter notebooks, the developer can use Jupyter Notebook extension and JupyterLab extension APIs to read and write the notebook content. visualization tools using this method include B2 (Wu et al., 2020) and Wrex (Drosos et al., 2020).

C.4. Always-on Display

If a developer intends to implement an always-on display (§ 5.3) for their notebook visualization tool, they can use platform-specific APIs. For JupyterLab, the developer can implement the tool as a JupyterLab extension, which enables the display on persistent panels outside of the notebook’s main UI. Examples of such implementations include NVDashboard (NVIDIA, 2021) and AutoProfiler (Epperson et al., 2023). If the visualization tool does not require extensive visualization customization, the developer can also use existing visualization packages that support persistent display (e.g., Jupyter-Dash (Parmer, 2020)) to implement the tool. Alternatively, the developer can develop their visualization tool using a traditional on-demand display and instruct users to use StickyLand (Wang et al., 2022a) to enable persistent display. StickyLand allows users to easily create persistent “sticky” cells and dashboards by dragging any notebook cell to the edge of the notebook’s UI (Fig. 11).