-
Understanding Business Users' Data-Driven Decision-Making: Practices, Challenges, and Opportunities
Authors:
Sneha Gathani,
Zhicheng Liu,
Peter J. Haas,
Çağatay Demiralp
Abstract:
Business users perform data analysis to inform decisions for improving business processes and outcomes despite having limited formal technical training. While earlier work has focused on data analysts' and data scientists' practices and challenges, little is known about business users' decision-making practices and how they incorporate data and visual analytics into their workflows. To address thi…
▽ More
Business users perform data analysis to inform decisions for improving business processes and outcomes despite having limited formal technical training. While earlier work has focused on data analysts' and data scientists' practices and challenges, little is known about business users' decision-making practices and how they incorporate data and visual analytics into their workflows. To address this gap, we first conduct an interview study with 22 business users to understand the general practices and challenges in their data-driven decision-making processes. We contribute an end-to-end model of business users' data-driven decision-making processes elaborating the tasks, tools, and challenges at each stage. We find that business users analyze data without relying on data analysts due to various practical constraints and considerations. However, their existing tools are inadequate, particularly in hel** understand the relationship between data variables and business goals and facilitating the exploration of what-if scenarios. These findings suggest a need for advanced predictive and prescriptive analytics (PPA) tools to support what-if analysis. Motivated by this need, we perform a follow-up, task-based study to understand PPA's role and potential in business users' decision-making processes. We find that PPA helps improve efficiency and confidence in decision-making. However, business users also believe that PPA-powered what-if analysis tools are currently in their nascent stages and report improvements before fully integrating them into their decision-making processes. Building upon these findings, we discuss the opportunities and challenges in incorporating PPA into data-driven decision-making and its implications for future data and visual analytics systems.
△ Less
Submitted 17 October, 2023; v1 submitted 27 December, 2022;
originally announced December 2022.
-
A Grammar-Based Approach for Applying Visualization Taxonomies to Interaction Logs
Authors:
Sneha Gathani,
Shayan Monadjemi,
Alvitta Ottley,
Leilani Battle
Abstract:
Researchers collect large amounts of user interaction data with the goal of map** user's workflows and behaviors to their higher-level motivations, intuitions, and goals. Although the visual analytics community has proposed numerous taxonomies to facilitate this map** process, no formal methods exist for systematically applying these existing theories to user interaction logs. This paper seeks…
▽ More
Researchers collect large amounts of user interaction data with the goal of map** user's workflows and behaviors to their higher-level motivations, intuitions, and goals. Although the visual analytics community has proposed numerous taxonomies to facilitate this map** process, no formal methods exist for systematically applying these existing theories to user interaction logs. This paper seeks to bridge the gap between visualization task taxonomies and interaction log data by making the taxonomies more actionable for interaction log analysis. To achieve this, we leverage structural parallels between how people express themselves through interactions and language by reformulating existing theories as regular grammars. We represent interactions as terminals within a regular grammar, similar to the role of individual words in a language, and patterns of interactions or non-terminals as regular expressions over these terminals to capture common language patterns. To demonstrate our approach, we generate regular grammars for seven visualization taxonomies and develop code to apply them to three interaction log datasets. In analyzing our results, we find that existing taxonomies at the low-level (i.e., terminals) show mixed results in expressing multiple interaction log datasets, and taxonomies at the high-level (i.e., regular expressions) have limited expressiveness, due to primarily two challenges: inconsistencies in interaction log dataset granularity and structure, and under-expressiveness of certain terminals. Based on our findings, we suggest new research directions for the visualization community for augmenting existing taxonomies, develo** new ones, and building better interaction log recording processes to facilitate the data-driven development of user behavior taxonomies.
△ Less
Submitted 5 April, 2022; v1 submitted 10 January, 2022;
originally announced January 2022.
-
Augmenting Decision Making via Interactive What-If Analysis
Authors:
Sneha Gathani,
Madelon Hulsebos,
James Gale,
Peter J. Haas,
Çağatay Demiralp
Abstract:
The fundamental goal of business data analysis is to improve business decisions using data. Business users often make decisions to achieve key performance indicators (KPIs) such as increasing customer retention or sales, or decreasing costs. To discover the relationship between data attributes hypothesized to be drivers and those corresponding to KPIs of interest, business users currently need to…
▽ More
The fundamental goal of business data analysis is to improve business decisions using data. Business users often make decisions to achieve key performance indicators (KPIs) such as increasing customer retention or sales, or decreasing costs. To discover the relationship between data attributes hypothesized to be drivers and those corresponding to KPIs of interest, business users currently need to perform lengthy exploratory analyses. This involves considering multitudes of combinations and scenarios and performing slicing, dicing, and transformations on the data accordingly, e.g., analyzing customer retention across quarters of the year or suggesting optimal media channels across strata of customers. However, the increasing complexity of datasets combined with the cognitive limitations of humans makes it challenging to carry over multiple hypotheses, even for simple datasets. Therefore mentally performing such analyses is hard. Existing commercial tools either provide partial solutions or fail to cater to business users altogether. Here we argue for four functionalities to enable business users to interactively learn and reason about the relationships between sets of data attributes thereby facilitating data-driven decision making. We implement these functionalities in SystemD, an interactive visual data analysis system enabling business users to experiment with the data by asking what-if questions. We evaluate the system through three business use cases: marketing mix modeling, customer retention analysis, and deal closing analysis, and report on feedback from multiple business users. Users find the SystemD functionalities highly useful for quick testing and validation of their hypotheses around their KPIs of interest, addressing their unmet analysis needs. The feedback also suggests that the UX design can be enhanced to further improve the understandability of these functionalities.
△ Less
Submitted 8 February, 2022; v1 submitted 13 September, 2021;
originally announced September 2021.
-
Making Table Understanding Work in Practice
Authors:
Madelon Hulsebos,
Sneha Gathani,
James Gale,
Isil Dillig,
Paul Groth,
Çağatay Demiralp
Abstract:
Understanding the semantics of tables at scale is crucial for tasks like data integration, preparation, and search. Table understanding methods aim at detecting a table's topic, semantic column types, column relations, or entities. With the rise of deep learning, powerful models have been developed for these tasks with excellent accuracy on benchmarks. However, we observe that there exists a gap b…
▽ More
Understanding the semantics of tables at scale is crucial for tasks like data integration, preparation, and search. Table understanding methods aim at detecting a table's topic, semantic column types, column relations, or entities. With the rise of deep learning, powerful models have been developed for these tasks with excellent accuracy on benchmarks. However, we observe that there exists a gap between the performance of these models on these benchmarks and their applicability in practice. In this paper, we address the question: what do we need for these models to work in practice?
We discuss three challenges of deploying table understanding models and propose a framework to address them. These challenges include 1) difficulty in customizing models to specific domains, 2) lack of training data for typical database tables often found in enterprises, and 3) lack of confidence in the inferences made by models. We present SigmaTyper which implements this framework for the semantic column type detection task. SigmaTyper encapsulates a hybrid model trained on GitTables and integrates a lightweight human-in-the-loop approach to customize the model. Lastly, we highlight avenues for future research that further close the gap towards making table understanding effective in practice.
△ Less
Submitted 10 September, 2021;
originally announced September 2021.
