-
Enhancing Single-Frame Supervision for Better Temporal Action Localization
Authors:
Changjian Chen,
Jiashu Chen,
Weikai Yang,
Haoze Wang,
Johannes Knittel,
Xibin Zhao,
Steffen Koch,
Thomas Ertl,
Shixia Liu
Abstract:
Temporal action localization aims to identify the boundaries and categories of actions in videos, such as scoring a goal in a football match. Single-frame supervision has emerged as a labor-efficient way to train action localizers as it requires only one annotated frame per action. However, it often suffers from poor performance due to the lack of precise boundary annotations. To address this issu…
▽ More
Temporal action localization aims to identify the boundaries and categories of actions in videos, such as scoring a goal in a football match. Single-frame supervision has emerged as a labor-efficient way to train action localizers as it requires only one annotated frame per action. However, it often suffers from poor performance due to the lack of precise boundary annotations. To address this issue, we propose a visual analysis method that aligns similar actions and then propagates a few user-provided annotations (e.g. , boundaries, category labels) to similar actions via the generated alignments. Our method models the alignment between actions as a heaviest path problem and the annotation propagation as a quadratic optimization problem. As the automatically generated alignments may not accurately match the associated actions and could produce inaccurate localization results, we develop a storyline visualization to explain the localization results of actions and their alignments. This visualization facilitates users in correcting wrong localization results and misalignments. The corrections are then used to improve the localization results of other actions. The effectiveness of our method in improving localization performance is demonstrated through quantitative evaluation and a case study.
△ Less
Submitted 8 December, 2023;
originally announced December 2023.
-
Visual Analysis of Displacement Processes in Porous Media using Spatio-Temporal Flow Graphs
Authors:
Alexander Straub,
Nikolaos Karadimitriou,
Guido Reina,
Steffen Frey,
Holger Steeb,
Thomas Ertl
Abstract:
We developed a new approach comprised of different visualizations for the comparative spatio-temporal analysis of displacement processes in porous media. We aim to analyze and compare ensemble datasets from experiments to gain insight into the influence of different parameters on fluid flow. To capture the displacement of a defending fluid by an invading fluid, we first condense an input image ser…
▽ More
We developed a new approach comprised of different visualizations for the comparative spatio-temporal analysis of displacement processes in porous media. We aim to analyze and compare ensemble datasets from experiments to gain insight into the influence of different parameters on fluid flow. To capture the displacement of a defending fluid by an invading fluid, we first condense an input image series to a single time map. From this map, we generate a spatio-temporal flow graph covering the whole process. This graph is further simplified to only reflect topological changes in the movement of the invading fluid. Our interactive tools allow the visual analysis of these processes by visualizing the graph structure and the context of the experimental setup, as well as by providing charts for multiple metrics. We apply our approach to analyze and compare ensemble datasets jointly with domain experts, where we vary either fluid properties or the solid structure of the porous medium. We finally report the generated insights from the domain experts and discuss our contribution's advantages, generality, and limitations.
△ Less
Submitted 13 November, 2023; v1 submitted 27 July, 2023;
originally announced July 2023.
-
Visual Ensemble Analysis of Fluid Flow in Porous Media across Simulation Codes and Experiment
Authors:
Ruben Bauer,
Quynh Quang Ngo,
Guido Reina,
Steffen Frey,
Bernd Flemisch,
Helwig Hauser,
Thomas Ertl,
Michael Sedlmair
Abstract:
We study the question of how visual analysis can support the comparison of spatio-temporal ensemble data of liquid and gas flow in porous media. To this end, we focus on a case study, in which nine different research groups concurrently simulated the process of injecting CO2 into the subsurface. We explore different data aggregation and interactive visualization approaches to compare and analyze t…
▽ More
We study the question of how visual analysis can support the comparison of spatio-temporal ensemble data of liquid and gas flow in porous media. To this end, we focus on a case study, in which nine different research groups concurrently simulated the process of injecting CO2 into the subsurface. We explore different data aggregation and interactive visualization approaches to compare and analyze these nine simulations. In terms of data aggregation, one key component is the choice of similarity metrics that define the relation between the different simulations. We test different metrics and find that a fine-tuned machine-learning based metric provides the best visualization results. Based on that, we propose different visualization methods. For overviewing the data, we use dimensionality reduction methods that allow us to plot and compare the different simulations in a scatterplot. To show details about the spatio-temporal data of each individual simulation, we employ a space-time cube volume rendering. We use the resulting interactive, multi-view visual analysis tool to explore the nine simulations and also to compare them to data from experimental setups. Our main findings include new insights into ranking of simulation results with respect to experimental data, and the development of gravity fingers in simulations.
△ Less
Submitted 27 November, 2023; v1 submitted 26 January, 2023;
originally announced February 2023.
-
Droplet-Local Line Integration for Multiphase Flow
Authors:
Alexander Straub,
Sebastian Boblest,
Grzegorz K. Karch,
Filip Sadlo,
Thomas Ertl
Abstract:
Line integration of stream-, streak-, and pathlines is widely used and popular for visualizing single-phase flow. In multiphase flow, i.e., where the fluid consists, e.g., of a liquid and a gaseous phase, these techniques could also provide valuable insights into the internal flow of droplets and ligaments and thus into their dynamics. However, since such structures tend to act as entities, high t…
▽ More
Line integration of stream-, streak-, and pathlines is widely used and popular for visualizing single-phase flow. In multiphase flow, i.e., where the fluid consists, e.g., of a liquid and a gaseous phase, these techniques could also provide valuable insights into the internal flow of droplets and ligaments and thus into their dynamics. However, since such structures tend to act as entities, high translational and rotational velocities often obfuscate their detail. As a remedy, we present a method for deriving a droplet-local velocity field, using a decomposition of the original velocity field removing translational and rotational velocity parts, and adapt path- and streaklines. Generally, the resulting integral lines are thus shorter and less tangled, which simplifies their analysis. We demonstrate and discuss the utility of our approach on droplets in two-phase flow data and visualize the removed velocity parts employing glyphs for context.
△ Less
Submitted 19 December, 2022; v1 submitted 27 July, 2022;
originally announced August 2022.
-
Real-Time Visual Analysis of High-Volume Social Media Posts
Authors:
Johannes Knittel,
Steffen Koch,
Tan Tang,
Wei Chen,
Yingcai Wu,
Shixia Liu,
Thomas Ertl
Abstract:
Breaking news and first-hand reports often trend on social media platforms before traditional news outlets cover them. The real-time analysis of posts on such platforms can reveal valuable and timely insights for journalists, politicians, business analysts, and first responders, but the high number and diversity of new posts pose a challenge. In this work, we present an interactive system that ena…
▽ More
Breaking news and first-hand reports often trend on social media platforms before traditional news outlets cover them. The real-time analysis of posts on such platforms can reveal valuable and timely insights for journalists, politicians, business analysts, and first responders, but the high number and diversity of new posts pose a challenge. In this work, we present an interactive system that enables the visual analysis of streaming social media data on a large scale in real-time. We propose an efficient and explainable dynamic clustering algorithm that powers a continuously updated visualization of the current thematic landscape as well as detailed visual summaries of specific topics of interest. Our parallel clustering strategy provides an adaptive stream with a digestible but diverse selection of recent posts related to relevant topics. We also integrate familiar visual metaphors that are highly interlinked for enabling both explorative and more focused monitoring tasks. Analysts can gradually increase the resolution to dive deeper into particular topics. In contrast to previous work, our system also works with non-geolocated posts and avoids extensive preprocessing such as detecting events. We evaluated our dynamic clustering algorithm and discuss several use cases that show the utility of our system.
△ Less
Submitted 6 August, 2021;
originally announced August 2021.
-
Efficient Sparse Spherical k-Means for Document Clustering
Authors:
Johannes Knittel,
Steffen Koch,
Thomas Ertl
Abstract:
Spherical k-Means is frequently used to cluster document collections because it performs reasonably well in many settings and is computationally efficient. However, the time complexity increases linearly with the number of clusters k, which limits the suitability of the algorithm for larger values of k depending on the size of the collection. Optimizations targeted at the Euclidean k-Means algorit…
▽ More
Spherical k-Means is frequently used to cluster document collections because it performs reasonably well in many settings and is computationally efficient. However, the time complexity increases linearly with the number of clusters k, which limits the suitability of the algorithm for larger values of k depending on the size of the collection. Optimizations targeted at the Euclidean k-Means algorithm largely do not apply because the cosine distance is not a metric. We therefore propose an efficient indexing structure to improve the scalability of Spherical k-Means with respect to k. Our approach exploits the sparsity of the input vectors and the convergence behavior of k-Means to reduce the number of comparisons on each iteration significantly.
△ Less
Submitted 30 July, 2021;
originally announced August 2021.
-
ELSKE: Efficient Large-Scale Keyphrase Extraction
Authors:
Johannes Knittel,
Steffen Koch,
Thomas Ertl
Abstract:
Keyphrase extraction methods can provide insights into large collections of documents such as social media posts. Existing methods, however, are less suited for the real-time analysis of streaming data, because they are computationally too expensive or require restrictive constraints regarding the structure of keyphrases. We propose an efficient approach to extract keyphrases from large document c…
▽ More
Keyphrase extraction methods can provide insights into large collections of documents such as social media posts. Existing methods, however, are less suited for the real-time analysis of streaming data, because they are computationally too expensive or require restrictive constraints regarding the structure of keyphrases. We propose an efficient approach to extract keyphrases from large document collections and show that the method also performs competitively on individual documents.
△ Less
Submitted 10 February, 2021;
originally announced February 2021.
-
Visual Neural Decomposition to Explain Multivariate Data Sets
Authors:
Johannes Knittel,
Andres Lalama,
Steffen Koch,
Thomas Ertl
Abstract:
Investigating relationships between variables in multi-dimensional data sets is a common task for data analysts and engineers. More specifically, it is often valuable to understand which ranges of which input variables lead to particular values of a given target variable. Unfortunately, with an increasing number of independent variables, this process may become cumbersome and time-consuming due to…
▽ More
Investigating relationships between variables in multi-dimensional data sets is a common task for data analysts and engineers. More specifically, it is often valuable to understand which ranges of which input variables lead to particular values of a given target variable. Unfortunately, with an increasing number of independent variables, this process may become cumbersome and time-consuming due to the many possible combinations that have to be explored. In this paper, we propose a novel approach to visualize correlations between input variables and a target output variable that scales to hundreds of variables. We developed a visual model based on neural networks that can be explored in a guided way to help analysts find and understand such correlations. First, we train a neural network to predict the target from the input variables. Then, we visualize the inner workings of the resulting model to help understand relations within the data set. We further introduce a new regularization term for the backpropagation algorithm that encourages the neural network to learn representations that are easier to interpret visually. We apply our method to artificial and real-world data sets to show its utility.
△ Less
Submitted 11 September, 2020;
originally announced September 2020.
-
PlotThread: Creating Expressive Storyline Visualizations using Reinforcement Learning
Authors:
Tan Tang,
Renzhong Li,
Xinke Wu,
Shuhan Liu,
Johannes Knittel,
Steffen Koch,
Thomas Ertl,
Lingyun Yu,
Peiran Ren,
Yingcai Wu
Abstract:
Storyline visualizations are an effective means to present the evolution of plots and reveal the scenic interactions among characters. However, the design of storyline visualizations is a difficult task as users need to balance between aesthetic goals and narrative constraints. Despite that the optimization-based methods have been improved significantly in terms of producing aesthetic and legible…
▽ More
Storyline visualizations are an effective means to present the evolution of plots and reveal the scenic interactions among characters. However, the design of storyline visualizations is a difficult task as users need to balance between aesthetic goals and narrative constraints. Despite that the optimization-based methods have been improved significantly in terms of producing aesthetic and legible layouts, the existing (semi-) automatic methods are still limited regarding 1) efficient exploration of the storyline design space and 2) flexible customization of storyline layouts. In this work, we propose a reinforcement learning framework to train an AI agent that assists users in exploring the design space efficiently and generating well-optimized storylines. Based on the framework, we introduce PlotThread, an authoring tool that integrates a set of flexible interactions to support easy customization of storyline visualizations. To seamlessly integrate the AI agent into the authoring process, we employ a mixed-initiative approach where both the agent and designers work on the same canvas to boost the collaborative design of storylines. We evaluate the reinforcement learning model through qualitative and quantitative experiments and demonstrate the usage of PlotThread using a collection of use cases.
△ Less
Submitted 1 September, 2020;
originally announced September 2020.
-
Towards realistic HPC models of the neuromuscular system
Authors:
Chris Bradley,
Nehzat Emamy,
Thomas Ertl,
Dominik Göddeke,
Andreas Hessenthaler,
Thomas Klotz,
Aaron Krämer,
Michael Krone,
Benjamin Maier,
Miriam Mehl,
Tobias Rau,
Oliver Röhrle
Abstract:
Realistic simulations of detailed, biophysics-based, multi-scale models require very high resolution and, thus, large-scale compute facilities. Existing simulation environments, especially for biomedical applications, are designed to allow for a high flexibility and generality in model development. Flexibility and model development, however, are often a limiting factor for large-scale simulations.…
▽ More
Realistic simulations of detailed, biophysics-based, multi-scale models require very high resolution and, thus, large-scale compute facilities. Existing simulation environments, especially for biomedical applications, are designed to allow for a high flexibility and generality in model development. Flexibility and model development, however, are often a limiting factor for large-scale simulations. Therefore, new models are typically tested and run on small-scale compute facilities. By using a detailed biophysics-based, chemo-electromechanical skeletal muscle model and the international open-source software library OpenCMISS as an example, we present an approach to upgrade an existing muscle simulation framework from a moderately parallel version towards a massively parallel one that scales both in terms of problem size and in terms of the number of parallel processes. For this purpose, we investigate different modeling, algorithmic and implementational aspects. We present improvements addressing both numerical and parallel scalability. In addition, our approach includes a novel visualization environment, which is based on the MegaMol environment capable of handling large amounts of simulated data. It offers a platform for fast visualization prototy**, distributed rendering, and advanced visualization techniques. We present results of a variety of scaling studies at the Tier-1 supercomputer HazelHen at the High Performance Computing Center Stuttgart (HLRS). We improve the overall runtime by a factor of up to 2.6 and achieved good scalability on up to 768 cores, where the previous implementation used only 4 cores.
△ Less
Submitted 9 February, 2018;
originally announced February 2018.
-
Visualization of Feature Separation in Advected Scalar Fields
Authors:
Grzegorz K. Karch,
Filip Sadlo,
Sebastian Boblest,
Moritz Ertl,
Bernhard Weigand,
Kelly Gaither,
Thomas Ertl
Abstract:
Scalar features in time-dependent fluid flow are traditionally visualized using 3D representation, and their topology changes over time are often conveyed with abstract graphs. Using such techniques, however, the structural details of feature separation and the temporal evolution of features undergoing topological changes are difficult to analyze. In this paper, we propose a novel approach for the…
▽ More
Scalar features in time-dependent fluid flow are traditionally visualized using 3D representation, and their topology changes over time are often conveyed with abstract graphs. Using such techniques, however, the structural details of feature separation and the temporal evolution of features undergoing topological changes are difficult to analyze. In this paper, we propose a novel approach for the spatio-temporal visualization of feature separation that segments feature volumes into regions with respect to their contribution to distinct features after separation. To this end, we employ particle-based feature tracking to find volumetric correspondences between features at two different instants of time. We visualize this segmentation by constructing mesh boundaries around each volume segment of a feature at the initial time that correspond to the separated features at the later time. To convey temporal evolution of the partitioning within the investigated time interval, we complement our approach with spatio-temporal separation surfaces. For the application of our approach to multiphase flow, we additionally present a feature-based corrector method to ensure phase-consistent particle trajectories. The utility of our technique is demonstrated by application to two-phase (liquid-gas) and multi-component (liquid-liquid) flows where the scalar field represents the fraction of one of the phases.
△ Less
Submitted 11 July, 2017; v1 submitted 15 May, 2017;
originally announced May 2017.
-
Flux-Limited Diffusion for Multiple Scattering in Participating Media
Authors:
David Koerner,
Jamie Portsmouth,
Filip Sadlo,
Thomas Ertl,
Bernd Eberhardt
Abstract:
For the rendering of multiple scattering effects in participating media, methods based on the diffusion approximation are an extremely efficient alternative to Monte Carlo path tracing. However, in sufficiently transparent regions, classical diffusion approximation suffers from non-physical radiative fluxes which leads to a poor match to correct light transport. In particular, this prevents the ap…
▽ More
For the rendering of multiple scattering effects in participating media, methods based on the diffusion approximation are an extremely efficient alternative to Monte Carlo path tracing. However, in sufficiently transparent regions, classical diffusion approximation suffers from non-physical radiative fluxes which leads to a poor match to correct light transport. In particular, this prevents the application of classical diffusion approximation to heterogeneous media, where opaque material is embedded within transparent regions. To address this limitation, we introduce flux-limited diffusion, a technique from the astrophysics domain. This method provides a better approximation to light transport than classical diffusion approximation, particularly when applied to heterogeneous media, and hence broadens the applicability of diffusion-based techniques. We provide an algorithm for flux-limited diffusion, which is validated using the transport theory for a point light source in an infinite homogeneous medium. We further demonstrate that our implementation of flux-limited diffusion produces more accurate renderings of multiple scattering in various heterogeneous datasets than classical diffusion approximation, by comparing both methods to ground truth renderings obtained via volumetric path tracing.
△ Less
Submitted 31 March, 2014;
originally announced March 2014.
