-
Evaluating Text-to-Image Synthesis: Survey and Taxonomy of Image Quality Metrics
Authors:
Sebastian Hartwig,
Dominik Engel,
Leon Sick,
Hannah Kniesel,
Tristan Payer,
Poonam Poonam,
Michael Glöckler,
Alex Bäuerle,
Timo Ropinski
Abstract:
Recent advances in text-to-image synthesis enabled through a combination of language and vision foundation models have led to a proliferation of the tools available and an increased attention to the field. When conducting text-to-image synthesis, a central goal is to ensure that the content between text and image is aligned. As such, there exist numerous evaluation metrics that aim to mimic human…
▽ More
Recent advances in text-to-image synthesis enabled through a combination of language and vision foundation models have led to a proliferation of the tools available and an increased attention to the field. When conducting text-to-image synthesis, a central goal is to ensure that the content between text and image is aligned. As such, there exist numerous evaluation metrics that aim to mimic human judgement. However, it is often unclear which metric to use for evaluating text-to-image synthesis systems as their evaluation is highly nuanced. In this work, we provide a comprehensive overview of existing text-to-image evaluation metrics. Based on our findings, we propose a new taxonomy for categorizing these metrics. Our taxonomy is grounded in the assumption that there are two main quality criteria, namely compositionality and generality, which ideally map to human preferences. Ultimately, we derive guidelines for practitioners conducting text-to-image evaluation, discuss open challenges of evaluation mechanisms, and surface limitations of current metrics.
△ Less
Submitted 15 April, 2024; v1 submitted 18 March, 2024;
originally announced March 2024.
-
Materials for Fusion Energy: Key Challenges and Guiding Principles for Develo** Under High Design Uncertainty
Authors:
David Cohen-Tanugi,
Myles G. Stapelberg,
Michael P. Short,
Sara E. Ferry,
Dennis G. Whyte,
Zachary S. Hartwig,
Tonio Buonassisi
Abstract:
Fusion energy is at an important inflection point in its development: multiple government agencies and private companies are now planning fusion pilot plants to deliver electricity to the grid in the next decade. However, realizing fusion as a technically and economically viable energy source depends on develo** and qualifying materials that can withstand the extreme environment inside a fusion…
▽ More
Fusion energy is at an important inflection point in its development: multiple government agencies and private companies are now planning fusion pilot plants to deliver electricity to the grid in the next decade. However, realizing fusion as a technically and economically viable energy source depends on develo** and qualifying materials that can withstand the extreme environment inside a fusion power plant. This Perspective seeks to engage the materials science community in this effort. We first outline the principal materials challenges and research opportunities for fusion. Next, we argue that fusion is distinct from other energy applications with respect to materials, not just in the magnitude and complexity of the technical challenges but also in the present level of uncertainty in materials design requirements. To address this, we finally propose a research framework based on an iterative co-evolution of materials science and fusion power plant design requirements.
△ Less
Submitted 20 November, 2023;
originally announced November 2023.
-
ClusterNet: A Perception-Based Clustering Model for Scattered Data
Authors:
Sebastian Hartwig,
Christian van Onzenoodt,
Dominik Engel,
Pedro Hermosilla,
Timo Ropinski
Abstract:
Visualizations for scattered data are used to make users understand certain attributes of their data by solving different tasks, e.g. correlation estimation, outlier detection, cluster separation. In this paper, we focus on the later task, and develop a technique that is aligned to human perception, that can be used to understand how human subjects perceive clusterings in scattered data and possib…
▽ More
Visualizations for scattered data are used to make users understand certain attributes of their data by solving different tasks, e.g. correlation estimation, outlier detection, cluster separation. In this paper, we focus on the later task, and develop a technique that is aligned to human perception, that can be used to understand how human subjects perceive clusterings in scattered data and possibly optimize for better understanding. Cluster separation in scatterplots is a task that is typically tackled by widely used clustering techniques, such as for instance k-means or DBSCAN. However, as these algorithms are based on non-perceptual metrics, we can show in our experiments, that their output do not reflect human cluster perception. We propose a learning strategy which directly operates on scattered data. To learn perceptual cluster separation on this data, we crowdsourced a large scale dataset, consisting of 7,320 point-wise cluster affiliations for bivariate data, which has been labeled by 384 human crowd workers. Based on this data, we were able to train ClusterNet, a point-based deep learning model, trained to reflect human perception of cluster separability. In order to train ClusterNet on human annotated data, we use a PointNet++ architecture enabling inference on point clouds directly. In this work, we provide details on how we collected our dataset, report statistics of the resulting annotations, and investigate perceptual agreement of cluster separation for real-world data. We further report the training and evaluation protocol of ClusterNet and introduce a novel metric, that measures the accuracy between a clustering technique and a group of human annotators. Finally, we compare our approach against existing state-of-the-art clustering techniques and can show, that ClusterNet is able to generalize to unseen and out of scope data.
△ Less
Submitted 6 March, 2024; v1 submitted 27 April, 2023;
originally announced April 2023.
-
AutoTherm: A Dataset and Benchmark for Thermal Comfort Estimation Indoors and in Vehicles
Authors:
Mark Colley,
Sebastian Hartwig,
Albin Zeqiri,
Timo Ropinski,
Enrico Rukzio
Abstract:
Thermal comfort inside buildings is a well-studied field where human judgment for thermal comfort is collected and may be used for automatic thermal comfort estimation. However, indoor scenarios are rather static in terms of thermal state changes and, thus, cannot be applied to dynamic conditions, e.g., inside a vehicle. In this work, we present our findings of a gap between building and in-vehicl…
▽ More
Thermal comfort inside buildings is a well-studied field where human judgment for thermal comfort is collected and may be used for automatic thermal comfort estimation. However, indoor scenarios are rather static in terms of thermal state changes and, thus, cannot be applied to dynamic conditions, e.g., inside a vehicle. In this work, we present our findings of a gap between building and in-vehicle scenarios regarding thermal comfort estimation. We provide evidence by comparing deep neural classifiers for thermal comfort estimation for indoor and in-vehicle conditions. Further, we introduce a temporal dataset for indoor predictions incorporating 31 input signals and self-labeled user ratings by 18 subjects in a self-built climatic chamber. For in-vehicle scenarios, we acquired a second dataset featuring human judgments from 20 subjects in a BMW 3 Series. Our experimental results indicate superior performance for estimations from time series data over single vector input. Leveraging modern machine learning architectures enables us to recognize human thermal comfort states and estimate future states automatically. We provide details on training a recurrent network-based classifier and perform an initial performance benchmark of the proposed dataset. Ultimately, we compare our collected dataset to publicly available thermal comfort datasets.
△ Less
Submitted 21 May, 2024; v1 submitted 15 November, 2022;
originally announced November 2022.
-
Monocular Depth Decomposition of Semi-Transparent Volume Renderings
Authors:
Dominik Engel,
Sebastian Hartwig,
Timo Ropinski
Abstract:
Neural networks have shown great success in extracting geometric information from color images. Especially, monocular depth estimation networks are increasingly reliable in real-world scenes. In this work we investigate the applicability of such monocular depth estimation networks to semi-transparent volume rendered images. As depth is notoriously difficult to define in a volumetric scene without…
▽ More
Neural networks have shown great success in extracting geometric information from color images. Especially, monocular depth estimation networks are increasingly reliable in real-world scenes. In this work we investigate the applicability of such monocular depth estimation networks to semi-transparent volume rendered images. As depth is notoriously difficult to define in a volumetric scene without clearly defined surfaces, we consider different depth computations that have emerged in practice, and compare state-of-the-art monocular depth estimation approaches for these different interpretations during an evaluation considering different degrees of opacity in the renderings. Additionally, we investigate how these networks can be extended to further obtain color and opacity information, in order to create a layered representation of the scene based on a single color image. This layered representation consists of spatially separated semi-transparent intervals that composite to the original input rendering. In our experiments we show that existing approaches to monocular depth estimation can be adapted to perform well on semi-transparent volume renderings, which has several applications in the area of scientific visualization, like re-composition with additional objects and labels or additional shading.
△ Less
Submitted 18 February, 2023; v1 submitted 27 June, 2022;
originally announced June 2022.
-
An accelerator facility for intermediate energy proton irradiation and testing of nuclear materials
Authors:
S. J. Jepeal,
A. Danagoulian,
L. A. Kesler,
D. A. Korsun,
H. Y. Lee,
N. Schwartz,
B. N. Sorbom,
E. Velez Lopez,
Z. S. Hartwig
Abstract:
The bulk irradiation of materials with 10-30 MeV protons promises to advance the study of radiation damage for fission and fusion power plants. Intermediate energy proton beams can now be dedicated to materials irradiation within university-scale laboratories. This paper describes the first such facility, with an Ionetix ION-12SC cyclotron producing 12 MeV proton beams. Samples are mm-scale tensil…
▽ More
The bulk irradiation of materials with 10-30 MeV protons promises to advance the study of radiation damage for fission and fusion power plants. Intermediate energy proton beams can now be dedicated to materials irradiation within university-scale laboratories. This paper describes the first such facility, with an Ionetix ION-12SC cyclotron producing 12 MeV proton beams. Samples are mm-scale tensile specimens with thicknesses up to 300 um, mounted to a cooled beam target with control over temperature. A specialized tensile tester for radioactive specimens at high temperature (500+ °C) and/or vacuum represents the conditions in fission and fusion systems, while a digital image correlation system remotely measures strain. Overall, the facility provides university-scale irradiation and testing capability with intermediate energy protons to complement traditional in-core fission reactor and micro-scale ion irradiation. This facility demonstrates that bulk proton irradiation is a scalable and effective approach for nuclear materials research, down-selection, and qualification.
△ Less
Submitted 20 November, 2020; v1 submitted 12 October, 2020;
originally announced October 2020.
-
Training Object Detectors on Synthetic Images Containing Reflecting Materials
Authors:
Sebastian Hartwig,
Timo Ropinski
Abstract:
One of the grand challenges of deep learning is the requirement to obtain large labeled training data sets. While synthesized data sets can be used to overcome this challenge, it is important that these data sets close the reality gap, i.e., a model trained on synthetic image data is able to generalize to real images. Whereas, the reality gap can be considered bridged in several application scenar…
▽ More
One of the grand challenges of deep learning is the requirement to obtain large labeled training data sets. While synthesized data sets can be used to overcome this challenge, it is important that these data sets close the reality gap, i.e., a model trained on synthetic image data is able to generalize to real images. Whereas, the reality gap can be considered bridged in several application scenarios, training on synthesized images containing reflecting materials requires further research. Since the appearance of objects with reflecting materials is dominated by the surrounding environment, this interaction needs to be considered during training data generation. Therefore, within this paper we examine the effect of reflecting materials in the context of synthetic image generation for training object detectors. We investigate the influence of rendering approach used for image synthesis, the effect of domain randomization, as well as the amount of used training data. To be able to compare our results to the state-of-the-art, we focus on indoor scenes as they have been investigated extensively. Within this scenario, bathroom furniture is a natural choice for objects with reflecting materials, for which we report our findings on real and synthetic testing data.
△ Less
Submitted 29 March, 2019;
originally announced April 2019.
-
Neutron diagnostics for the physics of a high-field, compact, $Q\geq1$ tokamak
Authors:
R. A. Tinguely,
A. Rosenthal,
R. Simpson,
S. B. Ballinger,
A. J. Creely,
S. Frank,
A. Q. Kuang,
B. L. Linehan,
W. McCarthy,
L. M. Milanese,
K. J. Montes,
T. Mouratidis,
J. F. Picard,
P. Rodriguez-Fernandez,
A. J. Sandberg,
F. Sciortino,
E. A. Tolman,
M. Zhou,
B. N. Sorbom,
Z. S. Hartwig,
A. E. White
Abstract:
Advancements in high temperature superconducting technology have opened a path toward high-field, compact fusion devices. This new parameter space introduces both opportunities and challenges for diagnosis of the plasma. This paper presents a physics review of a neutron diagnostic suite for a SPARC-like tokamak [Greenwald et al 2018 doi:10.7910/DVN/OYYBNU]. A notional neutronics model was construc…
▽ More
Advancements in high temperature superconducting technology have opened a path toward high-field, compact fusion devices. This new parameter space introduces both opportunities and challenges for diagnosis of the plasma. This paper presents a physics review of a neutron diagnostic suite for a SPARC-like tokamak [Greenwald et al 2018 doi:10.7910/DVN/OYYBNU]. A notional neutronics model was constructed using plasma parameters from a conceptual device, called the MQ1 (Mission $Q \geq 1$) tokamak. The suite includes time-resolved micro-fission chamber (MFC) neutron flux monitors, energy-resolved radial and tangential magnetic proton recoil (MPR) neutron spectrometers, and a neutron camera system (radial and off-vertical) for spatially-resolved measurements of neutron emissivity. Geometries of the tokamak, neutron source, and diagnostics were modeled in the Monte Carlo N-Particle transport code MCNP6 to simulate expected signal and background levels of particle fluxes and energy spectra. From these, measurements of fusion power, neutron flux and fluence are feasible by the MFCs, and the number of independent measurements required for 95% confidence of a fusion gain $Q \geq 1$ is assessed. The MPR spectrometer is found to consistently overpredict the ion temperature and also have a 1000$\times$ improved detection of alpha knock-on neutrons compared to previous experiments. The deuterium-tritium fuel density ratio, however, is measurable in this setup only for trace levels of tritium, with an upper limit of $n_T/n_D \approx 6\%$, motivating further diagnostic exploration. Finally, modeling suggests that in order to adequately measure the self-heating profile, the neutron camera system will require energy and pulse-shape discrimination to suppress otherwise overwhelming fluxes of low energy neutrons and gamma radiation.
*Co-first-authorship
△ Less
Submitted 22 March, 2019;
originally announced March 2019.
-
The magnetic ground state properties of non-centrosymmetric CePt$_3$B$_{1-x}$Si$_x$
Authors:
D. Rauch,
P. Horenburg,
S. Hartwig,
F. J. Litterst,
S. Süllow,
H. Luetkens,
C. Baines,
S. Yamazaki,
H. Hidaka,
H. Amitsuka,
E. Bauer
Abstract:
We present a study of the alloying series of the non-centrosymmetric $f$-electron intermetallic CePt$_3$B$_{1-x}$Si$_x$ by means of muon spin rotation and relaxation measurements. In addition, we include a high pressure magnetization investigation of the stoichiometric parent compound CePt$_3$B. From our data we establish the nature of the magnetic ground state properties of the series, derive the…
▽ More
We present a study of the alloying series of the non-centrosymmetric $f$-electron intermetallic CePt$_3$B$_{1-x}$Si$_x$ by means of muon spin rotation and relaxation measurements. In addition, we include a high pressure magnetization investigation of the stoichiometric parent compound CePt$_3$B. From our data we establish the nature of the magnetic ground state properties of the series, derive the ordered magnetic moment as function of stoichiometry and gain insight into the evolution of the symmetry of the ordered magnetic state with $x$. We thus can verify the notion that the behavior of the sample series can essentially be understood within the framework of the Doniach phase diagram. Further, our findings raise the issue of the role the Dzyaloshinskii-Moriya magnetic interaction plays in correlated electron materials, and its effect on magnetic fluctuations in such materials.
△ Less
Submitted 10 June, 2015;
originally announced June 2015.
-
Coexistence of different magnetic moments in CeRuSn probed by polarized neutrons
Authors:
K. Prokes,
S. Hartwig,
A. Gukasov,
J. A. Mydosh,
Y. -K. Huang,
O. Niehaus,
R. Pottgen
Abstract:
We report on the spin densities in CeRuSn determined at elevated and at low temperatures using polarized neutron diffraction. At 285 K, where the CeRuSn crystal structure, commensurate with the CeCoAl type, contains two different crystallographic Ce sites, we observe that one Ce site is clearly more susceptible to the applied magnetic field whereas the other is hardly polarizable. This finding cle…
▽ More
We report on the spin densities in CeRuSn determined at elevated and at low temperatures using polarized neutron diffraction. At 285 K, where the CeRuSn crystal structure, commensurate with the CeCoAl type, contains two different crystallographic Ce sites, we observe that one Ce site is clearly more susceptible to the applied magnetic field whereas the other is hardly polarizable. This finding clearly documents that distnictly different local environment of the two Ce sites causes the Ce ions to split into magnetic Ce3+ and non-magnetic Ce(4-delta)+ valence states. With lowering the temperature, the crystal structure transforms to a structure incommensurately modulated along the c axis. This leads to new inequivalent crystallographic Ce sites resulting in a re-distribution of spin densities. Our analysis using the simplest structural approximant shows that in this metallic system Ce ions co-exist in different valence states. Localized 4f states that fulfill the third Hund's rule are found to be close to the ideal Ce3+ state (at sites with the largest Ce-Ru interatomic distances) whereas Ce(4-delta)+ valence states are found to be itinerant and situated at Ce sites with much shorter Ce-Ru distances. The similarity to the famous alpha-gamma transition in elemental cerium is discussed.
△ Less
Submitted 3 November, 2014;
originally announced November 2014.
