-
ManiFPT: Defining and Analyzing Fingerprints of Generative Models
Authors:
Hae ** Song,
Mahyar Khayatkhoei,
Wael AbdAlmageed
Abstract:
Recent works have shown that generative models leave traces of their underlying generative process on the generated samples, broadly referred to as fingerprints of a generative model, and have studied their utility in detecting synthetic images from real ones. However, the extend to which these fingerprints can distinguish between various types of synthetic image and help identify the underlying g…
▽ More
Recent works have shown that generative models leave traces of their underlying generative process on the generated samples, broadly referred to as fingerprints of a generative model, and have studied their utility in detecting synthetic images from real ones. However, the extend to which these fingerprints can distinguish between various types of synthetic image and help identify the underlying generative process remain under-explored. In particular, the very definition of a fingerprint remains unclear, to our knowledge. To that end, in this work, we formalize the definition of artifact and fingerprint in generative models, propose an algorithm for computing them in practice, and finally study its effectiveness in distinguishing a large array of different generative models. We find that using our proposed definition can significantly improve the performance on the task of identifying the underlying generative process from samples (model attribution) compared to existing methods. Additionally, we study the structure of the fingerprints, and observe that it is very predictive of the effect of different design choices on the generative process.
△ Less
Submitted 29 February, 2024; v1 submitted 15 February, 2024;
originally announced February 2024.
-
Fast Knowledge Graph Completion using Graphics Processing Units
Authors:
Chun-Hee Lee,
Dong-oh Kang,
Hwa Jeon Song
Abstract:
Knowledge graphs can be used in many areas related to data semantics such as question-answering systems, knowledge based systems. However, the currently constructed knowledge graphs need to be complemented for better knowledge in terms of relations. It is called knowledge graph completion. To add new relations to the existing knowledge graph by using knowledge graph embedding models, we have to ev…
▽ More
Knowledge graphs can be used in many areas related to data semantics such as question-answering systems, knowledge based systems. However, the currently constructed knowledge graphs need to be complemented for better knowledge in terms of relations. It is called knowledge graph completion. To add new relations to the existing knowledge graph by using knowledge graph embedding models, we have to evaluate $N\times N \times R$ vector operations, where $N$ is the number of entities and $R$ is the number of relation types. It is very costly.
In this paper, we provide an efficient knowledge graph completion framework on GPUs to get new relations using knowledge graph embedding vectors. In the proposed framework, we first define "transformable to a metric space" and then provide a method to transform the knowledge graph completion problem into the similarity join problem for a model which is "transformable to a metric space". After that, to efficiently process the similarity join problem, we derive formulas using the properties of a metric space. Based on the formulas, we develop a fast knowledge graph completion algorithm. Finally, we experimentally show that our framework can efficiently process the knowledge graph completion problem.
△ Less
Submitted 22 July, 2023;
originally announced July 2023.
-
Learning Robust Representations Of Generative Models Using Set-Based Artificial Fingerprints
Authors:
Hae ** Song,
Wael AbdAlmageed
Abstract:
With recent progress in deep generative models, the problem of identifying synthetic data and comparing their underlying generative processes has become an imperative task for various reasons, including fighting visual misinformation and source attribution. Existing methods often approximate the distance between the models via their sample distributions. In this paper, we approach the problem of f…
▽ More
With recent progress in deep generative models, the problem of identifying synthetic data and comparing their underlying generative processes has become an imperative task for various reasons, including fighting visual misinformation and source attribution. Existing methods often approximate the distance between the models via their sample distributions. In this paper, we approach the problem of fingerprinting generative models by learning representations that encode the residual artifacts left by the generative models as unique signals that identify the source models. We consider these unique traces (a.k.a. "artificial fingerprints") as representations of generative models, and demonstrate their usefulness in both the discriminative task of source attribution and the unsupervised task of defining a similarity between the underlying models. We first extend the existing studies on fingerprints of GANs to four representative classes of generative models (VAEs, Flows, GANs and score-based models), and demonstrate their existence and attributability. We then improve the stability and attributability of the fingerprints by proposing a new learning method based on set-encoding and contrastive training. Our set-encoder, unlike existing methods that operate on individual images, learns fingerprints from a \textit{set} of images. We demonstrate improvements in the stability and attributability through comparisons to state-of-the-art fingerprint methods and ablation studies. Further, our method employs contrastive training to learn an implicit similarity between models. We discover latent families of generative models using this metric in a standard hierarchical clustering algorithm.
△ Less
Submitted 4 June, 2022;
originally announced June 2022.
-
PES and XAS Study on Electronic Structures of Multiferroic RMnO$_3$ (R=Y, Er)
Authors:
J. -S. Kang,
S. W. Han,
J. -G. Park,
S. C. Wi,
S. S. Lee,
G. Kim,
H. J. Song,
H. J. Shin,
W. Jo,
B. I. Min
Abstract:
Electronic structures of multiferroic RMnO$_3$ (R=Y, Er) have been investigated by employing photoemission spectroscopy (PES) and x-ray absorption spectroscopy (XAS). We have found that Mn ions in RMnO$_3$ are in the trivalent high-spin state with the total spin of S=2. The occupied Mn ($d_{xz} - d_{yz}$) states lie deep below $\rm E_F$, while the occupied Mn ($d_{xy} - d_{x^2 -y^2}$) states ove…
▽ More
Electronic structures of multiferroic RMnO$_3$ (R=Y, Er) have been investigated by employing photoemission spectroscopy (PES) and x-ray absorption spectroscopy (XAS). We have found that Mn ions in RMnO$_3$ are in the trivalent high-spin state with the total spin of S=2. The occupied Mn ($d_{xz} - d_{yz}$) states lie deep below $\rm E_F$, while the occupied Mn ($d_{xy} - d_{x^2 -y^2}$) states overlap very much with the O $2p$ states. It is observed that the PES spectral intensity of Mn 3d states is negligible above the occupied O $2p$ bands, suggesting that YMnO$_3$ is likely to be a charge-transfer insulator. The Mn $d_{3z^2 -r^2}$ state is mostly unoccupied in the ferroelectric phase of YMnO$_3$.
△ Less
Submitted 22 November, 2004;
originally announced November 2004.
-
Electronic structures of Zn$_{1-x}$Co$_x$O using photoemission and x-ray absorption spectroscopy
Authors:
S. C. Wi,
J. -S. Kang,
J. H. Kim,
S. -B. Cho,
B. J. Kim,
S. Yoon,
B. J. Suh,
S. W. Han,
K. H. Kim,
K. J. Kim,
B. S. Kim,
H. J. Song,
H. J. Shin,
J. H. Shim,
B. I. Min
Abstract:
Electronic structures of Zn$_{1-x}$Co$_x$O have been investigated using photoemission spectroscopy (PES) and x-ray absorption spectroscopy (XAS). The Co 3d states are found to lie near the top of the O $2p$ valence band, with a peak around $\sim 3$ eV binding energy. The Co $2p$ XAS spectrum provides evidence that the Co ions in Zn$_{1-x}$Co$_{x}$O are in the divalent Co$^{2+}$ ($d^7$) states un…
▽ More
Electronic structures of Zn$_{1-x}$Co$_x$O have been investigated using photoemission spectroscopy (PES) and x-ray absorption spectroscopy (XAS). The Co 3d states are found to lie near the top of the O $2p$ valence band, with a peak around $\sim 3$ eV binding energy. The Co $2p$ XAS spectrum provides evidence that the Co ions in Zn$_{1-x}$Co$_{x}$O are in the divalent Co$^{2+}$ ($d^7$) states under the tetrahedral symmetry. Our finding indicates that the properly substituted Co ions for Zn sites will not produce the diluted ferromagnetic semiconductor property.
△ Less
Submitted 22 July, 2003;
originally announced July 2003.
