-
Face Reconstruction Transfer Attack as Out-of-Distribution Generalization
Authors:
Yoon Gyo Jung,
Jaewoo Park,
Xingbo Dong,
Ho** Park,
Andrew Beng ** Teoh,
Octavia Camps
Abstract:
Understanding the vulnerability of face recognition systems to malicious attacks is of critical importance. Previous works have focused on reconstructing face images that can penetrate a targeted verification system. Even in the white-box scenario, however, naively reconstructed images misrepresent the identity information, hence the attacks are easily neutralized once the face system is updated o…
▽ More
Understanding the vulnerability of face recognition systems to malicious attacks is of critical importance. Previous works have focused on reconstructing face images that can penetrate a targeted verification system. Even in the white-box scenario, however, naively reconstructed images misrepresent the identity information, hence the attacks are easily neutralized once the face system is updated or changed. In this paper, we aim to reconstruct face images which are capable of transferring face attacks on unseen encoders. We term this problem as Face Reconstruction Transfer Attack (FRTA) and show that it can be formulated as an out-of-distribution (OOD) generalization problem. Inspired by its OOD nature, we propose to solve FRTA by Averaged Latent Search and Unsupervised Validation with pseudo target (ALSUV). To strengthen the reconstruction attack on OOD unseen encoders, ALSUV reconstructs the face by searching the latent of amortized generator StyleGAN2 through multiple latent optimization, latent optimization trajectory averaging, and unsupervised validation with a pseudo target. We demonstrate the efficacy and generalization of our method on widely used face datasets, accompanying it with extensive ablation studies and visually, qualitatively, and quantitatively analyses. The source code will be released.
△ Less
Submitted 2 July, 2024;
originally announced July 2024.
-
Nearest Neighbor Guidance for Out-of-Distribution Detection
Authors:
Jaewoo Park,
Yoon Gyo Jung,
Andrew Beng ** Teoh
Abstract:
Detecting out-of-distribution (OOD) samples are crucial for machine learning models deployed in open-world environments. Classifier-based scores are a standard approach for OOD detection due to their fine-grained detection capability. However, these scores often suffer from overconfidence issues, misclassifying OOD samples distant from the in-distribution region. To address this challenge, we prop…
▽ More
Detecting out-of-distribution (OOD) samples are crucial for machine learning models deployed in open-world environments. Classifier-based scores are a standard approach for OOD detection due to their fine-grained detection capability. However, these scores often suffer from overconfidence issues, misclassifying OOD samples distant from the in-distribution region. To address this challenge, we propose a method called Nearest Neighbor Guidance (NNGuide) that guides the classifier-based score to respect the boundary geometry of the data manifold. NNGuide reduces the overconfidence of OOD samples while preserving the fine-grained capability of the classifier-based score. We conduct extensive experiments on ImageNet OOD detection benchmarks under diverse settings, including a scenario where the ID data undergoes natural distribution shift. Our results demonstrate that NNGuide provides a significant performance improvement on the base detection scores, achieving state-of-the-art results on both AUROC, FPR95, and AUPR metrics. The code is given at \url{https://github.com/roomo7time/nnguide}.
△ Less
Submitted 26 September, 2023;
originally announced September 2023.
-
Enhanced artificial intelligence-based diagnosis using CBCT with internal denoising: Clinical validation for discrimination of fungal ball, sinusitis, and normal cases in the maxillary sinus
Authors:
Kyungsu Kim,
Chae Yeon Lim,
Joong Bo Shin,
Myung ** Chung,
Yong Gi Jung
Abstract:
The cone-beam computed tomography (CBCT) provides 3D volumetric imaging of a target with low radiation dose and cost compared with conventional computed tomography, and it is widely used in the detection of paranasal sinus disease. However, it lacks the sensitivity to detect soft tissue lesions owing to reconstruction constraints. Consequently, only physicians with expertise in CBCT reading can di…
▽ More
The cone-beam computed tomography (CBCT) provides 3D volumetric imaging of a target with low radiation dose and cost compared with conventional computed tomography, and it is widely used in the detection of paranasal sinus disease. However, it lacks the sensitivity to detect soft tissue lesions owing to reconstruction constraints. Consequently, only physicians with expertise in CBCT reading can distinguish between inherent artifacts or noise and diseases, restricting the use of this imaging modality. The development of artificial intelligence (AI)-based computer-aided diagnosis methods for CBCT to overcome the shortage of experienced physicians has attracted substantial attention. However, advanced AI-based diagnosis addressing intrinsic noise in CBCT has not been devised, discouraging the practical use of AI solutions for CBCT. To address this issue, we propose an AI-based computer-aided diagnosis method using CBCT with a denoising module. This module is implemented before diagnosis to reconstruct the internal ground-truth full-dose scan corresponding to an input CBCT image and thereby improve the diagnostic performance. The external validation results for the unified diagnosis of sinus fungal ball, chronic rhinosinusitis, and normal cases show that the proposed method improves the micro-, macro-average AUC, and accuracy by 7.4, 5.6, and 9.6% (from 86.2, 87.0, and 73.4 to 93.6, 92.6, and 83.0%), respectively, compared with a baseline while improving human diagnosis accuracy by 11% (from 71.7 to 83.0%), demonstrating technical differentiation and clinical effectiveness. This pioneering study on AI-based diagnosis using CBCT indicates denoising can improve diagnostic performance and reader interpretability in images from the sinonasal area, thereby providing a new approach and direction to radiographic image reconstruction regarding the development of AI-based diagnostic solutions.
△ Less
Submitted 29 November, 2022;
originally announced November 2022.
-
Periocular Embedding Learning with Consistent Knowledge Distillation from Face
Authors:
Yoon Gyo Jung,
Jaewoo Park,
Cheng Yaw Low,
Jacky Chen Long Chai,
Leslie Ching Ow Tiong,
Andrew Beng ** Teoh
Abstract:
Periocular biometric, the peripheral area of the ocular, is a collaborative alternative to the face, especially when the face is occluded or masked. However, in practice, sole periocular biometric capture the least salient facial features, thereby lacking discriminative information, particularly in wild environments. To address these problems, we transfer discriminatory information from the face t…
▽ More
Periocular biometric, the peripheral area of the ocular, is a collaborative alternative to the face, especially when the face is occluded or masked. However, in practice, sole periocular biometric capture the least salient facial features, thereby lacking discriminative information, particularly in wild environments. To address these problems, we transfer discriminatory information from the face to support the training of a periocular network by using knowledge distillation. Specifically, we leverage face images for periocular embedding learning, but periocular alone is utilized for identity identification or verification. To enhance periocular embeddings by face effectively, we proposeConsistent Knowledge Distillation (CKD) that imposes consistency between face and periocular networks across prediction and feature layers. We find that imposing consistency at the prediction layer enables (1) extraction of global discriminative relationship information from face images and (2) effective transfer of the information from the face network to the periocular network. Particularly, consistency regularizes the prediction units to extract and store profound inter-class relationship information of face images. (3) The feature layer consistency, on the other hand, makes the periocular features robust against identity-irrelevant attributes. Overall, CKD empowers the sole periocular network to produce robust discriminative embeddings for periocular recognition in the wild. We theoretically and empirically validate the core principles of the distillation mechanism in CKD, discovering that CKD is equivalent to label smoothing with a novel sparsity-oriented regularizer that helps the network prediction to capture the global discriminative relationship. Extensive experiments reveal that CKD achieves state-of-the-art results on standard periocular recognition benchmark datasets.
△ Less
Submitted 28 January, 2024; v1 submitted 12 December, 2020;
originally announced December 2020.
-
Discriminative Multi-level Reconstruction under Compact Latent Space for One-Class Novelty Detection
Authors:
Jaewoo Park,
Yoon Gyo Jung,
Andrew Beng ** Teoh
Abstract:
In one-class novelty detection, a model learns solely on the in-class data to single out out-class instances. Autoencoder (AE) variants aim to compactly model the in-class data to reconstruct it exclusively, thus differentiating the in-class from out-class by the reconstruction error. However, compact modeling in an improper way might collapse the latent representations of the in-class data and th…
▽ More
In one-class novelty detection, a model learns solely on the in-class data to single out out-class instances. Autoencoder (AE) variants aim to compactly model the in-class data to reconstruct it exclusively, thus differentiating the in-class from out-class by the reconstruction error. However, compact modeling in an improper way might collapse the latent representations of the in-class data and thus their reconstruction, which would lead to performance deterioration. Moreover, to properly measure the reconstruction error of high-dimensional data, a metric is required that captures high-level semantics of the data. To this end, we propose Discriminative Compact AE (DCAE) that learns both compact and collapse-free latent representations of the in-class data, thereby reconstructing them both finely and exclusively. In DCAE, (a) we force a compact latent space to bijectively represent the in-class data by reconstructing them through internal discriminative layers of generative adversarial nets. (b) Based on the deep encoder's vulnerability to open set risk, out-class instances are encoded into the same compact latent space and reconstructed poorly without sacrificing the quality of in-class data reconstruction. (c) In inference, the reconstruction error is measured by a novel metric that computes the dissimilarity between a query and its reconstruction based on the class semantics captured by the internal discriminator. Extensive experiments on public image datasets validate the effectiveness of our proposed model on both novelty and adversarial example detection, delivering state-of-the-art performance.
△ Less
Submitted 17 February, 2021; v1 submitted 3 March, 2020;
originally announced March 2020.
-
The interfacial spin modulation of graphene on Fe(111)
Authors:
J. Hong,
H. -N. Hwang,
A. T. NDiaye,
J. Liang,
G. Chen,
Y. Park,
L. T. Singh,
Y. G. Jung,
J. -H. Yang,
J. -I. Jeong,
A. K. Schmid,
E. Arenholz,
H. Yang,
J. Bokor,
C. -C. Hwang,
L. You
Abstract:
When Fe, which is a typical ferromagnet using d- or f-orbital states, is combined with 2D materials such as graphene, it offers many opportunities for spintronics. The origin of 2D magnetism is from magnetic insulating behaviors, which could result in magnetic excitations and also proximity effects. However, the phenomena were only observed at extremely low temperatures. Fe and graphene interfaces…
▽ More
When Fe, which is a typical ferromagnet using d- or f-orbital states, is combined with 2D materials such as graphene, it offers many opportunities for spintronics. The origin of 2D magnetism is from magnetic insulating behaviors, which could result in magnetic excitations and also proximity effects. However, the phenomena were only observed at extremely low temperatures. Fe and graphene interfaces could control spin structures in which they show a unique atomic spin modulation and magnetic coupling through the interface. Another reason for covering graphene on Fe is to prevent oxidation under ambient conditions. We investigated the engineering of spin configurations by growing monolayer graphene on an Fe(111) single crystal surface and observed the presence of sharply branched, 3D tree-like domain structures. Magnetization by a swee** magnetic field (m-H) revealed that the interface showed canted magnetization in the in-plane (IP) orientation. Moreover, graphene could completely prevent the oxidation of the Fe surface. The results indicate possible control of the spin structures at the atomic scale and the interface phenomena in the 2D structure. The study introduces a new approach for room temperature 2D magnetism.
△ Less
Submitted 24 November, 2018;
originally announced November 2018.
