-
Inverse Rendering of Translucent Objects using Physical and Neural Renderers
Authors:
Chenhao Li,
Trung Thanh Ngo,
Hajime Nagahara
Abstract:
In this work, we propose an inverse rendering model that estimates 3D shape, spatially-varying reflectance, homogeneous subsurface scattering parameters, and an environment illumination jointly from only a pair of captured images of a translucent object. In order to solve the ambiguity problem of inverse rendering, we use a physically-based renderer and a neural renderer for scene reconstruction a…
▽ More
In this work, we propose an inverse rendering model that estimates 3D shape, spatially-varying reflectance, homogeneous subsurface scattering parameters, and an environment illumination jointly from only a pair of captured images of a translucent object. In order to solve the ambiguity problem of inverse rendering, we use a physically-based renderer and a neural renderer for scene reconstruction and material editing. Because two renderers are differentiable, we can compute a reconstruction loss to assist parameter estimation. To enhance the supervision of the proposed neural renderer, we also propose an augmented loss. In addition, we use a flash and no-flash image pair as the input. To supervise the training, we constructed a large-scale synthetic dataset of translucent objects, which consists of 117K scenes. Qualitative and quantitative results on both synthetic and real-world datasets demonstrated the effectiveness of the proposed model.
△ Less
Submitted 15 May, 2023;
originally announced May 2023.
-
Human-Imperceptible Identification with Learnable Lensless Imaging
Authors:
Thuong Nguyen Canh,
Trung Thanh Ngo,
Hajime Nagahara
Abstract:
Lensless imaging protects visual privacy by capturing heavily blurred images that are imperceptible for humans to recognize the subject but contain enough information for machines to infer information. Unfortunately, protecting visual privacy comes with a reduction in recognition accuracy and vice versa. We propose a learnable lensless imaging framework that protects visual privacy while maintaini…
▽ More
Lensless imaging protects visual privacy by capturing heavily blurred images that are imperceptible for humans to recognize the subject but contain enough information for machines to infer information. Unfortunately, protecting visual privacy comes with a reduction in recognition accuracy and vice versa. We propose a learnable lensless imaging framework that protects visual privacy while maintaining recognition accuracy. To make captured images imperceptible to humans, we designed several loss functions based on total variation, invertibility, and the restricted isometry property. We studied the effect of privacy protection with blurriness on the identification of personal identity via a quantitative method based on a subjective evaluation. Moreover, we validate our simulation by implementing a hardware realization of lensless imaging with photo-lithographically printed masks.
△ Less
Submitted 4 February, 2023;
originally announced February 2023.
-
Recurrent neural network transducer for Japanese and Chinese offline handwritten text recognition
Authors:
Trung Tan Ngo,
Hung Tuan Nguyen,
Nam Tuan Ly,
Masaki Nakagawa
Abstract:
In this paper, we propose an RNN-Transducer model for recognizing Japanese and Chinese offline handwritten text line images. As far as we know, it is the first approach that adopts the RNN-Transducer model for offline handwritten text recognition. The proposed model consists of three main components: a visual feature encoder that extracts visual features from an input image by CNN and then encodes…
▽ More
In this paper, we propose an RNN-Transducer model for recognizing Japanese and Chinese offline handwritten text line images. As far as we know, it is the first approach that adopts the RNN-Transducer model for offline handwritten text recognition. The proposed model consists of three main components: a visual feature encoder that extracts visual features from an input image by CNN and then encodes the visual features by BLSTM; a linguistic context encoder that extracts and encodes linguistic features from the input image by embedded layers and LSTM; and a joint decoder that combines and then decodes the visual features and the linguistic features into the final label sequence by fully connected and softmax layers. The proposed model takes advantage of both visual and linguistic information from the input image. In the experiments, we evaluated the performance of the proposed model on the two datasets: Kuzushiji and SCUT-EPT. Experimental results show that the proposed model achieves state-of-the-art performance on all datasets.
△ Less
Submitted 28 June, 2021;
originally announced June 2021.
-
Observation of thermal events on the plasma facing components of Wendelstein 7-X
Authors:
A. Puig Sitjes,
Y. Gao,
M. Jakubowski,
P. Drewelow,
H. Niemann,
A. Ali,
V. Moncada,
F. Pisano,
T. T. Ngo,
B. Cannas,
M. Sleczka,
W7-X Team
Abstract:
Long pulse operation of present and future magnetic fusion devices requires sophisticated methods for protection of plasma facing components from overheating. Typically, thermographic systems are being used to fulfill this task. Steady state operation requires, however, autonomous operation of the system and fully automatic detection of abnormal events. At Wendelstein 7-X (W7-X), a large advanced…
▽ More
Long pulse operation of present and future magnetic fusion devices requires sophisticated methods for protection of plasma facing components from overheating. Typically, thermographic systems are being used to fulfill this task. Steady state operation requires, however, autonomous operation of the system and fully automatic detection of abnormal events. At Wendelstein 7-X (W7-X), a large advanced stellarator, which aims at demonstrating the capabilities of the stellarator line as a future fusion power plant, significant efforts are being undertaken to develop a fully automatic system based on thermographic diagnostics. In October 2018, the first divertor-based experimental campaign has been finished. One of the goals of this operation phase (named OP1.2) was to study the capabilities of the island divertor concept using an uncooled test divertor made of fine-grain graphite tiles. Throughout this campaign, it was possible to test the infrared imaging diagnostic system, which will be used to protect the actively water-cooled plasma facing components (PFCs) during the steady-state operation in the next experimental campaign. An overview of the most relevant thermal events on the PFCs that were detected in OP1.2 using this system are presented. This includes events that limited operation during the campaign, like baffe hot spots and divertor overloads, events that are potentially critical in steady state operation like leading edges, events caused by the ECRH and NBI heating systems and other events which are a common source of false alarms like surface layers. The detected thermal events are now part of an important and extensive image database which will be used to further automate the system by means of computer vision and machine learning techniques in preparation for steady-state operation, when the system must be able to detect dangerous events and protect the machine in real-time.
△ Less
Submitted 9 October, 2019;
originally announced October 2019.