-
Casual Indoor HDR Radiance Capture from Omnidirectional Images
Authors:
Pulkit Gera,
Mohammad Reza Karimi Dastjerdi,
Charles Renaud,
P. J. Narayanan,
Jean-François Lalonde
Abstract:
We present PanoHDR-NeRF, a neural representation of the full HDR radiance field of an indoor scene, and a pipeline to capture it casually, without elaborate setups or complex capture protocols. First, a user captures a low dynamic range (LDR) omnidirectional video of the scene by freely waving an off-the-shelf camera around the scene. Then, an LDR2HDR network uplifts the captured LDR frames to HDR…
▽ More
We present PanoHDR-NeRF, a neural representation of the full HDR radiance field of an indoor scene, and a pipeline to capture it casually, without elaborate setups or complex capture protocols. First, a user captures a low dynamic range (LDR) omnidirectional video of the scene by freely waving an off-the-shelf camera around the scene. Then, an LDR2HDR network uplifts the captured LDR frames to HDR, which are used to train a tailored NeRF++ model. The resulting PanoHDR-NeRF can render full HDR images from any location of the scene. Through experiments on a novel test dataset of real scenes with the ground truth HDR radiance captured at locations not seen during training, we show that PanoHDR-NeRF predicts plausible HDR radiance from any scene point. We also show that the predicted radiance can synthesize correct lighting effects, enabling the augmentation of indoor scenes with synthetic objects that are lit correctly. Datasets and code are available at https://lvsn.github.io/PanoHDR-NeRF/.
△ Less
Submitted 19 October, 2022; v1 submitted 16 August, 2022;
originally announced August 2022.
-
Appearance Editing with Free-viewpoint Neural Rendering
Authors:
Pulkit Gera,
Aakash KT,
Dhawal Sirikonda,
Parikshit Sakurikar,
P. J. Narayanan
Abstract:
We present a neural rendering framework for simultaneous view synthesis and appearance editing of a scene from multi-view images captured under known environment illumination. Existing approaches either achieve view synthesis alone or view synthesis along with relighting, without direct control over the scene's appearance. Our approach explicitly disentangles the appearance and learns a lighting r…
▽ More
We present a neural rendering framework for simultaneous view synthesis and appearance editing of a scene from multi-view images captured under known environment illumination. Existing approaches either achieve view synthesis alone or view synthesis along with relighting, without direct control over the scene's appearance. Our approach explicitly disentangles the appearance and learns a lighting representation that is independent of it. Specifically, we independently estimate the BRDF and use it to learn a lighting-only representation of the scene. Such disentanglement allows our approach to generalize to arbitrary changes in appearance while performing view synthesis. We show results of editing the appearance of a real scene, demonstrating that our approach produces plausible appearance editing. The performance of our view synthesis approach is demonstrated to be at par with state-of-the-art approaches on both real and synthetic data.
△ Less
Submitted 14 October, 2021;
originally announced October 2021.
-
A Closer Look at Mobile App Usage as a Persistent Biometric: A Small Case Study
Authors:
Md A. Noor,
G. Kaptan,
V. Cherukupally,
P. Gera,
T. Neal
Abstract:
In this paper, we explore mobile app use as a behavioral biometric identifier. While several efforts have also taken on this challenge, many have alluded to the inconsistency in human behavior, resulting in updating the biometric template frequently and periodically. Here, we represent app usage as simple images wherein each pixel value provides some information about the user's app usage. Then, w…
▽ More
In this paper, we explore mobile app use as a behavioral biometric identifier. While several efforts have also taken on this challenge, many have alluded to the inconsistency in human behavior, resulting in updating the biometric template frequently and periodically. Here, we represent app usage as simple images wherein each pixel value provides some information about the user's app usage. Then, we feed use these images to train a deep learning network (convolutional neural net) to classify the user's identity. Our contribution lies in the random order in which the images are fed to the classifier, thereby presenting novel evidence that there are some aspects of app usage that are indeed persistent. Our results yield a 96.8% $F$-score without any updates to the template data.
△ Less
Submitted 25 December, 2019;
originally announced December 2019.
-
Louvre: Lightweight Ordering Using Versioning for Release Consistency
Authors:
Pranith Kumar,
Prasun Gera,
Hyojong Kim,
Hyesoon Kim
Abstract:
Fence instructions are fundamental primitives that ensure consistency in a weakly consistent shared memory multi-core processor. The execution cost of these instructions is significant and adds a non-trivial overhead to parallel programs. In a naive architecture implementation, we track the ordering constraints imposed by a fence by its entry in the reorder buffer and its execution overhead entail…
▽ More
Fence instructions are fundamental primitives that ensure consistency in a weakly consistent shared memory multi-core processor. The execution cost of these instructions is significant and adds a non-trivial overhead to parallel programs. In a naive architecture implementation, we track the ordering constraints imposed by a fence by its entry in the reorder buffer and its execution overhead entails stalling the processor's pipeline until the store buffer is drained and also conservatively invalidating speculative loads. These actions create a cascading effect of increased overhead on the execution of the following instructions in the program. We find these actions to be overly restrictive and that they can be further relaxed thereby allowing aggressive optimizations.
The current work proposes a lightweight mechanism in which we assign ordering tags, called versions, to load and store instructions when they reside in the load/store queues and the write buffer. The version assigned to a memory access allows us to fully exploit the relaxation allowed by the weak consistency model and restricts its execution in such a way that the ordering constraints by the model are satisfied. We utilize the information captured through the assigned versions to reduce stalls caused by waiting for the store buffer to drain and to avoid unnecessary squashing of speculative loads, thereby minimizing the re-execution penalty. This method is particularly effective for the release consistency model that employs uni-directional fence instructions. We show that this mechanism reduces the ordering instruction latency by 39.6% and improves program performance by 11% on average over the baseline implementation.
△ Less
Submitted 5 December, 2017; v1 submitted 29 October, 2017;
originally announced October 2017.
-
Inferring Fine-grained Control Flow Inside SGX Enclaves with Branch Shadowing
Authors:
Sangho Lee,
Ming-Wei Shih,
Prasun Gera,
Taesoo Kim,
Hyesoon Kim,
Marcus Peinado
Abstract:
In this paper, we explore a new, yet critical, side-channel attack against Intel Software Guard Extension (SGX), called a branch shadowing attack, which can reveal fine-grained control flows (i.e., each branch) of an enclave program running on real SGX hardware. The root cause of this attack is that Intel SGX does not clear the branch history when switching from enclave mode to non-enclave mode, l…
▽ More
In this paper, we explore a new, yet critical, side-channel attack against Intel Software Guard Extension (SGX), called a branch shadowing attack, which can reveal fine-grained control flows (i.e., each branch) of an enclave program running on real SGX hardware. The root cause of this attack is that Intel SGX does not clear the branch history when switching from enclave mode to non-enclave mode, leaving the fine-grained traces to the outside world through a branch-prediction side channel. However, exploiting the channel is not so straightforward in practice because 1) measuring branch prediction/misprediction penalties based on timing is too inaccurate to distinguish fine-grained control-flow changes and 2) it requires sophisticated control over the enclave execution to force its execution to the interesting code blocks. To overcome these challenges, we developed two novel exploitation techniques: 1) Intel PT- and LBR-based history-inferring techniques and 2) APIC-based technique to control the execution of enclave programs in a fine-grained manner. As a result, we could demonstrate our attack by breaking recent security constructs, including ORAM schemes, Sanctum, SGX-Shield, and T-SGX. Not limiting our work to the attack itself, we thoroughly studied the feasibility of hardware-based solutions (e.g., branch history clearing) and also proposed a software-based countermeasure, called Zigzagger, to mitigate the branch shadowing attack in practice.
△ Less
Submitted 1 June, 2017; v1 submitted 21 November, 2016;
originally announced November 2016.
