-
Casual Indoor HDR Radiance Capture from Omnidirectional Images
Authors:
Pulkit Gera,
Mohammad Reza Karimi Dastjerdi,
Charles Renaud,
P. J. Narayanan,
Jean-François Lalonde
Abstract:
We present PanoHDR-NeRF, a neural representation of the full HDR radiance field of an indoor scene, and a pipeline to capture it casually, without elaborate setups or complex capture protocols. First, a user captures a low dynamic range (LDR) omnidirectional video of the scene by freely waving an off-the-shelf camera around the scene. Then, an LDR2HDR network uplifts the captured LDR frames to HDR…
▽ More
We present PanoHDR-NeRF, a neural representation of the full HDR radiance field of an indoor scene, and a pipeline to capture it casually, without elaborate setups or complex capture protocols. First, a user captures a low dynamic range (LDR) omnidirectional video of the scene by freely waving an off-the-shelf camera around the scene. Then, an LDR2HDR network uplifts the captured LDR frames to HDR, which are used to train a tailored NeRF++ model. The resulting PanoHDR-NeRF can render full HDR images from any location of the scene. Through experiments on a novel test dataset of real scenes with the ground truth HDR radiance captured at locations not seen during training, we show that PanoHDR-NeRF predicts plausible HDR radiance from any scene point. We also show that the predicted radiance can synthesize correct lighting effects, enabling the augmentation of indoor scenes with synthetic objects that are lit correctly. Datasets and code are available at https://lvsn.github.io/PanoHDR-NeRF/.
△ Less
Submitted 19 October, 2022; v1 submitted 16 August, 2022;
originally announced August 2022.
-
Appearance Editing with Free-viewpoint Neural Rendering
Authors:
Pulkit Gera,
Aakash KT,
Dhawal Sirikonda,
Parikshit Sakurikar,
P. J. Narayanan
Abstract:
We present a neural rendering framework for simultaneous view synthesis and appearance editing of a scene from multi-view images captured under known environment illumination. Existing approaches either achieve view synthesis alone or view synthesis along with relighting, without direct control over the scene's appearance. Our approach explicitly disentangles the appearance and learns a lighting r…
▽ More
We present a neural rendering framework for simultaneous view synthesis and appearance editing of a scene from multi-view images captured under known environment illumination. Existing approaches either achieve view synthesis alone or view synthesis along with relighting, without direct control over the scene's appearance. Our approach explicitly disentangles the appearance and learns a lighting representation that is independent of it. Specifically, we independently estimate the BRDF and use it to learn a lighting-only representation of the scene. Such disentanglement allows our approach to generalize to arbitrary changes in appearance while performing view synthesis. We show results of editing the appearance of a real scene, demonstrating that our approach produces plausible appearance editing. The performance of our view synthesis approach is demonstrated to be at par with state-of-the-art approaches on both real and synthetic data.
△ Less
Submitted 14 October, 2021;
originally announced October 2021.
-
Swinging and tumbling of multicomponent vesicles in flow
Authors:
Prerna Gera,
David Salac,
Saverio E. Spagnolie
Abstract:
Biological membranes are host to proteins and molecules which may form domain-like structures resulting in spatially-varying material properties. Vesicles with such heterogeneous membranes can exhibit intricate shapes at equilibrium and rich dynamics when placed into a flow. Under the assumption of small deformations we develop a reduced order model to describe the fluid-structure interaction betw…
▽ More
Biological membranes are host to proteins and molecules which may form domain-like structures resulting in spatially-varying material properties. Vesicles with such heterogeneous membranes can exhibit intricate shapes at equilibrium and rich dynamics when placed into a flow. Under the assumption of small deformations we develop a reduced order model to describe the fluid-structure interaction between a viscous background shear flow and an inextensible membrane in two dimensions with spatially varying bending stiffness and spontaneous curvature. Material property variations of a critical magnitude, relative to the flow rate and internal/external viscosity contrast, can set off a qualitative change in the vesicle dynamics. A membrane of nearly constant bending stiffness or spontaneous curvature undergoes a small amplitude swinging motion (which includes tangential tank-treading), while for large enough material variations the dynamics pass through a regime featuring tumbling and periodic phase-lagging of the membrane material, and ultimately for very large material variation to a rigid body tumbling behavior. Distinct differences are found for even and odd spatial modes of domain distribution. Full numerical simulations are used to probe the theoretical predictions, which appear valid even when studying substantially deformed membranes.
△ Less
Submitted 4 February, 2022; v1 submitted 24 August, 2021;
originally announced August 2021.
-
A Closer Look at Mobile App Usage as a Persistent Biometric: A Small Case Study
Authors:
Md A. Noor,
G. Kaptan,
V. Cherukupally,
P. Gera,
T. Neal
Abstract:
In this paper, we explore mobile app use as a behavioral biometric identifier. While several efforts have also taken on this challenge, many have alluded to the inconsistency in human behavior, resulting in updating the biometric template frequently and periodically. Here, we represent app usage as simple images wherein each pixel value provides some information about the user's app usage. Then, w…
▽ More
In this paper, we explore mobile app use as a behavioral biometric identifier. While several efforts have also taken on this challenge, many have alluded to the inconsistency in human behavior, resulting in updating the biometric template frequently and periodically. Here, we represent app usage as simple images wherein each pixel value provides some information about the user's app usage. Then, we feed use these images to train a deep learning network (convolutional neural net) to classify the user's identity. Our contribution lies in the random order in which the images are fed to the classifier, thereby presenting novel evidence that there are some aspects of app usage that are indeed persistent. Our results yield a 96.8% $F$-score without any updates to the template data.
△ Less
Submitted 25 December, 2019;
originally announced December 2019.
-
Three-Dimensional Multicomponent Vesicles: Dynamics & Influence of Material Properties
Authors:
Prerna Gera,
David Salac
Abstract:
In this work, the nonlinear dynamics of a fully three-dimensional multicomponent vesicle in shear flow are explored. Using a volume- and area-conserving projection method coupled to a gradient-augmented level set and surface phase method, the dynamics are systematically studied as a function of the membrane bending rigidity difference between the components, the speed of diffusion compared to the…
▽ More
In this work, the nonlinear dynamics of a fully three-dimensional multicomponent vesicle in shear flow are explored. Using a volume- and area-conserving projection method coupled to a gradient-augmented level set and surface phase method, the dynamics are systematically studied as a function of the membrane bending rigidity difference between the components, the speed of diffusion compared to the underlying shear flow, and the strength of the phase domain energy compared to the bending energy. Using a pre-segregated vesicle, three dynamics are observed: stationary phase, phase-treading, and a new dynamic called vertical banding. These regimes are very sensitive to the strength of the domain line energy, as the vertical banding regime is not observed when line energy is larger than the bending energy. These findings demonstrate that a complete understanding of multicomponent vesicle dynamics require that the full three-dimensional system be modeled, and show the complexity obtained when considering heterogeneous material properties.
△ Less
Submitted 18 May, 2018;
originally announced May 2018.
-
Modeling of Multicomponent Three-Dimensional Vesicles
Authors:
Prerna Gera,
David Salac
Abstract:
In many interfacial flow systems, variations of surface properties lead to novel and interesting behaviors. In this work a three-dimensional model of flow dynamics for multicomponent vesicles is presented. The surface composition is modeled using a two-phase surface Cahn-Hilliard system, while the interface is captured using a level set jet scheme. The interface is coupled to the surrounding fluid…
▽ More
In many interfacial flow systems, variations of surface properties lead to novel and interesting behaviors. In this work a three-dimensional model of flow dynamics for multicomponent vesicles is presented. The surface composition is modeled using a two-phase surface Cahn-Hilliard system, while the interface is captured using a level set jet scheme. The interface is coupled to the surrounding fluid via a variation of energy approach. Sample energies considered include the total bending, variable surface tension energy, and phase segregation energy. The fully coupled system for surface inhomogeneities, and thus varying interface material properties is presented, as are the associated numerical methods. Numerical convergence and sample results demonstrate the validity of the model.
△ Less
Submitted 6 December, 2017;
originally announced December 2017.
-
Louvre: Lightweight Ordering Using Versioning for Release Consistency
Authors:
Pranith Kumar,
Prasun Gera,
Hyojong Kim,
Hyesoon Kim
Abstract:
Fence instructions are fundamental primitives that ensure consistency in a weakly consistent shared memory multi-core processor. The execution cost of these instructions is significant and adds a non-trivial overhead to parallel programs. In a naive architecture implementation, we track the ordering constraints imposed by a fence by its entry in the reorder buffer and its execution overhead entail…
▽ More
Fence instructions are fundamental primitives that ensure consistency in a weakly consistent shared memory multi-core processor. The execution cost of these instructions is significant and adds a non-trivial overhead to parallel programs. In a naive architecture implementation, we track the ordering constraints imposed by a fence by its entry in the reorder buffer and its execution overhead entails stalling the processor's pipeline until the store buffer is drained and also conservatively invalidating speculative loads. These actions create a cascading effect of increased overhead on the execution of the following instructions in the program. We find these actions to be overly restrictive and that they can be further relaxed thereby allowing aggressive optimizations.
The current work proposes a lightweight mechanism in which we assign ordering tags, called versions, to load and store instructions when they reside in the load/store queues and the write buffer. The version assigned to a memory access allows us to fully exploit the relaxation allowed by the weak consistency model and restricts its execution in such a way that the ordering constraints by the model are satisfied. We utilize the information captured through the assigned versions to reduce stalls caused by waiting for the store buffer to drain and to avoid unnecessary squashing of speculative loads, thereby minimizing the re-execution penalty. This method is particularly effective for the release consistency model that employs uni-directional fence instructions. We show that this mechanism reduces the ordering instruction latency by 39.6% and improves program performance by 11% on average over the baseline implementation.
△ Less
Submitted 5 December, 2017; v1 submitted 29 October, 2017;
originally announced October 2017.
-
Cahn-Hilliard on Surfaces: A Numerical Study
Authors:
Prerna Gera,
David Salac
Abstract:
The Cahn-Hilliard system has been used to describe a wide number of phase separation processes, from co-polymer systems to lipid membranes. In this work the convergence properties of a closest-point based scheme is investigated. In place of solving the original fourth-order system directly, two coupled second-order systems are solved. The system is solved using an incomplete Schur-decomposition as…
▽ More
The Cahn-Hilliard system has been used to describe a wide number of phase separation processes, from co-polymer systems to lipid membranes. In this work the convergence properties of a closest-point based scheme is investigated. In place of solving the original fourth-order system directly, two coupled second-order systems are solved. The system is solved using an incomplete Schur-decomposition as a preconditioner. The results indicate that with a sufficiently high-order time discretization the method only depends on the underlying spatial resolution.
△ Less
Submitted 27 February, 2017; v1 submitted 2 December, 2016;
originally announced December 2016.
-
Energy Variation of Soft Matter Interfaces
Authors:
Prerna Gera,
David Salac
Abstract:
The variation of energies associated with soft matter interfaces where surface inhomogeneities are present. These energies include the total bending and splay energy, the variable surface tension energy, a coupling energy between the total curvature and an underlying surface concentration field, the energy due to an external field, and a phase segregation energy. When considering these energies th…
▽ More
The variation of energies associated with soft matter interfaces where surface inhomogeneities are present. These energies include the total bending and splay energy, the variable surface tension energy, a coupling energy between the total curvature and an underlying surface concentration field, the energy due to an external field, and a phase segregation energy. When considering these energies the variation of material properties such a bending rigidity are taken into account, which results in more general variation expressions. These variations can be used to determine the equilibrium interface and concentration configuration or to determine the driving forces for non-equilibrium situations. While the focus of this work are energies associated with multicomponent vesicles, it can easily be extended to any soft matter interface.
△ Less
Submitted 28 November, 2016;
originally announced December 2016.
-
Inferring Fine-grained Control Flow Inside SGX Enclaves with Branch Shadowing
Authors:
Sangho Lee,
Ming-Wei Shih,
Prasun Gera,
Taesoo Kim,
Hyesoon Kim,
Marcus Peinado
Abstract:
In this paper, we explore a new, yet critical, side-channel attack against Intel Software Guard Extension (SGX), called a branch shadowing attack, which can reveal fine-grained control flows (i.e., each branch) of an enclave program running on real SGX hardware. The root cause of this attack is that Intel SGX does not clear the branch history when switching from enclave mode to non-enclave mode, l…
▽ More
In this paper, we explore a new, yet critical, side-channel attack against Intel Software Guard Extension (SGX), called a branch shadowing attack, which can reveal fine-grained control flows (i.e., each branch) of an enclave program running on real SGX hardware. The root cause of this attack is that Intel SGX does not clear the branch history when switching from enclave mode to non-enclave mode, leaving the fine-grained traces to the outside world through a branch-prediction side channel. However, exploiting the channel is not so straightforward in practice because 1) measuring branch prediction/misprediction penalties based on timing is too inaccurate to distinguish fine-grained control-flow changes and 2) it requires sophisticated control over the enclave execution to force its execution to the interesting code blocks. To overcome these challenges, we developed two novel exploitation techniques: 1) Intel PT- and LBR-based history-inferring techniques and 2) APIC-based technique to control the execution of enclave programs in a fine-grained manner. As a result, we could demonstrate our attack by breaking recent security constructs, including ORAM schemes, Sanctum, SGX-Shield, and T-SGX. Not limiting our work to the attack itself, we thoroughly studied the feasibility of hardware-based solutions (e.g., branch history clearing) and also proposed a software-based countermeasure, called Zigzagger, to mitigate the branch shadowing attack in practice.
△ Less
Submitted 1 June, 2017; v1 submitted 21 November, 2016;
originally announced November 2016.
-
Stochastic Phase Segregation on Surfaces
Authors:
Prerna Gera,
David Salac
Abstract:
Phase separation and coarsening is a phenomenon commonly seen in binary physical and chemical systems that occur in nature. Often times, thermal fluctuations, modeled as stochastic noise, are present in the system and the phase segregation process occurs on a surface. In this work, the segregation process is modeled via the Cahn-Hilliard-Cook model, which is a fourth-order parabolic stochastic sys…
▽ More
Phase separation and coarsening is a phenomenon commonly seen in binary physical and chemical systems that occur in nature. Often times, thermal fluctuations, modeled as stochastic noise, are present in the system and the phase segregation process occurs on a surface. In this work, the segregation process is modeled via the Cahn-Hilliard-Cook model, which is a fourth-order parabolic stochastic system. Coarsening is analyzed on two sample surfaces: a unit sphere and a dumbbell using a variety and a statistical analysis of the growth rate is performed. The influence of noise level and mobility is also investigated. It is also shown that a log-normal distribution fits the results well.
△ Less
Submitted 17 April, 2017; v1 submitted 23 May, 2016;
originally announced May 2016.
-
The Three-Dimensional Jump Conditions for the Stokes Equations with Discontinuous Viscosity, Singular Forces, and an Incompressible Interface
Authors:
Prerna Gera,
David Salac
Abstract:
The three-dimensional jump conditions for the pressure and velocity fields, up to the second normal derivative,across an incompressible/inextensible interface in the Stokes regime are derived herein. The fluid viscosity is only piecewise continuous in the domain while the embedded interface exerts singular forces on the surround fluids. This gives rise to discontinuous solutions in the pressure an…
▽ More
The three-dimensional jump conditions for the pressure and velocity fields, up to the second normal derivative,across an incompressible/inextensible interface in the Stokes regime are derived herein. The fluid viscosity is only piecewise continuous in the domain while the embedded interface exerts singular forces on the surround fluids. This gives rise to discontinuous solutions in the pressure and velocity field. These jump conditions are required to develop accurate numerical methods, such as the Immersed Interface Method, for the solutions of the Stokes equations in such situations.
△ Less
Submitted 4 September, 2013;
originally announced September 2013.
