-
fMPI: Fast Novel View Synthesis in the Wild with Layered Scene Representations
Authors:
Jonas Kohler,
Nicolas Griffiths Sanchez,
Luca Cavalli,
Catherine Herold,
Albert Pumarola,
Alberto Garcia Garcia,
Ali Thabet
Abstract:
In this study, we propose two novel input processing paradigms for novel view synthesis (NVS) methods based on layered scene representations that significantly improve their runtime without compromising quality. Our approach identifies and mitigates the two most time-consuming aspects of traditional pipelines: building and processing the so-called plane sweep volume (PSV), which is a high-dimensio…
▽ More
In this study, we propose two novel input processing paradigms for novel view synthesis (NVS) methods based on layered scene representations that significantly improve their runtime without compromising quality. Our approach identifies and mitigates the two most time-consuming aspects of traditional pipelines: building and processing the so-called plane sweep volume (PSV), which is a high-dimensional tensor of planar re-projections of the input camera views. In particular, we propose processing this tensor in parallel groups for improved compute efficiency as well as super-sampling adjacent input planes to generate denser, and hence more accurate scene representation. The proposed enhancements offer significant flexibility, allowing for a balance between performance and speed, thus making substantial steps toward real-time applications. Furthermore, they are very general in the sense that any PSV-based method can make use of them, including methods that employ multiplane images, multisphere images, and layered depth images. In a comprehensive set of experiments, we demonstrate that our proposed paradigms enable the design of an NVS method that achieves state-of-the-art on public benchmarks while being up to $50x$ faster than existing state-of-the-art methods. It also beats the current forerunner in terms of speed by over $3x$, while achieving significantly better rendering quality.
△ Less
Submitted 26 December, 2023;
originally announced December 2023.
-
Pushing the Limits of Quantum Computing for Simulating PFAS Chemistry
Authors:
Emil Dimitrov,
Goar Sanchez-Sanz,
James Nelson,
Lee O'Riordan,
Myles Doyle,
Sean Courtney,
Venkatesh Kannan,
Hassan Naseri,
Alberto Garcia Garcia,
James Tricker,
Marisa Faraggi,
Joshua Goings,
Luning Zhao
Abstract:
Accurate and scalable methods for computational quantum chemistry can accelerate research and development in many fields, ranging from drug discovery to advanced material design. Solving the electronic Schrodinger equation is the core problem of computational chemistry. However, the combinatorial complexity of this problem makes it intractable to find exact solutions, except for very small systems…
▽ More
Accurate and scalable methods for computational quantum chemistry can accelerate research and development in many fields, ranging from drug discovery to advanced material design. Solving the electronic Schrodinger equation is the core problem of computational chemistry. However, the combinatorial complexity of this problem makes it intractable to find exact solutions, except for very small systems. The idea of quantum computing originated from this computational challenge in simulating quantum-mechanics. We propose an end-to-end quantum chemistry pipeline based on the variational quantum eigensolver (VQE) algorithm and integrated with both HPC-based simulators and a trapped-ion quantum computer. Our platform orchestrates hundreds of simulation jobs on compute resources to efficiently complete a set of ab initio chemistry experiments with a wide range of parameterization. Per- and poly-fluoroalkyl substances (PFAS) are a large family of human-made chemicals that pose a major environmental and health issue globally. Our simulations includes breaking a Carbon-Fluorine bond in trifluoroacetic acid (TFA), a common PFAS chemical. This is a common pathway towards destruction and removal of PFAS. Molecules are modeled on both a quantum simulator and a trapped-ion quantum computer, specifically IonQ Aria. Using basic error mitigation techniques, the 11-qubit TFA model (56 entangling gates) on IonQ Aria yields near-quantitative results with milli-Hartree accuracy. Our novel results show the current state and future projections for quantum computing in solving the electronic structure problem, push the boundaries for the VQE algorithm and quantum computers, and facilitates development of quantum chemistry workflows.
△ Less
Submitted 2 November, 2023;
originally announced November 2023.
-
NeuralPassthrough: Learned Real-Time View Synthesis for VR
Authors:
Lei Xiao,
Salah Nouri,
Joel Hegland,
Alberto Garcia Garcia,
Douglas Lanman
Abstract:
Virtual reality (VR) headsets provide an immersive, stereoscopic visual experience, but at the cost of blocking users from directly observing their physical environment. Passthrough techniques are intended to address this limitation by leveraging outward-facing cameras to reconstruct the images that would otherwise be seen by the user without the headset. This is inherently a real-time view synthe…
▽ More
Virtual reality (VR) headsets provide an immersive, stereoscopic visual experience, but at the cost of blocking users from directly observing their physical environment. Passthrough techniques are intended to address this limitation by leveraging outward-facing cameras to reconstruct the images that would otherwise be seen by the user without the headset. This is inherently a real-time view synthesis challenge, since passthrough cameras cannot be physically co-located with the eyes. Existing passthrough techniques suffer from distracting reconstruction artifacts, largely due to the lack of accurate depth information (especially for near-field and disoccluded objects), and also exhibit limited image quality (e.g., being low resolution and monochromatic). In this paper, we propose the first learned passthrough method and assess its performance using a custom VR headset that contains a stereo pair of RGB cameras. Through both simulations and experiments, we demonstrate that our learned passthrough method delivers superior image quality compared to state-of-the-art methods, while meeting strict VR requirements for real-time, perspective-correct stereoscopic view synthesis over a wide field of view for desktop-connected headsets.
△ Less
Submitted 5 July, 2022;
originally announced July 2022.
-
Sampling formulas involving differences in shift-invariant subspaces: a unified approach
Authors:
Antonio G. García,
María J. Muñoz-Bouzo
Abstract:
Successive differences on a sequence of data help to discover some smoothness features of this data. This was one of the main reasons for rewriting the classical interpolation formula in terms of such data differences. The aim of this paper is to mimic them to a sequence of regular samples of a function in a shift-invariant subspace allowing its stable recovery. A suitable expression for the funct…
▽ More
Successive differences on a sequence of data help to discover some smoothness features of this data. This was one of the main reasons for rewriting the classical interpolation formula in terms of such data differences. The aim of this paper is to mimic them to a sequence of regular samples of a function in a shift-invariant subspace allowing its stable recovery. A suitable expression for the functions in the shift-invariant subspace by means of an isomorphism with the $L^2(0,1)$ space is the key to identify the simple pattern followed by the dual Riesz bases involved in the derived formulas. The paper contains examples illustrating different non-exhaustive situations including also the two-dimensional case.
△ Less
Submitted 12 September, 2017;
originally announced September 2017.
-
Filter Banks on Discrete Abelian Groups
Authors:
Antonio Garcia Garcia,
Miguel Angel Hernandez-Medina,
Gerardo Perez-Villalon
Abstract:
In this work we provide polyphase, modulation, and frame theoretical analyses of a filter bank on a discrete abelian group. Thus, multidimensional or cyclic filter banks as well as filter banks for signals in $\ell^2(\mathbb{Z}^d\times \mathbb{Z}_s)$ or $\ell^2(\mathbb{Z}_r \times \mathbb{Z}_s)$ spaces are studied in a unified way. We obtain perfect reconstruction conditions and the corresponding…
▽ More
In this work we provide polyphase, modulation, and frame theoretical analyses of a filter bank on a discrete abelian group. Thus, multidimensional or cyclic filter banks as well as filter banks for signals in $\ell^2(\mathbb{Z}^d\times \mathbb{Z}_s)$ or $\ell^2(\mathbb{Z}_r \times \mathbb{Z}_s)$ spaces are studied in a unified way. We obtain perfect reconstruction conditions and the corresponding frame bounds.
△ Less
Submitted 23 November, 2016; v1 submitted 10 March, 2016;
originally announced March 2016.
-
On the existence of compactly supported reconstruction functions in a sampling problem
Authors:
A. G. Garcia,
M. A. Hernandez-Medina,
G. Perez-Villalon
Abstract:
Assume that samples of a filtered version of a function in a shift-invariant space are avalaible. This work deals with the existence of a sampling formula involving these samples and having reconstruction functions with compact support. Thus, low computational complexity is involved and truncation errors are avoided. This is done in the light of the generalized sampling theory by using the overs…
▽ More
Assume that samples of a filtered version of a function in a shift-invariant space are avalaible. This work deals with the existence of a sampling formula involving these samples and having reconstruction functions with compact support. Thus, low computational complexity is involved and truncation errors are avoided. This is done in the light of the generalized sampling theory by using the oversampling technique: more samples than strictly necessary are used. For a suitable choice of the sampling period, a necessary and sufficient condition is given in terms of the Kronecker canonical form of a matrix pencil. Comparing with other characterizations in the mathematical literature, the given here has an important advantage: it can be reliable computed by using the GUPTRI form of the matrix pencil. Finally, a practical method for computing the compactly supported reconstruction functions is given for the important case where the oversampling rate is minimum.
△ Less
Submitted 12 June, 2008;
originally announced June 2008.
