-
SV3D: Novel Multi-view Synthesis and 3D Generation from a Single Image using Latent Video Diffusion
Authors:
Vikram Voleti,
Chun-Han Yao,
Mark Boss,
Adam Letts,
David Pankratz,
Dmitry Tochilkin,
Christian Laforte,
Robin Rombach,
Varun Jampani
Abstract:
We present Stable Video 3D (SV3D) -- a latent video diffusion model for high-resolution, image-to-multi-view generation of orbital videos around a 3D object. Recent work on 3D generation propose techniques to adapt 2D generative models for novel view synthesis (NVS) and 3D optimization. However, these methods have several disadvantages due to either limited views or inconsistent NVS, thereby affec…
▽ More
We present Stable Video 3D (SV3D) -- a latent video diffusion model for high-resolution, image-to-multi-view generation of orbital videos around a 3D object. Recent work on 3D generation propose techniques to adapt 2D generative models for novel view synthesis (NVS) and 3D optimization. However, these methods have several disadvantages due to either limited views or inconsistent NVS, thereby affecting the performance of 3D object generation. In this work, we propose SV3D that adapts image-to-video diffusion model for novel multi-view synthesis and 3D generation, thereby leveraging the generalization and multi-view consistency of the video models, while further adding explicit camera control for NVS. We also propose improved 3D optimization techniques to use SV3D and its NVS outputs for image-to-3D generation. Extensive experimental results on multiple datasets with 2D and 3D metrics as well as user study demonstrate SV3D's state-of-the-art performance on NVS as well as 3D reconstruction compared to prior works.
△ Less
Submitted 18 March, 2024;
originally announced March 2024.
-
TripoSR: Fast 3D Object Reconstruction from a Single Image
Authors:
Dmitry Tochilkin,
David Pankratz,
Zexiang Liu,
Zixuan Huang,
Adam Letts,
Yangguang Li,
Ding Liang,
Christian Laforte,
Varun Jampani,
Yan-Pei Cao
Abstract:
This technical report introduces TripoSR, a 3D reconstruction model leveraging transformer architecture for fast feed-forward 3D generation, producing 3D mesh from a single image in under 0.5 seconds. Building upon the LRM network architecture, TripoSR integrates substantial improvements in data processing, model design, and training techniques. Evaluations on public datasets show that TripoSR exh…
▽ More
This technical report introduces TripoSR, a 3D reconstruction model leveraging transformer architecture for fast feed-forward 3D generation, producing 3D mesh from a single image in under 0.5 seconds. Building upon the LRM network architecture, TripoSR integrates substantial improvements in data processing, model design, and training techniques. Evaluations on public datasets show that TripoSR exhibits superior performance, both quantitatively and qualitatively, compared to other open-source alternatives. Released under the MIT license, TripoSR is intended to empower researchers, developers, and creatives with the latest advancements in 3D generative AI.
△ Less
Submitted 4 March, 2024;
originally announced March 2024.
-
HD 63021: Chromospheric Activity and Mass Transfer in a Close Binary
Authors:
D. G. Whelan,
S. D. Chojnowski,
J. Labadie-Bartz,
J. Daglen,
K. Hudson,
G. M. Casey,
G. S. Stringfellow,
K. V. Lester,
J. Barry,
J. Heinerikson,
D. Pankratz,
M. Schreffler,
R. Maderak,
N. Lotspeich,
M. Vitale-Sullivan,
M. Woodard
Abstract:
Prompted by X-ray detections from multiple surveys, we investigated the A-type star HD 63021 and found that it is a double-lined spectroscopic binary with highly variable emission associated with the primary star. Analysis of our multi-epoch spectroscopic observations, the majority of which were carried out on small aperture telescopes, indicates a very short orbital period of just $2.9$ days, and…
▽ More
Prompted by X-ray detections from multiple surveys, we investigated the A-type star HD 63021 and found that it is a double-lined spectroscopic binary with highly variable emission associated with the primary star. Analysis of our multi-epoch spectroscopic observations, the majority of which were carried out on small aperture telescopes, indicates a very short orbital period of just $2.9$ days, and a mass ratio M$_2$/M$_1$ of $0.23$. The A1 V star is a slow rotator, with a rotational speed of $\sim34$ km/s. Assuming its mass is $2.3$ M$_{\odot}$, the present-day secondary is an evolved star of $\sim0.5$ M$_{\odot}$ that nearly fills its Roche lobe. This secondary star rotates comparatively rapidly at $\sim44$ km/s, and we see evidence that it is chromospherically active. Analysis of a photometric lightcurve from TESS reveals two strong periods, one at the orbital period for the system and another at half the orbital period. These findings suggest that HD 63021 is a close binary system undergoing mass transfer from the secondary star onto the primary star -- in all ways like an Algol eclipsing binary system, except without the eclipse. We discuss the system's mass transfer, which is not steady but seems to run in fits and bursts, and infer the system's basic physical properties from an orbital parameter study, the Roche lobe geometry, and its extant X-ray emission.
△ Less
Submitted 23 November, 2020;
originally announced November 2020.
