-
Learning elementary structures for 3D shape generation and matching
Authors:
Theo Deprelle,
Thibault Groueix,
Matthew Fisher,
Vladimir G. Kim,
Bryan C. Russell,
Mathieu Aubry
Abstract:
We propose to represent shapes as the deformation and combination of learnable elementary 3D structures, which are primitives resulting from training over a collection of shape. We demonstrate that the learned elementary 3D structures lead to clear improvements in 3D shape generation and matching. More precisely, we present two complementary approaches for learning elementary structures: (i) patch…
▽ More
We propose to represent shapes as the deformation and combination of learnable elementary 3D structures, which are primitives resulting from training over a collection of shape. We demonstrate that the learned elementary 3D structures lead to clear improvements in 3D shape generation and matching. More precisely, we present two complementary approaches for learning elementary structures: (i) patch deformation learning and (ii) point translation learning. Both approaches can be extended to abstract structures of higher dimensions for improved results. We evaluate our method on two tasks: reconstructing ShapeNet objects and estimating dense correspondences between human scans (FAUST inter challenge). We show 16% improvement over surface deformation approaches for shape reconstruction and outperform FAUST inter challenge state of the art by 6%.
△ Less
Submitted 14 August, 2019; v1 submitted 13 August, 2019;
originally announced August 2019.
-
Unsupervised cycle-consistent deformation for shape matching
Authors:
Thibault Groueix,
Matthew Fisher,
Vladimir G. Kim,
Bryan C. Russell,
Mathieu Aubry
Abstract:
We propose a self-supervised approach to deep surface deformation. Given a pair of shapes, our algorithm directly predicts a parametric transformation from one shape to the other respecting correspondences. Our insight is to use cycle-consistency to define a notion of good correspondences in groups of objects and use it as a supervisory signal to train our network. Our method does not rely on a te…
▽ More
We propose a self-supervised approach to deep surface deformation. Given a pair of shapes, our algorithm directly predicts a parametric transformation from one shape to the other respecting correspondences. Our insight is to use cycle-consistency to define a notion of good correspondences in groups of objects and use it as a supervisory signal to train our network. Our method does not rely on a template, assume near isometric deformations or rely on point-correspondence supervision. We demonstrate the efficacy of our approach by using it to transfer segmentation across shapes. We show, on Shapenet, that our approach is competitive with comparable state-of-the-art methods when annotated training data is readily available, but outperforms them by a large margin in the few-shot segmentation scenario.
△ Less
Submitted 6 July, 2019;
originally announced July 2019.
-
Photometric Mesh Optimization for Video-Aligned 3D Object Reconstruction
Authors:
Chen-Hsuan Lin,
Oliver Wang,
Bryan C. Russell,
Eli Shechtman,
Vladimir G. Kim,
Matthew Fisher,
Simon Lucey
Abstract:
In this paper, we address the problem of 3D object mesh reconstruction from RGB videos. Our approach combines the best of multi-view geometric and data-driven methods for 3D reconstruction by optimizing object meshes for multi-view photometric consistency while constraining mesh deformations with a shape prior. We pose this as a piecewise image alignment problem for each mesh face projection. Our…
▽ More
In this paper, we address the problem of 3D object mesh reconstruction from RGB videos. Our approach combines the best of multi-view geometric and data-driven methods for 3D reconstruction by optimizing object meshes for multi-view photometric consistency while constraining mesh deformations with a shape prior. We pose this as a piecewise image alignment problem for each mesh face projection. Our approach allows us to update shape parameters from the photometric error without any depth or mask information. Moreover, we show how to avoid a degeneracy of zero photometric gradients via rasterizing from a virtual viewpoint. We demonstrate 3D object mesh reconstruction results from both synthetic and real-world videos with our photometric mesh optimization, which is unachievable with either naïve mesh generation networks or traditional pipelines of surface reconstruction without heavy manual post-processing.
△ Less
Submitted 20 March, 2019;
originally announced March 2019.
-
3D-CODED : 3D Correspondences by Deep Deformation
Authors:
Thibault Groueix,
Matthew Fisher,
Vladimir G. Kim,
Bryan C. Russell,
Mathieu Aubry
Abstract:
We present a new deep learning approach for matching deformable shapes by introducing {\it Shape Deformation Networks} which jointly encode 3D shapes and correspondences. This is achieved by factoring the surface representation into (i) a template, that parameterizes the surface, and (ii) a learnt global feature vector that parameterizes the transformation of the template into the input surface. B…
▽ More
We present a new deep learning approach for matching deformable shapes by introducing {\it Shape Deformation Networks} which jointly encode 3D shapes and correspondences. This is achieved by factoring the surface representation into (i) a template, that parameterizes the surface, and (ii) a learnt global feature vector that parameterizes the transformation of the template into the input surface. By predicting this feature for a new shape, we implicitly predict correspondences between this shape and the template. We show that these correspondences can be improved by an additional step which improves the shape feature by minimizing the Chamfer distance between the input and transformed template. We demonstrate that our simple approach improves on state-of-the-art results on the difficult FAUST-inter challenge, with an average correspondence error of 2.88cm. We show, on the TOSCA dataset, that our method is robust to many types of perturbations, and generalizes to non-human shapes. This robustness allows it to perform well on real unclean, meshes from the the SCAPE dataset.
△ Less
Submitted 27 July, 2018; v1 submitted 13 June, 2018;
originally announced June 2018.
-
Quantitative Homogenization with Relatively Soft Inclusions and Interior Estimates
Authors:
B. Chase Russell
Abstract:
We establish large-scale interior Lipschitz estimates for solutions to systems of linear elasticity with rapidly oscillating periodic coefficients and Dirichlet boundary conditions in domains with periodically placed inclusions of size $\mathcal{O}(\varepsilon)$ and magnitude $δ$ by establishing $H^1$-convergence rates for such solutions. The interior estimates at the macroscopic scale are derived…
▽ More
We establish large-scale interior Lipschitz estimates for solutions to systems of linear elasticity with rapidly oscillating periodic coefficients and Dirichlet boundary conditions in domains with periodically placed inclusions of size $\mathcal{O}(\varepsilon)$ and magnitude $δ$ by establishing $H^1$-convergence rates for such solutions. The interior estimates at the macroscopic scale are derived directly without the use of compactness via a Campanato-type scheme presented by S. Armstrong and C.K. Smart and that was adapted for uniformly elliptic equations in by Armstrong and Z. Shen.
△ Less
Submitted 10 May, 2018;
originally announced May 2018.
-
AtlasNet: A Papier-Mâché Approach to Learning 3D Surface Generation
Authors:
Thibault Groueix,
Matthew Fisher,
Vladimir G. Kim,
Bryan C. Russell,
Mathieu Aubry
Abstract:
We introduce a method for learning to generate the surface of 3D shapes. Our approach represents a 3D shape as a collection of parametric surface elements and, in contrast to methods generating voxel grids or point clouds, naturally infers a surface representation of the shape. Beyond its novelty, our new shape generation framework, AtlasNet, comes with significant advantages, such as improved pre…
▽ More
We introduce a method for learning to generate the surface of 3D shapes. Our approach represents a 3D shape as a collection of parametric surface elements and, in contrast to methods generating voxel grids or point clouds, naturally infers a surface representation of the shape. Beyond its novelty, our new shape generation framework, AtlasNet, comes with significant advantages, such as improved precision and generalization capabilities, and the possibility to generate a shape of arbitrary resolution without memory issues. We demonstrate these benefits and compare to strong baselines on the ShapeNet benchmark for two applications: (i) auto-encoding shapes, and (ii) single-view reconstruction from a still image. We also provide results showing its potential for other applications, such as morphing, parametrization, super-resolution, matching, and co-segmentation.
△ Less
Submitted 20 July, 2018; v1 submitted 14 February, 2018;
originally announced February 2018.
-
Homogenization in Perforated Domains and Interior Lipschitz Estimates
Authors:
B. Chase Russell
Abstract:
We establish interior Lipschitz estimates at the macroscopic scale for solutions to systems of linear elasticity with rapidly oscillating periodic coefficients and mixed boundary conditions in domains periodically perforated at a microscopic scale $\varepsilon$ by establishing $H^1$-convergence rates for such solutions. The interior estimates are derived directly without the use of compactness via…
▽ More
We establish interior Lipschitz estimates at the macroscopic scale for solutions to systems of linear elasticity with rapidly oscillating periodic coefficients and mixed boundary conditions in domains periodically perforated at a microscopic scale $\varepsilon$ by establishing $H^1$-convergence rates for such solutions. The interior estimates are derived directly without the use of compactness via an argument presented in [3] that was adapted for elliptic equations in [2] and [11]. As a consequence, we derive a Liouville type estimate for solutions to the systems of linear elasticity in unbounded periodically perforated domains.
△ Less
Submitted 11 April, 2017;
originally announced April 2017.
-
Transition from reconstruction towards thin film on the (110) surface of strontium titanate
Authors:
Z. Wang,
A. Loon,
A. Subramanian,
S. Gerhold,
E. McDermott,
J. A. Enterkin,
M. Hieckel,
B. C. Russell,
R. J. Green,
A. Moewes,
J. Guo,
P. Blaha,
M. R. Castell,
U. Diebold,
L. D. Marks
Abstract:
The surfaces of metal oxides often are reconstructed with a geometry and composition that is considerably different from a simple termination of the bulk. Such structures can also be viewed as ultrathin films, epitaxed on a substrate. Here, the reconstructions of the SrTiO3 (110) surface are studied combining scanning tunneling microscopy, transmission electron diffraction, and X-ray absorption sp…
▽ More
The surfaces of metal oxides often are reconstructed with a geometry and composition that is considerably different from a simple termination of the bulk. Such structures can also be viewed as ultrathin films, epitaxed on a substrate. Here, the reconstructions of the SrTiO3 (110) surface are studied combining scanning tunneling microscopy, transmission electron diffraction, and X-ray absorption spectroscopy, and analyzed with density functional theory calculations. While SrTiO3 (110) invariably terminates with an overlayer of titania, with increasing density its structure switches from nx1 and 2xn. At the same time the coordination of the Ti atoms changes from a network of corner-sharing tetrahedra to a double layer of edge-shared octahedra with bridging units of octahedrally coordinated strontium. This transition from the nx1 to 2xn reconstructions is a transition from a pseudomorphically stabilized tetrahedral network towards an octahedral titania thin film with stress-relief from octahedral strontia units at the surface.
△ Less
Submitted 10 February, 2016;
originally announced February 2016.
