-
SuperPrimitive: Scene Reconstruction at a Primitive Level
Authors:
Kirill Mazur,
Gwangbin Bae,
Andrew J. Davison
Abstract:
Joint camera pose and dense geometry estimation from a set of images or a monocular video remains a challenging problem due to its computational complexity and inherent visual ambiguities. Most dense incremental reconstruction systems operate directly on image pixels and solve for their 3D positions using multi-view geometry cues. Such pixel-level approaches suffer from ambiguities or violations o…
▽ More
Joint camera pose and dense geometry estimation from a set of images or a monocular video remains a challenging problem due to its computational complexity and inherent visual ambiguities. Most dense incremental reconstruction systems operate directly on image pixels and solve for their 3D positions using multi-view geometry cues. Such pixel-level approaches suffer from ambiguities or violations of multi-view consistency (e.g. caused by textureless or specular surfaces).
We address this issue with a new image representation which we call a SuperPrimitive. SuperPrimitives are obtained by splitting images into semantically correlated local regions and enhancing them with estimated surface normal directions, both of which are predicted by state-of-the-art single image neural networks. This provides a local geometry estimate per SuperPrimitive, while their relative positions are adjusted based on multi-view observations.
We demonstrate the versatility of our new representation by addressing three 3D reconstruction tasks: depth completion, few-view structure from motion, and monocular dense visual odometry.
△ Less
Submitted 17 April, 2024; v1 submitted 10 December, 2023;
originally announced December 2023.
-
Feature-Realistic Neural Fusion for Real-Time, Open Set Scene Understanding
Authors:
Kirill Mazur,
Edgar Sucar,
Andrew J. Davison
Abstract:
General scene understanding for robotics requires flexible semantic representation, so that novel objects and structures which may not have been known at training time can be identified, segmented and grouped. We present an algorithm which fuses general learned features from a standard pre-trained network into a highly efficient 3D geometric neural field representation during real-time SLAM. The f…
▽ More
General scene understanding for robotics requires flexible semantic representation, so that novel objects and structures which may not have been known at training time can be identified, segmented and grouped. We present an algorithm which fuses general learned features from a standard pre-trained network into a highly efficient 3D geometric neural field representation during real-time SLAM. The fused 3D feature maps inherit the coherence of the neural field's geometry representation. This means that tiny amounts of human labelling interacting at runtime enable objects or even parts of objects to be robustly and accurately segmented in an open set manner.
△ Less
Submitted 6 October, 2022;
originally announced October 2022.
-
Point-Based Modeling of Human Clothing
Authors:
Ilya Zakharkin,
Kirill Mazur,
Artur Grigorev,
Victor Lempitsky
Abstract:
We propose a new approach to human clothing modeling based on point clouds. Within this approach, we learn a deep model that can predict point clouds of various outfits, for various human poses, and for various human body shapes. Notably, outfits of various types and topologies can be handled by the same model. Using the learned model, we can infer the geometry of new outfits from as little as a s…
▽ More
We propose a new approach to human clothing modeling based on point clouds. Within this approach, we learn a deep model that can predict point clouds of various outfits, for various human poses, and for various human body shapes. Notably, outfits of various types and topologies can be handled by the same model. Using the learned model, we can infer the geometry of new outfits from as little as a single image, and perform outfit retargeting to new bodies in new poses. We complement our geometric model with appearance modeling that uses the point cloud geometry as a geometric scaffolding and employs neural point-based graphics to capture outfit appearance from videos and to re-render the captured outfits. We validate both geometric modeling and appearance modeling aspects of the proposed approach against recently proposed methods and establish the viability of point-based clothing modeling.
△ Less
Submitted 6 October, 2021; v1 submitted 16 April, 2021;
originally announced April 2021.
-
Cloud Transformers: A Universal Approach To Point Cloud Processing Tasks
Authors:
Kirill Mazur,
Victor Lempitsky
Abstract:
We present a new versatile building block for deep point cloud processing architectures that is equally suited for diverse tasks. This building block combines the ideas of spatial transformers and multi-view convolutional networks with the efficiency of standard convolutional layers in two and three-dimensional dense grids. The new block operates via multiple parallel heads, whereas each head diff…
▽ More
We present a new versatile building block for deep point cloud processing architectures that is equally suited for diverse tasks. This building block combines the ideas of spatial transformers and multi-view convolutional networks with the efficiency of standard convolutional layers in two and three-dimensional dense grids. The new block operates via multiple parallel heads, whereas each head differentiably rasterizes feature representations of individual points into a low-dimensional space, and then uses dense convolution to propagate information across points. The results of the processing of individual heads are then combined together resulting in the update of point features. Using the new block, we build architectures for both discriminative (point cloud segmentation, point cloud classification) and generative (point cloud inpainting and image-based point cloud reconstruction) tasks. The resulting architectures achieve state-of-the-art performance for these tasks, demonstrating the versatility of the new block for point cloud processing.
△ Less
Submitted 3 October, 2021; v1 submitted 22 July, 2020;
originally announced July 2020.
-
A Partial Solution to Continuous Blotto
Authors:
Kostyantyn Mazur
Abstract:
This paper analyzes the structure of mixed-strategy equilibria for Colonel Blotto games, where the outcome on each battlefield is a polynomial function of the difference between the two players' allocations. This paper severely reduces the set of strategies that needs to be searched to find a Nash equilibrium. It finds that there exists a Nash equilibrium where both players' mixed strategies are d…
▽ More
This paper analyzes the structure of mixed-strategy equilibria for Colonel Blotto games, where the outcome on each battlefield is a polynomial function of the difference between the two players' allocations. This paper severely reduces the set of strategies that needs to be searched to find a Nash equilibrium. It finds that there exists a Nash equilibrium where both players' mixed strategies are discrete distributions, and it places an upper bound on the number of points in the supports of these discrete distributions.
△ Less
Submitted 14 September, 2017; v1 submitted 7 June, 2017;
originally announced June 2017.
-
Approval Voting in Product Societies
Authors:
Kristen Mazur,
Mutiara Sondjaja,
Matthew Wright,
Carolyn Yarnall
Abstract:
In approval voting, individuals vote for all platforms that they find acceptable. In this situation it is natural to ask: When is agreement possible? What conditions guarantee that some fraction of the voters agree on even a single platform? Berg et. al. found such conditions when voters are asked to make a decision on a single issue that can be represented on a linear spectrum. In particular, the…
▽ More
In approval voting, individuals vote for all platforms that they find acceptable. In this situation it is natural to ask: When is agreement possible? What conditions guarantee that some fraction of the voters agree on even a single platform? Berg et. al. found such conditions when voters are asked to make a decision on a single issue that can be represented on a linear spectrum. In particular, they showed that if two out of every three voters agree on a platform, there is a platform that is acceptable to a majority of the voters. Hardin developed an analogous result when the issue can be represented on a circular spectrum. We examine scenarios in which voters must make two decisions simultaneously. For example, if voters must decide on the day of the week to hold a meeting and the length of the meeting, then the space of possible options forms a cylindrical spectrum. Previous results do not apply to these multi-dimensional voting societies because a voter's preference on one issue often impacts their preference on another. We present a general lower bound on agreement in a two-dimensional voting society, and then examine specific results for societies whose spectra are cylinders and tori.
△ Less
Submitted 28 March, 2017;
originally announced March 2017.
-
On Data Flow Management: the Multilevel Analysis of Data Center Total Cost
Authors:
Katarzyna Mazur,
Bogdan Ksiezopolski
Abstract:
Information management is one of the most significant issues in nowadays data centers. Selection of appropriate software, security mechanisms and effective energy consumption management together with caring for the environment enforces a profound analysis of the considered system. Besides these factors, financial analysis of data center maintenance is another important aspect that needs to be cons…
▽ More
Information management is one of the most significant issues in nowadays data centers. Selection of appropriate software, security mechanisms and effective energy consumption management together with caring for the environment enforces a profound analysis of the considered system. Besides these factors, financial analysis of data center maintenance is another important aspect that needs to be considered. Data centers are mission-critical components of all large enterprises and frequently cost hundreds of millions of dollars to build, yet few high-level executives understand the true cost of operating such facilities. Costs are typically spread across the IT, networking, and facilities, which makes management of these costs and assessment of alternatives difficult. This paper deals with a research on multilevel analysis of data center management and presents an approach to estimate the true total costs of operating data center physical facilities, taking into account the proper management of the information flow.
△ Less
Submitted 27 March, 2017;
originally announced March 2017.
