-
Order-One Rolling Shutter Cameras
Authors:
Marvin Anas Hahn,
Kathlén Kohn,
Orlando Marigliano,
Tomas Pajdla
Abstract:
Rolling shutter (RS) cameras dominate consumer and smartphone markets. Several methods for computing the absolute pose of RS cameras have appeared in the last 20 years, but the relative pose problem has not been fully solved yet. We provide a unified theory for the important class of order-one rolling shutter (RS$_1$) cameras. These cameras generalize the perspective projection to RS cameras, proj…
▽ More
Rolling shutter (RS) cameras dominate consumer and smartphone markets. Several methods for computing the absolute pose of RS cameras have appeared in the last 20 years, but the relative pose problem has not been fully solved yet. We provide a unified theory for the important class of order-one rolling shutter (RS$_1$) cameras. These cameras generalize the perspective projection to RS cameras, projecting a generic space point to exactly one image point via a rational map. We introduce a new back-projection RS camera model, characterize RS$_1$ cameras, construct explicit parameterizations of such cameras, and determine the image of a space line. We classify all minimal problems for solving the relative camera pose problem with linear RS$_1$ cameras and discover new practical cases. Finally, we show how the theory can be used to explain RS models previously used for absolute pose computation.
△ Less
Submitted 17 March, 2024;
originally announced March 2024.
-
Geometry of Linear Neural Networks: Equivariance and Invariance under Permutation Groups
Authors:
Kathlén Kohn,
Anna-Laura Sattelberger,
Vahid Shahverdi
Abstract:
The set of functions parameterized by a linear fully-connected neural network is a determinantal variety. We investigate the subvariety of functions that are equivariant or invariant under the action of a permutation group. Examples of such group actions are translations or $90^\circ$ rotations on images. We describe such equivariant or invariant subvarieties as direct products of determinantal va…
▽ More
The set of functions parameterized by a linear fully-connected neural network is a determinantal variety. We investigate the subvariety of functions that are equivariant or invariant under the action of a permutation group. Examples of such group actions are translations or $90^\circ$ rotations on images. We describe such equivariant or invariant subvarieties as direct products of determinantal varieties, from which we deduce their dimension, degree, Euclidean distance degree, and their singularities. We fully characterize invariance for arbitrary permutation groups, and equivariance for cyclic groups. We draw conclusions for the parameterization and the design of equivariant and invariant linear networks in terms of sparsity and weight-sharing properties. We prove that all invariant linear functions can be parameterized by a single linear autoencoder with a weight-sharing property imposed by the cycle decomposition of the considered permutation. The space of rank-bounded equivariant functions has several irreducible components, so it can not be parameterized by a single network-but each irreducible component can. Finally, we show that minimizing the squared-error loss on our invariant or equivariant networks reduces to minimizing the Euclidean distance from determinantal varieties via the Eckart-Young theorem.
△ Less
Submitted 26 January, 2024; v1 submitted 24 September, 2023;
originally announced September 2023.
-
Function Space and Critical Points of Linear Convolutional Networks
Authors:
Kathlén Kohn,
Guido Montúfar,
Vahid Shahverdi,
Matthew Trager
Abstract:
We study the geometry of linear networks with one-dimensional convolutional layers. The function spaces of these networks can be identified with semi-algebraic families of polynomials admitting sparse factorizations. We analyze the impact of the network's architecture on the function space's dimension, boundary, and singular points. We also describe the critical points of the network's parameteriz…
▽ More
We study the geometry of linear networks with one-dimensional convolutional layers. The function spaces of these networks can be identified with semi-algebraic families of polynomials admitting sparse factorizations. We analyze the impact of the network's architecture on the function space's dimension, boundary, and singular points. We also describe the critical points of the network's parameterization map. Furthermore, we study the optimization problem of training a network with the squared error loss. We prove that for architectures where all strides are larger than one and generic data, the non-zero critical points of that optimization problem are smooth interior points of the function space. This property is known to be false for dense linear networks and linear convolutional networks with stride one.
△ Less
Submitted 26 January, 2024; v1 submitted 12 April, 2023;
originally announced April 2023.
-
Snapshot of Algebraic Vision
Authors:
Joe Kileel,
Kathlén Kohn
Abstract:
In this survey article, we present interactions between algebraic geometry and computer vision, which have recently come under the header of algebraic vision. The subject has given new insights in multiple view geometry and its application to 3D scene reconstruction and carried a host of novel problems and ideas back into algebraic geometry.
In this survey article, we present interactions between algebraic geometry and computer vision, which have recently come under the header of algebraic vision. The subject has given new insights in multiple view geometry and its application to 3D scene reconstruction and carried a host of novel problems and ideas back into algebraic geometry.
△ Less
Submitted 17 October, 2023; v1 submitted 20 October, 2022;
originally announced October 2022.
-
Geometry of Linear Convolutional Networks
Authors:
Kathlén Kohn,
Thomas Merkh,
Guido Montúfar,
Matthew Trager
Abstract:
We study the family of functions that are represented by a linear convolutional neural network (LCN). These functions form a semi-algebraic subset of the set of linear maps from input space to output space. In contrast, the families of functions represented by fully-connected linear networks form algebraic sets. We observe that the functions represented by LCNs can be identified with polynomials t…
▽ More
We study the family of functions that are represented by a linear convolutional neural network (LCN). These functions form a semi-algebraic subset of the set of linear maps from input space to output space. In contrast, the families of functions represented by fully-connected linear networks form algebraic sets. We observe that the functions represented by LCNs can be identified with polynomials that admit certain factorizations, and we use this perspective to describe the impact of the network's architecture on the geometry of the resulting function space. We further study the optimization of an objective function over an LCN, analyzing critical points in function space and in parameter space, and describing dynamical invariants for gradient descent. Overall, our theory predicts that the optimized parameters of an LCN will often correspond to repeated filters across layers, or filters that can be decomposed as repeated filters. We also conduct numerical and symbolic experiments that illustrate our results and present an in-depth analysis of the landscape for small architectures.
△ Less
Submitted 8 June, 2022; v1 submitted 3 August, 2021;
originally announced August 2021.
-
PL${}_{1}$P -- Point-line Minimal Problems under Partial Visibility in Three Views
Authors:
Timothy Duff,
Kathlén Kohn,
Anton Leykin,
Tomas Pajdla
Abstract:
We present a complete classification of minimal problems for generic arrangements of points and lines in space observed partially by three calibrated perspective cameras when each line is incident to at most one point. This is a large class of interesting minimal problems that allows missing observations in images due to occlusions and missed detections. There is an infinite number of such minimal…
▽ More
We present a complete classification of minimal problems for generic arrangements of points and lines in space observed partially by three calibrated perspective cameras when each line is incident to at most one point. This is a large class of interesting minimal problems that allows missing observations in images due to occlusions and missed detections. There is an infinite number of such minimal problems; however, we show that they can be reduced to 140616 equivalence classes by removing superfluous features and relabeling the cameras. We also introduce camera-minimal problems, which are practical for designing minimal solvers, and show how to pick a simplest camera-minimal problem for each minimal problem. This simplification results in 74575 equivalence classes. Only 76 of these were known; the rest are new. In order to identify problems that have potential for practical solving of image matching and 3D reconstruction, we present several smaller natural subfamilies of camera-minimal problems as well as compute solution counts for all camera-minimal problems which have less than 300 solutions for generic data.
△ Less
Submitted 10 March, 2020;
originally announced March 2020.
-
Pure and Spurious Critical Points: a Geometric Study of Linear Networks
Authors:
Matthew Trager,
Kathlén Kohn,
Joan Bruna
Abstract:
The critical locus of the loss function of a neural network is determined by the geometry of the functional space and by the parameterization of this space by the network's weights. We introduce a natural distinction between pure critical points, which only depend on the functional space, and spurious critical points, which arise from the parameterization. We apply this perspective to revisit and…
▽ More
The critical locus of the loss function of a neural network is determined by the geometry of the functional space and by the parameterization of this space by the network's weights. We introduce a natural distinction between pure critical points, which only depend on the functional space, and spurious critical points, which arise from the parameterization. We apply this perspective to revisit and extend the literature on the loss function of linear neural networks. For this type of network, the functional space is either the set of all linear maps from input to output space, or a determinantal variety, i.e., a set of linear maps with bounded rank. We use geometric properties of determinantal varieties to derive new results on the landscape of linear networks with different loss functions and different parameterizations. Our analysis clearly illustrates that the absence of "bad" local minima in the loss landscape of linear networks is due to two distinct phenomena that apply in different settings: it is true for arbitrary smooth convex losses in the case of architectures that can express all linear maps ("filling architectures") but it holds only for the quadratic loss when the functional space is a determinantal variety ("non-filling architectures"). Without any assumption on the architecture, smooth convex losses may lead to landscapes with many bad minima.
△ Less
Submitted 2 April, 2020; v1 submitted 3 October, 2019;
originally announced October 2019.
-
PLMP -- Point-Line Minimal Problems in Complete Multi-View Visibility
Authors:
Timothy Duff,
Kathlén Kohn,
Anton Leykin,
Tomas Pajdla
Abstract:
We present a complete classification of all minimal problems for generic arrangements of points and lines completely observed by calibrated perspective cameras. We show that there are only 30 minimal problems in total, no problems exist for more than 6 cameras, for more than 5 points, and for more than 6 lines. We present a sequence of tests for detecting minimality starting with counting degrees…
▽ More
We present a complete classification of all minimal problems for generic arrangements of points and lines completely observed by calibrated perspective cameras. We show that there are only 30 minimal problems in total, no problems exist for more than 6 cameras, for more than 5 points, and for more than 6 lines. We present a sequence of tests for detecting minimality starting with counting degrees of freedom and ending with full symbolic and numeric verification of representative examples. For all minimal problems discovered, we present their algebraic degrees, i.e. the number of solutions, which measure their intrinsic difficulty. It shows how exactly the difficulty of problems grows with the number of views. Importantly, several new minimal problems have small degrees that might be practical in image matching and 3D reconstruction.
△ Less
Submitted 5 September, 2019; v1 submitted 24 March, 2019;
originally announced March 2019.
-
Changing Views on Curves and Surfaces
Authors:
Kathlén Kohn,
Bernd Sturmfels,
Matthew Trager
Abstract:
Visual events in computer vision are studied from the perspective of algebraic geometry. Given a sufficiently general curve or surface in 3-space, we consider the image or contour curve that arises by projecting from a viewpoint. Qualitative changes in that curve occur when the viewpoint crosses the visual event surface. We examine the components of this ruled surface, and observe that these coinc…
▽ More
Visual events in computer vision are studied from the perspective of algebraic geometry. Given a sufficiently general curve or surface in 3-space, we consider the image or contour curve that arises by projecting from a viewpoint. Qualitative changes in that curve occur when the viewpoint crosses the visual event surface. We examine the components of this ruled surface, and observe that these coincide with the iterated singular loci of the coisotropic hypersurfaces associated with the original curve or surface. We derive formulas, due to Salmon and Petitjean, for the degrees of these surfaces, and show how to compute exact representations for all visual event surfaces using algebraic methods.
△ Less
Submitted 11 November, 2017; v1 submitted 6 July, 2017;
originally announced July 2017.
-
Voronoi Cells of Lattices with Respect to Arbitrary Norms
Authors:
Johannes Blömer,
Kathlén Kohn
Abstract:
We study the geometry and complexity of Voronoi cells of lattices with respect to arbitrary norms. On the positive side, we show for strictly convex and smooth norms that the geometry of Voronoi cells of lattices in any dimension is similar to the Euclidean case, i.e., the Voronoi cells are defined by the so-called Voronoi-relevant vectors and the facets of a Voronoi cell are in one-to-one corresp…
▽ More
We study the geometry and complexity of Voronoi cells of lattices with respect to arbitrary norms. On the positive side, we show for strictly convex and smooth norms that the geometry of Voronoi cells of lattices in any dimension is similar to the Euclidean case, i.e., the Voronoi cells are defined by the so-called Voronoi-relevant vectors and the facets of a Voronoi cell are in one-to-one correspondence with these vectors. On the negative side, we show that Voronoi cells are combinatorially much more complicated for arbitrary strictly convex and smooth norms than in the Euclidean case. In particular, we construct a family of three-dimensional lattices whose number of Voronoi-relevant vectors with respect to the $\ell_3$-norm is unbounded. Our result indicates, that the break through single exponential time algorithm of Micciancio and Voulgaris for solving the shortest and closest vector problem in the Euclidean norm cannot be extended to achieve deterministic single exponential time algorithms for lattice problems with respect to arbitrary $\ell_p$-norms. In fact, the algorithm of Micciancio and Voulgaris and its run time analysis crucially depend on the fact that for the Euclidean norm the number of Voronoi-relevant vectors is single exponential in the lattice dimension.
△ Less
Submitted 11 November, 2017; v1 submitted 27 November, 2015;
originally announced December 2015.
-
Computing the Chow variety of quadratic space curves
Authors:
Peter Bürgisser,
Kathlén Kohn,
Pierre Lairez,
Bernd Sturmfels
Abstract:
Quadrics in the Grassmannian of lines in 3-space form a 19-dimensional projective space. We study the subvariety of coisotropic hypersurfaces. Following Gel'fand, Kapranov and Zelevinsky, it decomposes into Chow forms of plane conics, Chow forms of pairs of lines, and Hurwitz forms of quadric surfaces. We compute the ideals of these loci.
Quadrics in the Grassmannian of lines in 3-space form a 19-dimensional projective space. We study the subvariety of coisotropic hypersurfaces. Following Gel'fand, Kapranov and Zelevinsky, it decomposes into Chow forms of plane conics, Chow forms of pairs of lines, and Hurwitz forms of quadric surfaces. We compute the ideals of these loci.
△ Less
Submitted 27 November, 2015; v1 submitted 28 August, 2015;
originally announced August 2015.
