-
Neural collapse with unconstrained features
Authors:
Dustin G. Mixon,
Hans Parshall,
Jianzong Pi
Abstract:
Neural collapse is an emergent phenomenon in deep learning that was recently discovered by Papyan, Han and Donoho. We propose a simple "unconstrained features model" in which neural collapse also emerges empirically. By studying this model, we provide some explanation for the emergence of neural collapse in terms of the landscape of empirical risk.
Neural collapse is an emergent phenomenon in deep learning that was recently discovered by Papyan, Han and Donoho. We propose a simple "unconstrained features model" in which neural collapse also emerges empirically. By studying this model, we provide some explanation for the emergence of neural collapse in terms of the landscape of empirical risk.
△ Less
Submitted 23 November, 2020;
originally announced November 2020.
-
Uniquely optimal codes of low complexity are symmetric
Authors:
Christopher Cox,
Emily J. King,
Dustin G. Mixon,
Hans Parshall
Abstract:
We formulate explicit predictions concerning the symmetry of optimal codes in compact metric spaces. This motivates the study of optimal codes in various spaces where these predictions can be tested.
We formulate explicit predictions concerning the symmetry of optimal codes in compact metric spaces. This motivates the study of optimal codes in various spaces where these predictions can be tested.
△ Less
Submitted 15 September, 2020; v1 submitted 28 August, 2020;
originally announced August 2020.
-
Lie PCA: Density estimation for symmetric manifolds
Authors:
Jameson Cahill,
Dustin G. Mixon,
Hans Parshall
Abstract:
We introduce an extension to local principal component analysis for learning symmetric manifolds. In particular, we use a spectral method to approximate the Lie algebra corresponding to the symmetry group of the underlying manifold. We derive the sample complexity of our method for a variety of manifolds before applying it to various data sets for improved density estimation.
We introduce an extension to local principal component analysis for learning symmetric manifolds. In particular, we use a spectral method to approximate the Lie algebra corresponding to the symmetry group of the underlying manifold. We derive the sample complexity of our method for a variety of manifolds before applying it to various data sets for improved density estimation.
△ Less
Submitted 13 September, 2020; v1 submitted 10 August, 2020;
originally announced August 2020.
-
Globally optimizing small codes in real projective spaces
Authors:
Dustin G. Mixon,
Hans Parshall
Abstract:
For $d\in\{5,6\}$, we classify arrangements of $d + 2$ points in $\mathbf{RP}^{d-1}$ for which the minimum distance is as large as possible. To do so, we leverage ideas from matrix and convex analysis to determine the best possible codes that contain equiangular lines, and we introduce a notion of approximate Positivstellensatz certificates that promotes numerical approximations of Stengle's Posit…
▽ More
For $d\in\{5,6\}$, we classify arrangements of $d + 2$ points in $\mathbf{RP}^{d-1}$ for which the minimum distance is as large as possible. To do so, we leverage ideas from matrix and convex analysis to determine the best possible codes that contain equiangular lines, and we introduce a notion of approximate Positivstellensatz certificates that promotes numerical approximations of Stengle's Positivstellensatz certificates to honest certificates.
△ Less
Submitted 6 December, 2019;
originally announced December 2019.
-
Linear programming bounds for cliques in Paley graphs
Authors:
Mark Magsino,
Dustin G. Mixon,
Hans Parshall
Abstract:
The Lovász theta number is a semidefinite programming bound on the clique number of (the complement of) a given graph. Given a vertex-transitive graph, every vertex belongs to a maximal clique, and so one can instead apply this semidefinite programming bound to the local graph. In the case of the Paley graph, the local graph is circulant, and so this bound reduces to a linear programming bound, al…
▽ More
The Lovász theta number is a semidefinite programming bound on the clique number of (the complement of) a given graph. Given a vertex-transitive graph, every vertex belongs to a maximal clique, and so one can instead apply this semidefinite programming bound to the local graph. In the case of the Paley graph, the local graph is circulant, and so this bound reduces to a linear programming bound, allowing for fast computations. Impressively, the value of this program with Schrijver's nonnegativity constraint rivals the state-of-the-art closed-form bound recently proved by Hanson and Petridis. We conjecture that this linear programming bound improves on the Hanson-Petridis bound infinitely often, and we derive the dual program to facilitate proving this conjecture.
△ Less
Submitted 12 July, 2019;
originally announced July 2019.
-
Kesten-McKay law for random subensembles of Paley equiangular tight frames
Authors:
Mark Magsino,
Dustin G. Mixon,
Hans Parshall
Abstract:
We apply the method of moments to prove a recent conjecture of Haikin, Zamir and Gavish (2017) concerning the distribution of the singular values of random subensembles of Paley equiangular tight frames. Our analysis applies more generally to real equiangular tight frames of redundancy 2, and we suspect similar ideas will eventually produce more general results for arbitrary choices of redundancy.
We apply the method of moments to prove a recent conjecture of Haikin, Zamir and Gavish (2017) concerning the distribution of the singular values of random subensembles of Paley equiangular tight frames. Our analysis applies more generally to real equiangular tight frames of redundancy 2, and we suspect similar ideas will eventually produce more general results for arbitrary choices of redundancy.
△ Less
Submitted 10 May, 2019;
originally announced May 2019.
-
The optimal packing of eight points in the real projective plane
Authors:
Dustin G. Mixon,
Hans Parshall
Abstract:
How can we arrange $n$ lines through the origin in three-dimensional Euclidean space in a way that maximizes the minimum interior angle between pairs of lines? Conway, Hardin and Sloane (1996) produced line packings for $n \leq 55$ that they conjectured to be within numerical precision of optimal in this sense, but until now only the cases $n \leq 7$ have been solved. In this paper, we resolve the…
▽ More
How can we arrange $n$ lines through the origin in three-dimensional Euclidean space in a way that maximizes the minimum interior angle between pairs of lines? Conway, Hardin and Sloane (1996) produced line packings for $n \leq 55$ that they conjectured to be within numerical precision of optimal in this sense, but until now only the cases $n \leq 7$ have been solved. In this paper, we resolve the case $n = 8$. Drawing inspiration from recent work on the Tammes problem, we enumerate contact graph candidates for an optimal configuration and eliminate those that violate various combinatorial and geometric necessary conditions. The contact graph of the putatively optimal numerical packing of Conway, Hardin and Sloane is the only graph that survives, and we recover from this graph an exact expression for the minimum distance of eight optimally packed points in the real projective plane.
△ Less
Submitted 26 February, 2019;
originally announced February 2019.
-
A Delsarte-Style Proof of the Bukh-Cox Bound
Authors:
Mark Magsino,
Dustin G. Mixon,
Hans Parshall
Abstract:
The line packing problem is concerned with the optimal packing of points in real or complex projective space so that the minimum distance between points is maximized. Until recently, all bounds on optimal line packings were known to be derivable from Delsarte's linear program. Last year, Bukh and Cox introduced a new bound for the line packing problem using completely different techniques. In this…
▽ More
The line packing problem is concerned with the optimal packing of points in real or complex projective space so that the minimum distance between points is maximized. Until recently, all bounds on optimal line packings were known to be derivable from Delsarte's linear program. Last year, Bukh and Cox introduced a new bound for the line packing problem using completely different techniques. In this paper, we use ideas from the Bukh--Cox proof to find a new proof of the Welch bound, and then we use ideas from Delsarte's linear program to find a new proof of the Bukh--Cox bound. Hopefully, these unifying principles will lead to further refinements.
△ Less
Submitted 1 May, 2019; v1 submitted 3 February, 2019;
originally announced February 2019.
-
Exact Line Packings from Numerical Solutions
Authors:
Dustin G. Mixon,
Hans Parshall
Abstract:
Recent progress in Zauner's conjecture has leveraged deep conjectures in algebraic number theory to promote numerical line packings to exact and verifiable solutions to the line packing problem. We introduce a numerical-to-exact technique in the real setting that does not require such conjectures. Our approach is completely reproducible, matching Sloane's database of putatively optimal numerical l…
▽ More
Recent progress in Zauner's conjecture has leveraged deep conjectures in algebraic number theory to promote numerical line packings to exact and verifiable solutions to the line packing problem. We introduce a numerical-to-exact technique in the real setting that does not require such conjectures. Our approach is completely reproducible, matching Sloane's database of putatively optimal numerical line packings with Mathematica's built-in implementation of cylindrical algebraic decomposition. As a proof of concept, we promote a putatively optimal numerical packing of eight points in the real projective plane to an exact packing, whose optimality we establish in a forthcoming paper.
△ Less
Submitted 1 May, 2019; v1 submitted 29 January, 2019;
originally announced February 2019.
