Showing 1–6 of 6 results for author: Kolpakov, A

Machine Learning of the Prime Distribution

Authors: Alexander Kolpakov, A. Alistair Rocke

Abstract: In the present work we use maximum entropy methods to derive several theorems in probabilistic number theory, including a version of the Hardy-Ramanujan Theorem. We also provide a theoretical argument explaining the experimental observations of Yang-Hui He about the learnability of primes, and posit that the Erdős-Kac law would very unlikely be discovered by current machine learning techniques. Nu… ▽ More In the present work we use maximum entropy methods to derive several theorems in probabilistic number theory, including a version of the Hardy-Ramanujan Theorem. We also provide a theoretical argument explaining the experimental observations of Yang-Hui He about the learnability of primes, and posit that the Erdős-Kac law would very unlikely be discovered by current machine learning techniques. Numerical experiments that we perform corroborate our theoretical findings. △ Less

Submitted 2 June, 2024; v1 submitted 19 March, 2024; originally announced March 2024.

Comments: 10 pages; parts of arXiv:2308.10817 reworked and amended; author's draft; accepted in PLOS ONE

MSC Class: 11N05

A ripple in time: a discontinuity in American history

Authors: Alexander Kolpakov, Igor Rivin

Abstract: In this note we use the State of the Union Address (SOTU) dataset from Kaggle to make some surprising (and some not so surprising) observations pertaining to the general timeline of American history, and the character and nature of the addresses themselves. Our main approach is using vector embeddings, such as BERT (DistilBERT) and GPT-2. While it is widely believed that BERT (and its variations… ▽ More In this note we use the State of the Union Address (SOTU) dataset from Kaggle to make some surprising (and some not so surprising) observations pertaining to the general timeline of American history, and the character and nature of the addresses themselves. Our main approach is using vector embeddings, such as BERT (DistilBERT) and GPT-2. While it is widely believed that BERT (and its variations) is most suitable for NLP classification tasks, we find out that GPT-2 in conjunction with nonlinear dimension reduction methods such as UMAP provide better separation and stronger clustering. This makes GPT-2 + UMAP an interesting alternative. In our case, no model fine-tuning is required, and the pre-trained out-of-the-box GPT-2 model is enough. We also used a fine-tuned DistilBERT model for classification detecting which President delivered which address, with very good results (accuracy 93% - 95% depending on the run). An analogous task was performed to determine the year of writing, and we were able to pin it down to about 4 years (which is a single presidential term). It is worth noting that SOTU addresses provide relatively small writing samples (with about 8'000 words on average, and varying widely from under 2'000 words to more than 20'000), and that the number of authors is relatively large (we used SOTU addresses of 42 US presidents). This shows that the techniques employed turn out to be rather efficient, while all the computations described in this note can be performed using a single GPU instance of Google Colab. The accompanying code is available on GitHub. △ Less

Submitted 4 May, 2024; v1 submitted 2 December, 2023; originally announced December 2023.

Comments: 7 pages, 8 figures; GitHub repository (https://github.com/sashakolpakov/ripple_in_time); Section 3: added comparison to (https://doi.org/10.1016/j.ins.2019.01.040); comments on a misleading accuracy claim in (https://doi.org/10.1002/asi.23283)

ACM Class: I.2.7; I.5.4; H.3.1; H.3.3

The Information Geometry of UMAP

Authors: Alexander Kolpakov, A. Alistair Rocke

Abstract: Although UMAP was derived from Category Theory observations, its underlying mechanisms may be clarified using Information Geometry. Although UMAP was derived from Category Theory observations, its underlying mechanisms may be clarified using Information Geometry. △ Less

Submitted 25 June, 2024; v1 submitted 3 September, 2023; originally announced September 2023.

Comments: 11 pages, 2 figures, 3 tables; Github repo (https://github.com/sashakolpakov/info-geometry-umap)

MSC Class: 53B12; 94A15

On the impossibility of discovering a formula for primes using AI

Authors: Alexander Kolpakov, A. Alistair Rocke

Abstract: The present work explores the theoretical limits of Machine Learning (ML) within the framework of Kolmogorov's theory of Algorithmic Probability, which clarifies the notion of entropy as Expected Kolmogorov Complexity and formalizes other fundamental concepts such as Occam's razor via Levin's Universal Distribution. As a fundamental application, we develop Maximum Entropy methods that allow us to… ▽ More The present work explores the theoretical limits of Machine Learning (ML) within the framework of Kolmogorov's theory of Algorithmic Probability, which clarifies the notion of entropy as Expected Kolmogorov Complexity and formalizes other fundamental concepts such as Occam's razor via Levin's Universal Distribution. As a fundamental application, we develop Maximum Entropy methods that allow us to derive the Erdős-Kac Law and Hardy-Ramanujan theorem in Probabilistic Number Theory, and establish the impossibility of discovering a formula for primes using Machine Learning via the Prime Coding Theorem. △ Less

Submitted 2 June, 2024; v1 submitted 27 July, 2023; originally announced August 2023.

Comments: 29 pages; parts of this manuscript are accepted as a separate paper in PLOS ONE

MSC Class: 11N05 11N05 11N05

Robust affine point matching via quadratic assignment on Grassmannians

Authors: Alexander Kolpakov, Michael Werman

Abstract: Robust Affine matching with Grassmannians (RAG) is a new algorithm to perform affine registration of point clouds. The algorithm is based on minimizing the Frobenius distance between two elements of the Grassmannian. For this purpose, an indefinite relaxation of the Quadratic Assignment Problem (QAP) is used, and several approaches to affine feature matching are studied and compared. Experiments d… ▽ More Robust Affine matching with Grassmannians (RAG) is a new algorithm to perform affine registration of point clouds. The algorithm is based on minimizing the Frobenius distance between two elements of the Grassmannian. For this purpose, an indefinite relaxation of the Quadratic Assignment Problem (QAP) is used, and several approaches to affine feature matching are studied and compared. Experiments demonstrate that RAG is more robust to noise and point discrepancy than previous methods. △ Less

Submitted 4 May, 2024; v1 submitted 5 March, 2023; originally announced March 2023.

Comments: 8 pages, 23 figures; GitHub repository at (https://github.com/sashakolpakov/rag); Section IV: added comparison to GrassGraph (https://doi.org/10.1109/TIP.2019.2959722); notably, GrassGraph quickly loses accuracy on our test examples with noise and occlusion

An approach to robust ICP initialization

Authors: Alexander Kolpakov, Michael Werman

Abstract: In this note, we propose an approach to initialize the Iterative Closest Point (ICP) algorithm to match unlabelled point clouds related by rigid transformations. The method is based on matching the ellipsoids defined by the points' covariance matrices and then testing the various principal half-axes matchings that differ by elements of a finite reflection group. We derive bounds on the robustness… ▽ More In this note, we propose an approach to initialize the Iterative Closest Point (ICP) algorithm to match unlabelled point clouds related by rigid transformations. The method is based on matching the ellipsoids defined by the points' covariance matrices and then testing the various principal half-axes matchings that differ by elements of a finite reflection group. We derive bounds on the robustness of our approach to noise and numerical experiments confirm our theoretical findings. △ Less

Submitted 25 June, 2023; v1 submitted 10 December, 2022; originally announced December 2022.

Comments: 9 pages, 18 figures, 1 table; GitHub repository at (https://github.com/sashakolpakov/icp-init)