-
A Unifying Perspective on Non-Stationary Kernels for Deeper Gaussian Processes
Authors:
Marcus M. Noack,
Hengrui Luo,
Mark D. Risser
Abstract:
The Gaussian process (GP) is a popular statistical technique for stochastic function approximation and uncertainty quantification from data. GPs have been adopted into the realm of machine learning in the last two decades because of their superior prediction abilities, especially in data-sparse scenarios, and their inherent ability to provide robust uncertainty estimates. Even so, their performanc…
▽ More
The Gaussian process (GP) is a popular statistical technique for stochastic function approximation and uncertainty quantification from data. GPs have been adopted into the realm of machine learning in the last two decades because of their superior prediction abilities, especially in data-sparse scenarios, and their inherent ability to provide robust uncertainty estimates. Even so, their performance highly depends on intricate customizations of the core methodology, which often leads to dissatisfaction among practitioners when standard setups and off-the-shelf software tools are being deployed. Arguably the most important building block of a GP is the kernel function which assumes the role of a covariance operator. Stationary kernels of the Matérn class are used in the vast majority of applied studies; poor prediction performance and unrealistic uncertainty quantification are often the consequences. Non-stationary kernels show improved performance but are rarely used due to their more complicated functional form and the associated effort and expertise needed to define and tune them optimally. In this perspective, we want to help ML practitioners make sense of some of the most common forms of non-stationarity for Gaussian processes. We show a variety of kernels in action using representative datasets, carefully study their properties, and compare their performances. Based on our findings, we propose a new kernel that combines some of the identified advantages of existing kernels.
△ Less
Submitted 18 September, 2023;
originally announced September 2023.
-
A substitutional quantum defect in WS$_2$ discovered by high-throughput computational screening and fabricated by site-selective STM manipulation
Authors:
John C. Thomas,
Wei Chen,
Yihuang Xiong,
Bradford A. Barker,
Junze Zhou,
Weiru Chen,
Antonio Rossi,
Nolan Kelly,
Zhuohang Yu,
Da Zhou,
Shalini Kumari,
Edward S. Barnard,
Joshua A. Robinson,
Mauricio Terrones,
Adam Schwartzberg,
D. Frank Ogletree,
Eli Rotenberg,
Marcus M. Noack,
Sinéad Griffin,
Archana Raja,
David A. Strubbe,
Gian-Marco Rignanese,
Alexander Weber-Bargioni,
Geoffroy Hautier
Abstract:
Point defects in two-dimensional materials are of key interest for quantum information science. However, the space of possible defects is immense, making the identification of high-performance quantum defects extremely challenging. Here, we perform high-throughput (HT) first-principles computational screening to search for promising quantum defects within WS$_2$, which present localized levels in…
▽ More
Point defects in two-dimensional materials are of key interest for quantum information science. However, the space of possible defects is immense, making the identification of high-performance quantum defects extremely challenging. Here, we perform high-throughput (HT) first-principles computational screening to search for promising quantum defects within WS$_2$, which present localized levels in the band gap that can lead to bright optical transitions in the visible or telecom regime. Our computed database spans more than 700 charged defects formed through substitution on the tungsten or sulfur site. We found that sulfur substitutions enable the most promising quantum defects. We computationally identify the neutral cobalt substitution to sulfur (Co$_{\rm S}^{0}$) as very promising and fabricate it with scanning tunneling microscopy (STM). The Co$_{\rm S}^{0}$ electronic structure measured by STM agrees with first principles and showcases an attractive new quantum defect. Our work shows how HT computational screening and novel defect synthesis routes can be combined to design new quantum defects.
△ Less
Submitted 14 September, 2023;
originally announced September 2023.
-
Exact Gaussian Processes for Massive Datasets via Non-Stationary Sparsity-Discovering Kernels
Authors:
Marcus M. Noack,
Harinarayan Krishnan,
Mark D. Risser,
Kristofer G. Reyes
Abstract:
A Gaussian Process (GP) is a prominent mathematical framework for stochastic function approximation in science and engineering applications. This success is largely attributed to the GP's analytical tractability, robustness, non-parametric structure, and natural inclusion of uncertainty quantification. Unfortunately, the use of exact GPs is prohibitively expensive for large datasets due to their u…
▽ More
A Gaussian Process (GP) is a prominent mathematical framework for stochastic function approximation in science and engineering applications. This success is largely attributed to the GP's analytical tractability, robustness, non-parametric structure, and natural inclusion of uncertainty quantification. Unfortunately, the use of exact GPs is prohibitively expensive for large datasets due to their unfavorable numerical complexity of $O(N^3)$ in computation and $O(N^2)$ in storage. All existing methods addressing this issue utilize some form of approximation -- usually considering subsets of the full dataset or finding representative pseudo-points that render the covariance matrix well-structured and sparse. These approximate methods can lead to inaccuracies in function approximations and often limit the user's flexibility in designing expressive kernels. Instead of inducing sparsity via data-point geometry and structure, we propose to take advantage of naturally-occurring sparsity by allowing the kernel to discover -- instead of induce -- sparse structure. The premise of this paper is that GPs, in their most native form, are often naturally sparse, but commonly-used kernels do not allow us to exploit this sparsity. The core concept of exact, and at the same time sparse GPs relies on kernel definitions that provide enough flexibility to learn and encode not only non-zero but also zero covariances. This principle of ultra-flexible, compactly-supported, and non-stationary kernels, combined with HPC and constrained optimization, lets us scale exact GPs well beyond 5 million data points.
△ Less
Submitted 18 May, 2022;
originally announced May 2022.
-
Autonomous Investigations over WS$_2$ and Au{111} with Scanning Probe Microscopy
Authors:
John C. Thomas,
Antonio Rossi,
Darian Smalley,
Luca Francaviglia,
Zhuohang Yu,
Tianyi Zhang,
Shalini Kumari,
Joshua A. Robinson,
Mauricio Terrones,
Masahiro Ishigami,
Eli Rotenberg,
Edward S. Barnard,
Archana Raja,
Ed Wong,
D. Frank Ogletree,
Marcus M. Noack,
Alexander Weber-Bargioni
Abstract:
Individual atomic defects in 2D materials impact their macroscopic functionality. Correlating the interplay is challenging, however, intelligent hyperspectral scanning tunneling spectroscopy (STS) map** provides a feasible solution to this technically difficult and time consuming problem. Here, dense spectroscopic volume is collected autonomously via Gaussian process regression, where convolutio…
▽ More
Individual atomic defects in 2D materials impact their macroscopic functionality. Correlating the interplay is challenging, however, intelligent hyperspectral scanning tunneling spectroscopy (STS) map** provides a feasible solution to this technically difficult and time consuming problem. Here, dense spectroscopic volume is collected autonomously via Gaussian process regression, where convolutional neural networks are used in tandem for spectral identification. Acquired data enable defect segmentation, and a workflow is provided for machine-driven decision making during experimentation with capability for user customization. We provide a means towards autonomous experimentation for the benefit of both enhanced reproducibility and user-accessibility. Hyperspectral investigations on WS$_2$ sulfur vacancy sites are explored, which is combined with local density of states confirmation on the Au{111} herringbone reconstruction. Chalcogen vacancies, pristine WS$_2$, Au face-centered cubic, and Au hexagonal close packed regions are examined and detected by machine learning methods to demonstrate the potential of artificial intelligence for hyperspectral STS map**.
△ Less
Submitted 2 May, 2022; v1 submitted 7 October, 2021;
originally announced October 2021.
-
Advanced Stationary and Non-Stationary Kernel Designs for Domain-Aware Gaussian Processes
Authors:
Marcus M. Noack,
James A. Sethian
Abstract:
Gaussian process regression is a widely-applied method for function approximation and uncertainty quantification. The technique has gained popularity recently in the machine learning community due to its robustness and interpretability. The mathematical methods we discuss in this paper are an extension of the Gaussian-process framework. We are proposing advanced kernel designs that only allow for…
▽ More
Gaussian process regression is a widely-applied method for function approximation and uncertainty quantification. The technique has gained popularity recently in the machine learning community due to its robustness and interpretability. The mathematical methods we discuss in this paper are an extension of the Gaussian-process framework. We are proposing advanced kernel designs that only allow for functions with certain desirable characteristics to be elements of the reproducing kernel Hilbert space (RKHS) that underlies all kernel methods and serves as the sample space for Gaussian process regression. These desirable characteristics reflect the underlying physics; two obvious examples are symmetry and periodicity constraints. In addition, non-stationary kernel designs can be defined in the same framework to yield flexible multi-task Gaussian processes. We will show the impact of advanced kernel designs on Gaussian processes using several synthetic and two scientific data sets. The results show that including domain knowledge, communicated through advanced kernel designs, has a significant impact on the accuracy and relevance of the function approximation.
△ Less
Submitted 27 January, 2022; v1 submitted 5 February, 2021;
originally announced February 2021.
-
Autonomous Materials Discovery Driven by Gaussian Process Regression with Inhomogeneous Measurement Noise and Anisotropic Kernels
Authors:
Marcus M. Noack,
Gregory S. Doerk,
Ruipeng Li,
Jason K. Streit,
Richard A. Vaia,
Kevin G. Yager,
Masafumi Fukuto
Abstract:
A majority of experimental disciplines face the challenge of exploring large and high-dimensional parameter spaces in search of new scientific discoveries. Materials science is no exception; the wide variety of synthesis, processing, and environmental conditions that influence material properties gives rise to particularly vast parameter spaces. Recent advances have led to an increase in efficienc…
▽ More
A majority of experimental disciplines face the challenge of exploring large and high-dimensional parameter spaces in search of new scientific discoveries. Materials science is no exception; the wide variety of synthesis, processing, and environmental conditions that influence material properties gives rise to particularly vast parameter spaces. Recent advances have led to an increase in efficiency of materials discovery by increasingly automating the exploration processes. Methods for autonomous experimentation have become more sophisticated recently, allowing for multi-dimensional parameter spaces to be explored efficiently and with minimal human intervention, thereby liberating the scientists to focus on interpretations and big-picture decisions. Gaussian process regression (GPR) techniques have emerged as the method of choice for steering many classes of experiments. We have recently demonstrated the positive impact of GPR-driven decision-making algorithms on autonomously steering experiments at a synchrotron beamline. However, due to the complexity of the experiments, GPR often cannot be used in its most basic form, but rather has to be tuned to account for the special requirements of the experiments. Two requirements seem to be of particular importance, namely inhomogeneous measurement noise (input dependent or non-i.i.d.) and anisotropic kernel functions, which are the two concepts that we tackle in this paper. Our synthetic and experimental tests demonstrate the importance of both concepts for experiments in materials science and the benefits that result from including them in the autonomous decision-making process.
△ Less
Submitted 3 June, 2020;
originally announced June 2020.
