-
Evaluating the diversity and utility of materials proposed by generative models
Authors:
Alexander New,
Michael Pekala,
Elizabeth A. Pogue,
Nam Q. Le,
Janna Domenico,
Christine D. Piatko,
Christopher D. Stiles
Abstract:
Generative machine learning models can use data generated by scientific modeling to create large quantities of novel material structures. Here, we assess how one state-of-the-art generative model, the physics-guided crystal generation model (PGCGM), can be used as part of the inverse design process. We show that the default PGCGM's input space is not smooth with respect to parameter variation, mak…
▽ More
Generative machine learning models can use data generated by scientific modeling to create large quantities of novel material structures. Here, we assess how one state-of-the-art generative model, the physics-guided crystal generation model (PGCGM), can be used as part of the inverse design process. We show that the default PGCGM's input space is not smooth with respect to parameter variation, making material optimization difficult and limited. We also demonstrate that most generated structures are predicted to be thermodynamically unstable by a separate property-prediction model, partially due to out-of-domain data challenges. Our findings suggest how generative models might be improved to enable better inverse design.
△ Less
Submitted 9 August, 2023;
originally announced September 2023.
-
Closed-loop machine learning for discovery of novel superconductors
Authors:
Elizabeth A. Pogue,
Alexander New,
Kyle McElroy,
Nam Q. Le,
Michael J. Pekala,
Ian McCue,
Eddie Gienger,
Janna Domenico,
Elizabeth Hedrick,
Tyrel M. McQueen,
Brandon Wilfong,
Christine D. Piatko,
Christopher R. Ratto,
Andrew Lennon,
Christine Chung,
Timothy Montalbano,
Gregory Bassen,
Christopher D. Stiles
Abstract:
The discovery of novel materials drives industrial innovation, although the pace of discovery tends to be slow due to the infrequency of "Eureka!" moments. These moments are typically tangential to the original target of the experimental work: "accidental discoveries". Here we demonstrate the acceleration of intentional materials discovery - targeting material properties of interest while generali…
▽ More
The discovery of novel materials drives industrial innovation, although the pace of discovery tends to be slow due to the infrequency of "Eureka!" moments. These moments are typically tangential to the original target of the experimental work: "accidental discoveries". Here we demonstrate the acceleration of intentional materials discovery - targeting material properties of interest while generalizing the search to a large materials space with machine learning (ML) methods. We demonstrate a closed-loop ML discovery process targeting novel superconducting materials, which have industrial applications ranging from quantum computing to sensors to power delivery. By closing the loop, i.e. by experimentally testing the results of the ML-generated superconductivity predictions and feeding data back into the ML model to refine, we demonstrate that success rates for superconductor discovery can be more than doubled. In four closed-loop cycles, we discovered a new superconductor in the Zr-In-Ni system, re-discovered five superconductors unknown in the training datasets, and identified two additional phase diagrams of interest for new superconducting materials. Our work demonstrates the critical role experimental feedback provides in ML-driven discovery, and provides definite evidence that such technologies can accelerate discovery even in the absence of knowledge of the underlying physics.
△ Less
Submitted 22 December, 2022;
originally announced December 2022.
-
Curvature-informed multi-task learning for graph networks
Authors:
Alexander New,
Michael J. Pekala,
Nam Q. Le,
Janna Domenico,
Christine D. Piatko,
Christopher D. Stiles
Abstract:
Properties of interest for crystals and molecules, such as band gap, elasticity, and solubility, are generally related to each other: they are governed by the same underlying laws of physics. However, when state-of-the-art graph neural networks attempt to predict multiple properties simultaneously (the multi-task learning (MTL) setting), they frequently underperform a suite of single property pred…
▽ More
Properties of interest for crystals and molecules, such as band gap, elasticity, and solubility, are generally related to each other: they are governed by the same underlying laws of physics. However, when state-of-the-art graph neural networks attempt to predict multiple properties simultaneously (the multi-task learning (MTL) setting), they frequently underperform a suite of single property predictors. This suggests graph networks may not be fully leveraging these underlying similarities. Here we investigate a potential explanation for this phenomenon: the curvature of each property's loss surface significantly varies, leading to inefficient learning. This difference in curvature can be assessed by looking at spectral properties of the Hessians of each property's loss function, which is done in a matrix-free manner via randomized numerical linear algebra. We evaluate our hypothesis on two benchmark datasets (Materials Project (MP) and QM8) and consider how these findings can inform the training of novel multi-task learning models.
△ Less
Submitted 2 August, 2022;
originally announced August 2022.
-
Maximal function pooling with applications
Authors:
Wojciech Czaja,
Weilin Li,
Yiran Li,
Mike Pekala
Abstract:
Inspired by the Hardy-Littlewood maximal function, we propose a novel pooling strategy which is called maxfun pooling. It is presented both as a viable alternative to some of the most popular pooling functions, such as max pooling and average pooling, and as a way of interpolating between these two algorithms. We demonstrate the features of maxfun pooling with two applications: first in the contex…
▽ More
Inspired by the Hardy-Littlewood maximal function, we propose a novel pooling strategy which is called maxfun pooling. It is presented both as a viable alternative to some of the most popular pooling functions, such as max pooling and average pooling, and as a way of interpolating between these two algorithms. We demonstrate the features of maxfun pooling with two applications: first in the context of convolutional sparse coding, and then for image classification.
△ Less
Submitted 1 March, 2021;
originally announced March 2021.
-
Adversarial Examples in Remote Sensing
Authors:
Wojciech Czaja,
Neil Fendley,
Michael Pekala,
Christopher Ratto,
I-Jeng Wang
Abstract:
This paper considers attacks against machine learning algorithms used in remote sensing applications, a domain that presents a suite of challenges that are not fully addressed by current research focused on natural image data such as ImageNet. In particular, we present a new study of adversarial examples in the context of satellite image classification problems. Using a recently curated data set a…
▽ More
This paper considers attacks against machine learning algorithms used in remote sensing applications, a domain that presents a suite of challenges that are not fully addressed by current research focused on natural image data such as ImageNet. In particular, we present a new study of adversarial examples in the context of satellite image classification problems. Using a recently curated data set and associated classifier, we provide a preliminary analysis of adversarial examples in settings where the targeted classifier is permitted multiple observations of the same location over time. While our experiments to date are purely digital, our problem setup explicitly incorporates a number of practical considerations that a real-world attacker would need to take into account when mounting a physical attack. We hope this work provides a useful starting point for future studies of potential vulnerabilities in this setting.
△ Less
Submitted 28 May, 2018;
originally announced May 2018.
-
Deep Learning based Retinal OCT Segmentation
Authors:
Mike Pekala,
Neil Joshi,
David E. Freund,
Neil M. Bressler,
Delia Cabrera DeBuc,
Philippe M Burlina
Abstract:
Our objective is to evaluate the efficacy of methods that use deep learning (DL) for the automatic fine-grained segmentation of optical coherence tomography (OCT) images of the retina. OCT images from 10 patients with mild non-proliferative diabetic retinopathy were used from a public (U. of Miami) dataset. For each patient, five images were available: one image of the fovea center, two images of…
▽ More
Our objective is to evaluate the efficacy of methods that use deep learning (DL) for the automatic fine-grained segmentation of optical coherence tomography (OCT) images of the retina. OCT images from 10 patients with mild non-proliferative diabetic retinopathy were used from a public (U. of Miami) dataset. For each patient, five images were available: one image of the fovea center, two images of the perifovea, and two images of the parafovea. For each image, two expert graders each manually annotated five retinal surfaces (i.e. boundaries between pairs of retinal layers). The first grader's annotations were used as ground truth and the second grader's annotations to compute inter-operator agreement. The proposed automated approach segments images using fully convolutional networks (FCNs) together with Gaussian process (GP)-based regression as a post-processing step to improve the quality of the estimates. Using 10-fold cross validation, the performance of the algorithms is determined by computing the per-pixel unsigned error (distance) between the automated estimates and the ground truth annotations generated by the first manual grader. We compare the proposed method against five state of the art automatic segmentation techniques. The results show that the proposed methods compare favorably with state of the art techniques, resulting in the smallest mean unsigned error values and associated standard deviations, and performance is comparable with human annotation of retinal layers from OCT when there is only mild retinopathy. The results suggest that semantic segmentation using FCNs, coupled with regression-based post-processing, can effectively solve the OCT segmentation problem on par with human capabilities with mild retinopathy.
△ Less
Submitted 29 January, 2018;
originally announced January 2018.
-
An Automated Images-to-Graphs Framework for High Resolution Connectomics
Authors:
William Gray Roncal,
Dean M. Kleissas,
Joshua T. Vogelstein,
Priya Manavalan,
Kunal Lillaney,
Michael Pekala,
Randal Burns,
R. Jacob Vogelstein,
Carey E. Priebe,
Mark A. Chevillet,
Gregory D. Hager
Abstract:
Reconstructing a map of neuronal connectivity is a critical challenge in contemporary neuroscience. Recent advances in high-throughput serial section electron microscopy (EM) have produced massive 3D image volumes of nanoscale brain tissue for the first time. The resolution of EM allows for individual neurons and their synaptic connections to be directly observed. Recovering neuronal networks by m…
▽ More
Reconstructing a map of neuronal connectivity is a critical challenge in contemporary neuroscience. Recent advances in high-throughput serial section electron microscopy (EM) have produced massive 3D image volumes of nanoscale brain tissue for the first time. The resolution of EM allows for individual neurons and their synaptic connections to be directly observed. Recovering neuronal networks by manually tracing each neuronal process at this scale is unmanageable, and therefore researchers are develo** automated image processing modules. Thus far, state-of-the-art algorithms focus only on the solution to a particular task (e.g., neuron segmentation or synapse identification).
In this manuscript we present the first fully automated images-to-graphs pipeline (i.e., a pipeline that begins with an imaged volume of neural tissue and produces a brain graph without any human interaction). To evaluate overall performance and select the best parameters and methods, we also develop a metric to assess the quality of the output graphs. We evaluate a set of algorithms and parameters, searching possible operating points to identify the best available brain graph for our assessment metric. Finally, we deploy a reference end-to-end version of the pipeline on a large, publicly available data set. This provides a baseline result and framework for community analysis and future algorithm development and testing. All code and data derivatives have been made publicly available toward eventually unlocking new biofidelic computational primitives and understanding of neuropathologies.
△ Less
Submitted 30 April, 2015; v1 submitted 25 November, 2014;
originally announced November 2014.
-
VESICLE: Volumetric Evaluation of Synaptic Interfaces using Computer vision at Large Scale
Authors:
William Gray Roncal,
Michael Pekala,
Verena Kaynig-Fittkau,
Dean M. Kleissas,
Joshua T. Vogelstein,
Hanspeter Pfister,
Randal Burns,
R. Jacob Vogelstein,
Mark A. Chevillet,
Gregory D. Hager
Abstract:
An open challenge problem at the forefront of modern neuroscience is to obtain a comprehensive map** of the neural pathways that underlie human brain function; an enhanced understanding of the wiring diagram of the brain promises to lead to new breakthroughs in diagnosing and treating neurological disorders. Inferring brain structure from image data, such as that obtained via electron microscopy…
▽ More
An open challenge problem at the forefront of modern neuroscience is to obtain a comprehensive map** of the neural pathways that underlie human brain function; an enhanced understanding of the wiring diagram of the brain promises to lead to new breakthroughs in diagnosing and treating neurological disorders. Inferring brain structure from image data, such as that obtained via electron microscopy (EM), entails solving the problem of identifying biological structures in large data volumes. Synapses, which are a key communication structure in the brain, are particularly difficult to detect due to their small size and limited contrast. Prior work in automated synapse detection has relied upon time-intensive biological preparations (post-staining, isotropic slice thicknesses) in order to simplify the problem.
This paper presents VESICLE, the first known approach designed for mammalian synapse detection in anisotropic, non-post-stained data. Our methods explicitly leverage biological context, and the results exceed existing synapse detection methods in terms of accuracy and scalability. We provide two different approaches - one a deep learning classifier (VESICLE-CNN) and one a lightweight Random Forest approach (VESICLE-RF) to offer alternatives in the performance-scalability space. Addressing this synapse detection challenge enables the analysis of high-throughput imaging data soon expected to reach petabytes of data, and provide tools for more rapid estimation of brain-graphs. Finally, to facilitate community efforts, we developed tools for large-scale object detection, and demonstrated this framework to find $\approx$ 50,000 synapses in 60,000 $μm ^3$ (220 GB on disk) of electron microscopy data.
△ Less
Submitted 7 September, 2015; v1 submitted 14 March, 2014;
originally announced March 2014.
-
Magnetocaloric effect in nano- and polycrystalline manganite $La_{0.7}Ca_{0.3}MnO_3$
Authors:
M. Pekala,
V. Drozd,
J. F. Fagnard,
Ph. Vanderbemden,
M. Ausloos
Abstract:
$La_{0.7}Ca_{0.3}MnO_3$ samples were prepared in nano- and polycrystalline forms by sol-gel and solid state reaction methods, respectively, and structurally characterized by synchrotron X-ray diffraction. The magnetic properties determined by ac susceptibility and dc magnetization measurements are discussed. The magnetocaloric effect in this nanocrystalline manganite is spread over a broader tem…
▽ More
$La_{0.7}Ca_{0.3}MnO_3$ samples were prepared in nano- and polycrystalline forms by sol-gel and solid state reaction methods, respectively, and structurally characterized by synchrotron X-ray diffraction. The magnetic properties determined by ac susceptibility and dc magnetization measurements are discussed. The magnetocaloric effect in this nanocrystalline manganite is spread over a broader temperature interval than in the polycrystalline case. The relative cooling power of the poly- and nanocrystalline manganites is used to evaluate a possible application for magnetic cooling below room temperature.
△ Less
Submitted 29 October, 2007;
originally announced October 2007.
-
Influence of Zeeman splitting and thermally excited polaron states on magneto-electrical and magneto-thermal properties of magnetoresistive polycrystalline manganite La_{0.8}Sr_{0.2}MnO_3
Authors:
S. Sergeenkov,
J. Mucha,
M. Pekala,
V. Drozd,
M. Ausloos
Abstract:
Some possible connection between spin and charge degrees of freedom in magneto-resistive manganites is investigated through a thorough experimental study of the magnetic (AC susceptibility and DC magnetization) and transport (resistivity and thermal conductivity) properties. Measurements are reported in the case of well characterized polycrystalline La_{0.8}Sr_{0.2}MnO_3 samples. The experimenta…
▽ More
Some possible connection between spin and charge degrees of freedom in magneto-resistive manganites is investigated through a thorough experimental study of the magnetic (AC susceptibility and DC magnetization) and transport (resistivity and thermal conductivity) properties. Measurements are reported in the case of well characterized polycrystalline La_{0.8}Sr_{0.2}MnO_3 samples. The experimental results suggest rather strong field-induced polarization effects in our material, clearly indicating the presence of ordered FM regions inside the semiconducting phase. Using an analytical expression which fits the spontaneous DC magnetization, the temperature and magnetic field dependences of both electrical resistivity and thermal conductivity data are found to be well reproduced through a universal scenario based on two mechanisms: (i) a magnetization dependent spin polaron hop** influenced by a Zeeman splitting effect, and (ii) properly defined thermally excited polaron states which have to be taken into account in order to correctly describe the behavior of the less conducting region. Using the experimentally found values of the magnetic and electron localization temperatures, we obtain L=0.5nm and m_p=3.2m_e for estimates of the localization length (size of the spin polaron) and effective polaron mass, respectively.
△ Less
Submitted 26 September, 2007;
originally announced September 2007.
-
Effect of Ga do** on magnetotransport properties in collosal magnetoresistive La0.7Ca0.3Mn1-xGaxO3 (0 < x < 0.1)
Authors:
Marek Pekala,
Jan Mucha,
Benedicte Vertruyen,
Rudi Cloots,
Marcel Ausloos
Abstract:
Samples of La0.7Ca0.3Mn1-xGaxO3 with x = 0, 0.025, 0.05 and 0.10 were prepared by standard solid-state reaction. They were first characterized chemically, including the microstructure. The magnetic properties and various transport properties, i.e. the electrical resistivity, magnetoresistivity (for a field below 8T), thermoelectric power and thermal conductivity measured each time on the same sa…
▽ More
Samples of La0.7Ca0.3Mn1-xGaxO3 with x = 0, 0.025, 0.05 and 0.10 were prepared by standard solid-state reaction. They were first characterized chemically, including the microstructure. The magnetic properties and various transport properties, i.e. the electrical resistivity, magnetoresistivity (for a field below 8T), thermoelectric power and thermal conductivity measured each time on the same sample, are reported. The markedly different behavior of the x = 0.1 sample from those with a smaller Ga content, is discussed. The dilution of the Mn3+/Mn4+ interactions with Ga do** considerably reduces the ferromagnetic double exchange interaction within the manganese lattice leading to a decrease of the Curie temperature. The polaron binding energy varies from 224 to 243 meV with increased Ga do**.
△ Less
Submitted 16 June, 2006;
originally announced June 2006.
-
Granular Matter: a wonderful world of clusters in far-from-equilibrium systems
Authors:
M. Ausloos,
R. Lambiotte,
K. Trojan,
Z. Koza,
M. Pekala
Abstract:
In this paper, we recall various features of non equilibrium granular systems. Clusters with specific properties are found depending on the packing density, going from loose (a granular gas) to sintered (though brittle) polycrystalline materials. The phase space available can be quite different. Unexpected features, with respect to standard or expected ones in classical fluids or solids, are obs…
▽ More
In this paper, we recall various features of non equilibrium granular systems. Clusters with specific properties are found depending on the packing density, going from loose (a granular gas) to sintered (though brittle) polycrystalline materials. The phase space available can be quite different. Unexpected features, with respect to standard or expected ones in classical fluids or solids, are observed, - like slow relaxation processes or anomalous electrical and thermoelectrical transport property dependences. The cases of various pile structures and the interplay between classical phase transitions and self-organized criticality for avalanches are also outlined.
△ Less
Submitted 24 October, 2005; v1 submitted 12 September, 2005;
originally announced September 2005.
-
Unusual thermoelectric behavior of packed crystalline granular metals
Authors:
M. Ausloos,
M. Pekala,
J. Latuch,
J. Mucha,
Ph. Vanderbemden,
B. Vertruyen,
R. Cloots
Abstract:
Loosely packed granular materials are intensively studied nowadays. Electrical and thermal transport properties should reflect the granular structure as well as intrinsic properties. We have compacted crystalline $CaAl$ based metallic grains and studied the electrical resistivity and the thermoelectric power as a function of temperature ($T$) from 15 to 300K. Both properties show three regimes a…
▽ More
Loosely packed granular materials are intensively studied nowadays. Electrical and thermal transport properties should reflect the granular structure as well as intrinsic properties. We have compacted crystalline $CaAl$ based metallic grains and studied the electrical resistivity and the thermoelectric power as a function of temperature ($T$) from 15 to 300K. Both properties show three regimes as a function of temperature. It should be pointed out : (i) The electrical resistivity continuously decreases between 15 and 235 K (ii) with various dependences, e.g. $\simeq$ $T^{-3/4}$ at low $T$, while (iii) the thermoelectric power (TEP) is positive, (iv) shows a bump near 60K, and (v) presents a rather unusual square root of temperature dependence at low temperature. It is argued that these three regimes indicate a competition between geometric and thermal processes, - for which a theory seems to be missing in the case of TEP. The microchemical analysis results are also reported indicating a complex microstructure inherent to the phase diagram peritectic intricacies of this binary alloy.
△ Less
Submitted 13 December, 2004;
originally announced December 2004.
-
Hidden Superconductivity in Solid Solutions Sr(1-x)La(x)PbO3 detected by tunneling
Authors:
V. A. Drozd,
T. Ekino,
A. M. Gabovich,
M. Pekala,
R. A. Ribeiro
Abstract:
Solid solutions of Sr1-xLaxPbO3 (x = 0, 0.05, 0.10, 0.15 and 0.20) were synthesized with the help of high pressure. Temperature (T) dependences of the resistivity r and differential thermoelectric power S were measured. All samples demonstrate an increase of r for low T and no signs of a bulk superconducting transition. The sign of S(T) is positive for x = 0.05 and 0.10, whereas S(T) changes sig…
▽ More
Solid solutions of Sr1-xLaxPbO3 (x = 0, 0.05, 0.10, 0.15 and 0.20) were synthesized with the help of high pressure. Temperature (T) dependences of the resistivity r and differential thermoelectric power S were measured. All samples demonstrate an increase of r for low T and no signs of a bulk superconducting transition. The sign of S(T) is positive for x = 0.05 and 0.10, whereas S(T) changes sign from positive to negative for x = 0.15. Tunnel measurements were carried out using the break-junction technique. The sample SrPbO3 reveals a correlation dielectric gap Sigma = 1.5 ~ 2 eV. On the other hand, Sr(0.9)La(0.1)PbO3 shows a Josephson tunnel current and a superconducting gap of approximately 10 meV. This is the first such observation for the studied family of oxides. The results may help to resolve the existing controversy over superconductivity in the conglomerates Cu24Pb2Sr2Ag2Ox.
△ Less
Submitted 6 September, 2004;
originally announced September 2004.
-
Magneto - transport characterization of Dy123 monodomain superconductors
Authors:
M. Pekala,
J. Mucha,
Ph. Vanderbemden,
R. Cloots,
M. Ausloos
Abstract:
The following report considers textured materials of the DyBa2Cu3O7 type seeded with a Nd123 seed as initiator. They are grown with an excess 20% Dy211 phase on a Dy2O3 substrate. We report chemical characterizations, electrical resistivity, thermoelectric power and thermal conductivity over a broad temperature range as a function of an applied magnetic field up to 6 T. We show that specific fea…
▽ More
The following report considers textured materials of the DyBa2Cu3O7 type seeded with a Nd123 seed as initiator. They are grown with an excess 20% Dy211 phase on a Dy2O3 substrate. We report chemical characterizations, electrical resistivity, thermoelectric power and thermal conductivity over a broad temperature range as a function of an applied magnetic field up to 6 T. We show that specific features appear on the magneto thermal transport properties different in these materials from those found in single crystals and polycrystalline samples. We propose that two vortex regimes can be distinguished in the mixed phase, - due to the intrinsic microstructure. We calculate the viscosity, entropy and figure of merit of the samples.
△ Less
Submitted 17 July, 2004;
originally announced July 2004.
-
Heat capacity of mesoscopically disordered superconductors:implications to MgB$_{2}$
Authors:
A. M. Gabovich,
A. I. Voitenko,
Mai Suan Li,
H. Szymczak,
M. Pekala
Abstract:
Electronic heat capacity was calculated for a mesoscopically disordered s-wave superconductor treated as a spatial ensemble of domains with a continuously varying superconducting properties. The domains are assumed to have sizes larger than the coherence length. Each domain is characterized by a certain critical temperature. The averaging over the superconducting gap distribution leads to quadra…
▽ More
Electronic heat capacity was calculated for a mesoscopically disordered s-wave superconductor treated as a spatial ensemble of domains with a continuously varying superconducting properties. The domains are assumed to have sizes larger than the coherence length. Each domain is characterized by a certain critical temperature. The averaging over the superconducting gap distribution leads to quadratic temperature dependence of the specific heat at low temperatures, whereas its anomaly at the critical temperature is substantially smeared. The results explain well the heat capacity data for MgB$_{2}$, where a multiple-gap structure is observed.
△ Less
Submitted 6 December, 2001;
originally announced December 2001.
