Showing 1–15 of 15 results for author: Loveland, D

Network Design through Graph Neural Networks: Identifying Challenges and Improving Performance

Authors: Donald Loveland, Rajmonda Caceres

Abstract: Graph Neural Network (GNN) research has produced strategies to modify a graph's edges using gradients from a trained GNN, with the goal of network design. However, the factors which govern gradient-based editing are understudied, obscuring why edges are chosen and if edits are grounded in an edge's importance. Thus, we begin by analyzing the gradient computation in previous works, elucidating the… ▽ More Graph Neural Network (GNN) research has produced strategies to modify a graph's edges using gradients from a trained GNN, with the goal of network design. However, the factors which govern gradient-based editing are understudied, obscuring why edges are chosen and if edits are grounded in an edge's importance. Thus, we begin by analyzing the gradient computation in previous works, elucidating the factors that influence edits and highlighting the potential over-reliance on structural properties. Specifically, we find that edges can achieve high gradients due to structural biases, rather than importance, leading to erroneous edits when the factors are unrelated to the design task. To improve editing, we propose ORE, an iterative editing method that (a) edits the highest scoring edges and (b) re-embeds the edited graph to refresh gradients, leading to less biased edge choices. We empirically study ORE through a set of proposed design tasks, each with an external validation method, demonstrating that ORE improves upon previous methods by up to 50%. △ Less

Submitted 25 October, 2023; originally announced October 2023.

On Performance Discrepancies Across Local Homophily Levels in Graph Neural Networks

Authors: Donald Loveland, Jiong Zhu, Mark Heimann, Benjamin Fish, Michael T. Schaub, Danai Koutra

Abstract: Graph Neural Network (GNN) research has highlighted a relationship between high homophily (i.e., the tendency of nodes of the same class to connect) and strong predictive performance in node classification. However, recent work has found the relationship to be more nuanced, demonstrating that simple GNNs can learn in certain heterophilous settings. To resolve these conflicting findings and align c… ▽ More Graph Neural Network (GNN) research has highlighted a relationship between high homophily (i.e., the tendency of nodes of the same class to connect) and strong predictive performance in node classification. However, recent work has found the relationship to be more nuanced, demonstrating that simple GNNs can learn in certain heterophilous settings. To resolve these conflicting findings and align closer to real-world datasets, we go beyond the assumption of a global graph homophily level and study the performance of GNNs when the local homophily level of a node deviates from the global homophily level. Through theoretical and empirical analysis, we systematically demonstrate how shifts in local homophily can introduce performance degradation, leading to performance discrepancies across local homophily levels. We ground the practical implications of this work through granular analysis on five real-world datasets with varying global homophily levels, demonstrating that (a) GNNs can fail to generalize to test nodes that deviate from the global homophily of a graph, and (b) high local homophily does not necessarily confer high performance for a node. We further show that GNNs designed for globally heterophilous graphs can alleviate performance discrepancy by improving performance across local homophily levels, offering a new perspective on how these GNNs achieve stronger global performance. △ Less

Submitted 20 November, 2023; v1 submitted 8 June, 2023; originally announced June 2023.

Comments: 30 pages

On Graph Neural Network Fairness in the Presence of Heterophilous Neighborhoods

Authors: Donald Loveland, Jiong Zhu, Mark Heimann, Ben Fish, Michael T. Schaub, Danai Koutra

Abstract: We study the task of node classification for graph neural networks (GNNs) and establish a connection between group fairness, as measured by statistical parity and equal opportunity, and local assortativity, i.e., the tendency of linked nodes to have similar attributes. Such assortativity is often induced by homophily, the tendency for nodes of similar properties to connect. Homophily can be common… ▽ More We study the task of node classification for graph neural networks (GNNs) and establish a connection between group fairness, as measured by statistical parity and equal opportunity, and local assortativity, i.e., the tendency of linked nodes to have similar attributes. Such assortativity is often induced by homophily, the tendency for nodes of similar properties to connect. Homophily can be common in social networks where systemic factors have forced individuals into communities which share a sensitive attribute. Through synthetic graphs, we study the interplay between locally occurring homophily and fair predictions, finding that not all node neighborhoods are equal in this respect -- neighborhoods dominated by one category of a sensitive attribute often struggle to obtain fair treatment, especially in the case of diverging local class and sensitive attribute homophily. After determining that a relationship between local homophily and fairness exists, we investigate if the issue of unfairness can be associated to the design of the applied GNN model. We show that by adopting heterophilous GNN designs capable of handling disassortative group labels, group fairness in locally heterophilous neighborhoods can be improved by up to 25% over homophilous designs in real and synthetic datasets. △ Less

Submitted 14 November, 2022; v1 submitted 10 July, 2022; originally announced July 2022.

Comments: 6 pages, KDD 2022 DLG Workshop

Zeroth-Order SciML: Non-intrusive Integration of Scientific Software with Deep Learning

Authors: Ioannis Tsaknakis, Bhavya Kailkhura, Sijia Liu, Donald Loveland, James Diffenderfer, Anna Maria Hiszpanski, Mingyi Hong

Abstract: Using deep learning (DL) to accelerate and/or improve scientific workflows can yield discoveries that are otherwise impossible. Unfortunately, DL models have yielded limited success in complex scientific domains due to large data requirements. In this work, we propose to overcome this issue by integrating the abundance of scientific knowledge sources (SKS) with the DL training process. Existing kn… ▽ More Using deep learning (DL) to accelerate and/or improve scientific workflows can yield discoveries that are otherwise impossible. Unfortunately, DL models have yielded limited success in complex scientific domains due to large data requirements. In this work, we propose to overcome this issue by integrating the abundance of scientific knowledge sources (SKS) with the DL training process. Existing knowledge integration approaches are limited to using differentiable knowledge source to be compatible with first-order DL training paradigm. In contrast, our proposed approach treats knowledge source as a black-box in turn allowing to integrate virtually any knowledge source. To enable an end-to-end training of SKS-coupled-DL, we propose to use zeroth-order optimization (ZOO) based gradient-free training schemes, which is non-intrusive, i.e., does not require making any changes to the SKS. We evaluate the performance of our ZOO training scheme on two real-world material science applications. We show that proposed scheme is able to effectively integrate scientific knowledge with DL training and is able to outperform purely data-driven model for data-limited scientific applications. We also discuss some limitations of the proposed method and mention potentially worthwhile future directions. △ Less

Submitted 4 June, 2022; originally announced June 2022.

The Lick AGN Monitoring Project 2016: Dynamical Modeling of Velocity-Resolved H\b{eta} Lags in Luminous Seyfert Galaxies

Authors: Lizvette Villafaña, Peter R. Williams, Tommaso Treu, Brendon J. Brewer, Aaron J. Barth, Vivian U, Vardha N. Bennert, H. Alexander Vogler, Hengxiao Guo, Misty C. Bentz, Gabriela Canalizo, Alexei V. Filippenko, Elinor Gates, Frederick Hamann, Michael D. Joner, Matthew A. Malkan, Jong-Hak Woo, Bela Abolfathi, L. E. Abramson, Stephen F. Armen, Hyun-** Bae, Thomas Bohn, Benjamin D. Boizelle, Azalee Bostroem, Andrew Brandel , et al. (40 additional authors not shown)

Abstract: We have modeled the velocity-resolved reverberation response of the H\b{eta} broad emission line in nine Seyfert 1 galaxies from the Lick Active Galactic Nucleus (AGN) Monitioring Project 2016 sample, drawing inferences on the geometry and structure of the low-ionization broad-line region (BLR) and the mass of the central supermassive black hole. Overall, we find that the H\b{eta} BLR is generally… ▽ More We have modeled the velocity-resolved reverberation response of the H\b{eta} broad emission line in nine Seyfert 1 galaxies from the Lick Active Galactic Nucleus (AGN) Monitioring Project 2016 sample, drawing inferences on the geometry and structure of the low-ionization broad-line region (BLR) and the mass of the central supermassive black hole. Overall, we find that the H\b{eta} BLR is generally a thick disk viewed at low to moderate inclination angles. We combine our sample with prior studies and investigate line-profile shape dependence, such as log10(FWHM/σ), on BLR structure and kinematics and search for any BLR luminosity-dependent trends. We find marginal evidence for an anticorrelation between the profile shape of the broad H\b{eta} emission line and the Eddington ratio, when using the root-mean-square spectrum. However, we do not find any luminosity-dependent trends, and conclude that AGNs have diverse BLR structure and kinematics, consistent with the hypothesis of transient AGN/BLR conditions rather than systematic trends. △ Less

Submitted 28 March, 2022; originally announced March 2022.

FairEdit: Preserving Fairness in Graph Neural Networks through Greedy Graph Editing

Authors: Donald Loveland, Jiayi Pan, Aaresh Farrokh Bhathena, Yiyang Lu

Abstract: Graph Neural Networks (GNNs) have proven to excel in predictive modeling tasks where the underlying data is a graph. However, as GNNs are extensively used in human-centered applications, the issue of fairness has arisen. While edge deletion is a common method used to promote fairness in GNNs, it fails to consider when data is inherently missing fair connections. In this work we consider the unexpl… ▽ More Graph Neural Networks (GNNs) have proven to excel in predictive modeling tasks where the underlying data is a graph. However, as GNNs are extensively used in human-centered applications, the issue of fairness has arisen. While edge deletion is a common method used to promote fairness in GNNs, it fails to consider when data is inherently missing fair connections. In this work we consider the unexplored method of edge addition, accompanied by deletion, to promote fairness. We propose two model-agnostic algorithms to perform edge editing: a brute force approach and a continuous approximation approach, FairEdit. FairEdit performs efficient edge editing by leveraging gradient information of a fairness loss to find edges that improve fairness. We find that FairEdit outperforms standard training for many data sets and GNN methods, while performing comparably to many state-of-the-art methods, demonstrating FairEdit's ability to improve fairness across many domains and models. △ Less

Submitted 16 February, 2022; v1 submitted 10 January, 2022; originally announced January 2022.

The Lick AGN Monitoring Project 2016: Velocity-Resolved Hβ Lags in Luminous Seyfert Galaxies

Authors: Vivian U, Aaron J. Barth, H. Alexander Vogler, Hengxiao Guo, Tommaso Treu, Vardha N. Bennert, Gabriela Canalizo, Alexei V. Filippenko, Elinor Gates, Frederick Hamann, Michael D. Joner, Matthew A. Malkan, Anna Pancoast, Peter R. Williams, Jong-Hak Woo, Bela Abolfathi, L. E. Abramson, Stephen F. Armen, Hyun-** Bae, Thomas Bohn, Benjamin D. Boizelle, Azalee Bostroem, Andrew Brandel, Thomas G. Brink, Sanyum Channa , et al. (39 additional authors not shown)

Abstract: We carried out spectroscopic monitoring of 21 low-redshift Seyfert 1 galaxies using the Kast double spectrograph on the 3-m Shane telescope at Lick Observatory from April 2016 to May 2017. Targeting active galactic nuclei (AGN) with luminosities of λLλ (5100 Å) = 10^44 erg/s and predicted Hβ lags of 20-30 days or black hole masses of 10^7-10^8.5 Msun, our campaign probes luminosity-dependent trend… ▽ More We carried out spectroscopic monitoring of 21 low-redshift Seyfert 1 galaxies using the Kast double spectrograph on the 3-m Shane telescope at Lick Observatory from April 2016 to May 2017. Targeting active galactic nuclei (AGN) with luminosities of λLλ (5100 Å) = 10^44 erg/s and predicted Hβ lags of 20-30 days or black hole masses of 10^7-10^8.5 Msun, our campaign probes luminosity-dependent trends in broad-line region (BLR) structure and dynamics as well as to improve calibrations for single-epoch estimates of quasar black hole masses. Here we present the first results from the campaign, including Hβ emission-line light curves, integrated Hβ lag times (8-30 days) measured against V-band continuum light curves, velocity-resolved reverberation lags, line widths of the broad Hβ components, and virial black hole mass estimates (10^7.1-10^8.1 Msun). Our results add significantly to the number of existing velocity-resolved lag measurements and reveal a diversity of BLR gas kinematics at moderately high AGN luminosities. AGN continuum luminosity appears not to be correlated with the type of kinematics that its BLR gas may exhibit. Follow-up direct modeling of this dataset will elucidate the detailed kinematics and provide robust dynamical black hole masses for several objects in this sample. △ Less

Submitted 29 November, 2021; originally announced November 2021.

Comments: 34 pages, 14 figures, accepted for publication in ApJ

Reliable Graph Neural Network Explanations Through Adversarial Training

Authors: Donald Loveland, Shusen Liu, Bhavya Kailkhura, Anna Hiszpanski, Yong Han

Abstract: Graph neural network (GNN) explanations have largely been facilitated through post-hoc introspection. While this has been deemed successful, many post-hoc explanation methods have been shown to fail in capturing a model's learned representation. Due to this problem, it is worthwhile to consider how one might train a model so that it is more amenable to post-hoc analysis. Given the success of adver… ▽ More Graph neural network (GNN) explanations have largely been facilitated through post-hoc introspection. While this has been deemed successful, many post-hoc explanation methods have been shown to fail in capturing a model's learned representation. Due to this problem, it is worthwhile to consider how one might train a model so that it is more amenable to post-hoc analysis. Given the success of adversarial training in the computer vision domain to train models with more reliable representations, we propose a similar training paradigm for GNNs and analyze the respective impact on a model's explanations. In instances without ground truth labels, we also determine how well an explanation method is utilizing a model's learned representation through a new metric and demonstrate adversarial training can help better extract domain-relevant insights in chemistry. △ Less

Submitted 25 June, 2021; originally announced June 2021.

Comments: 4 pages, 3 figures, ICML Workshop on Theoretic Foundation, Criticism, and Application Trend of Explainable AI

How does Heterophily Impact the Robustness of Graph Neural Networks? Theoretical Connections and Practical Implications

Authors: Jiong Zhu, Junchen **, Donald Loveland, Michael T. Schaub, Danai Koutra

Abstract: We bridge two research directions on graph neural networks (GNNs), by formalizing the relation between heterophily of node labels (i.e., connected nodes tend to have dissimilar labels) and the robustness of GNNs to adversarial attacks. Our theoretical and empirical analyses show that for homophilous graph data, impactful structural attacks always lead to reduced homophily, while for heterophilous… ▽ More We bridge two research directions on graph neural networks (GNNs), by formalizing the relation between heterophily of node labels (i.e., connected nodes tend to have dissimilar labels) and the robustness of GNNs to adversarial attacks. Our theoretical and empirical analyses show that for homophilous graph data, impactful structural attacks always lead to reduced homophily, while for heterophilous graph data the change in the homophily level depends on the node degrees. These insights have practical implications for defending against attacks on real-world graphs: we deduce that separate aggregators for ego- and neighbor-embeddings, a design principle which has been identified to significantly improve prediction for heterophilous graph data, can also offer increased robustness to GNNs. Our comprehensive experiments show that GNNs merely adopting this design achieve improved empirical and certifiable robustness compared to the best-performing unvaccinated model. Additionally, combining this design with explicit defense mechanisms against adversarial attacks leads to an improved robustness with up to 18.33% performance increase under attacks compared to the best-performing vaccinated model. △ Less

Submitted 22 July, 2022; v1 submitted 14 June, 2021; originally announced June 2021.

Comments: KDD 2022 camera ready version + full appendix; 20 pages, 2 figures

Explainable Deep Learning for Uncovering Actionable Scientific Insights for Materials Discovery and Design

Authors: Shusen Liu, Bhavya Kailkhura, Jize Zhang, Anna M. Hiszpanski, Emily Robertson, Donald Loveland, T. Yong-** Han

Abstract: The scientific community has been increasingly interested in harnessing the power of deep learning to solve various domain challenges. However, despite the effectiveness in building predictive models, fundamental challenges exist in extracting actionable knowledge from deep neural networks due to their opaque nature. In this work, we propose techniques for exploring the behavior of deep learning m… ▽ More The scientific community has been increasingly interested in harnessing the power of deep learning to solve various domain challenges. However, despite the effectiveness in building predictive models, fundamental challenges exist in extracting actionable knowledge from deep neural networks due to their opaque nature. In this work, we propose techniques for exploring the behavior of deep learning models by injecting domain-specific actionable attributes as tunable "knobs" in the analysis pipeline. By incorporating the domain knowledge in a generative modeling framework, we are not only able to better understand the behavior of these black-box models, but also provide scientists with actionable insights that can potentially lead to fundamental discoveries. △ Less

Submitted 16 July, 2020; originally announced July 2020.

Report number: LLNL-JRNL-811201

Actionable Attribution Maps for Scientific Machine Learning

Authors: Shusen Liu, Bhavya Kailkhura, Jize Zhang, Anna M. Hiszpanski, Emily Robertson, Donald Loveland, T. Yong-** Han

Abstract: The scientific community has been increasingly interested in harnessing the power of deep learning to solve various domain challenges. However, despite the effectiveness in building predictive models, fundamental challenges exist in extracting actionable knowledge from the deep neural network due to their opaque nature. In this work, we propose techniques for exploring the behavior of deep learnin… ▽ More The scientific community has been increasingly interested in harnessing the power of deep learning to solve various domain challenges. However, despite the effectiveness in building predictive models, fundamental challenges exist in extracting actionable knowledge from the deep neural network due to their opaque nature. In this work, we propose techniques for exploring the behavior of deep learning models by injecting domain-specific actionable concepts as tunable ``knobs'' in the analysis pipeline. By incorporating the domain knowledge with generative modeling, we are not only able to better understand the behavior of these black-box models, but also provide scientists with actionable insights that can potentially lead to fundamental discoveries. △ Less

Submitted 30 June, 2020; originally announced June 2020.

Generative Counterfactual Introspection for Explainable Deep Learning

Authors: Shusen Liu, Bhavya Kailkhura, Donald Loveland, Yong Han

Abstract: In this work, we propose an introspection technique for deep neural networks that relies on a generative model to instigate salient editing of the input image for model interpretation. Such modification provides the fundamental interventional operation that allows us to obtain answers to counterfactual inquiries, i.e., what meaningful change can be made to the input image in order to alter the pre… ▽ More In this work, we propose an introspection technique for deep neural networks that relies on a generative model to instigate salient editing of the input image for model interpretation. Such modification provides the fundamental interventional operation that allows us to obtain answers to counterfactual inquiries, i.e., what meaningful change can be made to the input image in order to alter the prediction. We demonstrate how to reveal interesting properties of the given classifiers by utilizing the proposed introspection approach on both the MNIST and the CelebA dataset. △ Less

Submitted 6 July, 2019; originally announced July 2019.

Predicting Compressive Strength of Consolidated Molecular Solids Using Computer Vision and Deep Learning

Authors: Brian Gallagher, Matthew Rever, Donald Loveland, T. Nathan Mundhenk, Brock Beauchamp, Emily Robertson, Golam G. Jaman, Anna M. Hiszpanski, T. Yong-** Han

Abstract: We explore the application of computer vision and machine learning (ML) techniques to predict material properties (e.g. compressive strength) based on SEM images. We show that it's possible to train ML models to predict materials performance based on SEM images alone, demonstrating this capability on the real-world problem of predicting uniaxially compressed peak stress of consolidated molecular s… ▽ More We explore the application of computer vision and machine learning (ML) techniques to predict material properties (e.g. compressive strength) based on SEM images. We show that it's possible to train ML models to predict materials performance based on SEM images alone, demonstrating this capability on the real-world problem of predicting uniaxially compressed peak stress of consolidated molecular solids samples. Our image-based ML approach reduces mean absolute percent error (MAPE) by an average of 24% over baselines representative of the current state-of-the-practice (i.e., domain-expert's analysis and correlation). We compared two complementary approaches to this problem: (1) a traditional ML approach, random forest (RF), using state-of-the-art computer vision features and (2) an end-to-end deep learning (DL) approach, where features are learned automatically from raw images. We demonstrate the complementarity of these approaches, showing that RF performs best in the "small data" regime in which many real-world scientific applications reside (up to 24% lower RMSE than DL), whereas DL outpaces RF in the "big data" regime, where abundant training samples are available (up to 24% lower RMSE than RF). Finally, we demonstrate that models trained using machine learning techniques are capable of discovering and utilizing informative crystal attributes previously underutilized by domain experts. △ Less

Submitted 27 February, 2020; v1 submitted 5 June, 2019; originally announced June 2019.

Studying the [OIII]$λ$5007A emission-line width in a sample of $\sim$80 local active galaxies: A surrogate for $σ_{\star}$?

Authors: Vardha N. Bennert, Donald Loveland, Edward Donohue, Maren Cosens, Sean Lewis, S. Komossa, Tommaso Treu, Matthew A. Malkan, Nathan Milgram, Kelsi Flatland, Matthew W. Auger, Daeseong Park, Mariana S. Lazarova

Abstract: For a sample of $\sim$80 local ($0.02 \leq z \leq 0.1$) Seyfert-1 galaxies with high-quality long-slit Keck spectra and spatially-resolved stellar-velocity dispersion ($σ_{\star}$) measurements, we study the profile of the [OIII]$λ$5007A emission line to test the validity of using its width as a surrogate for $σ_{\star}$. Such an approach has often been used in the literature, since it is difficul… ▽ More For a sample of $\sim$80 local ($0.02 \leq z \leq 0.1$) Seyfert-1 galaxies with high-quality long-slit Keck spectra and spatially-resolved stellar-velocity dispersion ($σ_{\star}$) measurements, we study the profile of the [OIII]$λ$5007A emission line to test the validity of using its width as a surrogate for $σ_{\star}$. Such an approach has often been used in the literature, since it is difficult to measure $σ_{\star}$ for type-1 active galactic nuclei (AGNs) due to the AGN continuum outshining the stellar-absorption lines. Fitting the [OIII] line with a single Gaussian or Gauss-Hermite polynomials overestimates $σ_{\star}$ by 50-100%. When line asymmetries from non-gravitational gas motion are excluded in a double Gaussian fit, the average ratio between the core [OIII] width ($σ_{\rm {[OIII],D}}$) and $σ_{\star}$ is $\sim$1, but with individual data points off by up to a factor of two. The resulting black-hole-mass-$σ_{\rm {[OIII],D}}$ relation scatters around that of quiescent galaxies and reverberation-mapped AGNs. However, a direct comparison between $σ_{\star}$ and $σ_{\rm {[OIII],D}}$ shows no close correlation, only that both quantities have the same range, average and standard deviation, probably because they feel the same gravitational potential. The large scatter is likely due to the fact that line profiles are a luminosity-weighted average, dependent on the light distribution and underlying kinematic field. Within the range probed by our sample (80-260 km s$^{-1}$), our results strongly caution against the use of [OIII] width as a surrogate for $σ_{\star}$ on an individual basis. Even though our sample consists of radio-quiet AGNs, FIRST radio-detected objects have, on average, a $\sim$10% larger [OIII] core width. △ Less

Submitted 14 August, 2018; originally announced August 2018.

Comments: 15 pages, 10 figures, 6 tables. Accepted for publication in the Monthly Notices of the Royal Astronomical Society

Neutron-rich rare isotope production from projectile fission of heavy beams in the energy range of 20 MeV/nucleon

Authors: N. Vonta, G. A. Souliotis, W. D. Loveland, Y. K. Kwon, K. Tshoo, S. C. Jeong, M. Veselsky, A. Bonasera, A. Botvina

Abstract: We investigate the possibilities of producing neutron-rich nuclides in projectile fission of heavy beams in the energy range of 20 MeV/nucleon expected from low-energy facilities. We report our efforts to theoretically describe the reaction mechanism of projectile fission following a multinucleon transfer collision at this energy range. Our calculations are mainly based on a two-step approach: the… ▽ More We investigate the possibilities of producing neutron-rich nuclides in projectile fission of heavy beams in the energy range of 20 MeV/nucleon expected from low-energy facilities. We report our efforts to theoretically describe the reaction mechanism of projectile fission following a multinucleon transfer collision at this energy range. Our calculations are mainly based on a two-step approach: the dynamical stage of the collision is described with either the phenomenological Deep-Inelastic Transfer model (DIT), or with the microscopic Constrained Molecular Dynamics model (CoMD). The deexcitation/fission of the hot heavy projectile fragments is performed with the Statistical Mul- tifragmentation Model (SMM). We compared our model calculations with our previous experimental projectile-fission data of 238U (20 MeV/nucleon)+208Pb and 197Au (20 MeV/nucleon)+197Au and found an overall reasonable agreement. Our study suggests that projectile fission following periph- eral heavy-ion collisions at this energy range offers an effective route to access very neutron-rich rare isotopes toward and beyond the astrophysical r-process path. △ Less

Submitted 24 August, 2016; originally announced August 2016.

Journal ref: Phys. Rev. C 94, 064611 (2016)