Showing 1–7 of 7 results for author: Toscano, D

A comprehensive and FAIR comparison between MLP and KAN representations for differential equations and operator networks

Authors: Khemraj Shukla, Juan Diego Toscano, Zhicheng Wang, Zongren Zou, George Em Karniadakis

Abstract: Kolmogorov-Arnold Networks (KANs) were recently introduced as an alternative representation model to MLP. Herein, we employ KANs to construct physics-informed machine learning models (PIKANs) and deep operator models (DeepOKANs) for solving differential equations for forward and inverse problems. In particular, we compare them with physics-informed neural networks (PINNs) and deep operator network… ▽ More Kolmogorov-Arnold Networks (KANs) were recently introduced as an alternative representation model to MLP. Herein, we employ KANs to construct physics-informed machine learning models (PIKANs) and deep operator models (DeepOKANs) for solving differential equations for forward and inverse problems. In particular, we compare them with physics-informed neural networks (PINNs) and deep operator networks (DeepONets), which are based on the standard MLP representation. We find that although the original KANs based on the B-splines parameterization lack accuracy and efficiency, modified versions based on low-order orthogonal polynomials have comparable performance to PINNs and DeepONet although they still lack robustness as they may diverge for different random seeds or higher order orthogonal polynomials. We visualize their corresponding loss landscapes and analyze their learning dynamics using information bottleneck theory. Our study follows the FAIR principles so that other researchers can use our benchmarks to further advance this emerging topic. △ Less

Submitted 5 June, 2024; originally announced June 2024.

Learning in PINNs: Phase transition, total diffusion, and generalization

Authors: Sokratis J. Anagnostopoulos, Juan Diego Toscano, Nikolaos Stergiopulos, George Em Karniadakis

Abstract: We investigate the learning dynamics of fully-connected neural networks through the lens of gradient signal-to-noise ratio (SNR), examining the behavior of first-order optimizers like Adam in non-convex objectives. By interpreting the drift/diffusion phases in the information bottleneck theory, focusing on gradient homogeneity, we identify a third phase termed ``total diffusion", characterized by… ▽ More We investigate the learning dynamics of fully-connected neural networks through the lens of gradient signal-to-noise ratio (SNR), examining the behavior of first-order optimizers like Adam in non-convex objectives. By interpreting the drift/diffusion phases in the information bottleneck theory, focusing on gradient homogeneity, we identify a third phase termed ``total diffusion", characterized by equilibrium in the learning rates and homogeneous gradients. This phase is marked by an abrupt SNR increase, uniform residuals across the sample space and the most rapid training convergence. We propose a residual-based re-weighting scheme to accelerate this diffusion in quadratic loss functions, enhancing generalization. We also explore the information compression phenomenon, pinpointing a significant saturation-induced compression of activations at the total diffusion phase, with deeper layers experiencing negligible information loss. Supported by experimental data on physics-informed neural networks (PINNs), which underscore the importance of gradient homogeneity due to their PDE-based sample inter-dependence, our findings suggest that recognizing phase transitions could refine ML optimization strategies for improved generalization. △ Less

Submitted 27 March, 2024; originally announced March 2024.

Residual-based attention and connection to information bottleneck theory in PINNs

Authors: Sokratis J. Anagnostopoulos, Juan Diego Toscano, Nikolaos Stergiopulos, George Em Karniadakis

Abstract: Driven by the need for more efficient and seamless integration of physical models and data, physics-informed neural networks (PINNs) have seen a surge of interest in recent years. However, ensuring the reliability of their convergence and accuracy remains a challenge. In this work, we propose an efficient, gradient-less weighting scheme for PINNs, that accelerates the convergence of dynamic or sta… ▽ More Driven by the need for more efficient and seamless integration of physical models and data, physics-informed neural networks (PINNs) have seen a surge of interest in recent years. However, ensuring the reliability of their convergence and accuracy remains a challenge. In this work, we propose an efficient, gradient-less weighting scheme for PINNs, that accelerates the convergence of dynamic or static systems. This simple yet effective attention mechanism is a function of the evolving cumulative residuals and aims to make the optimizer aware of problematic regions at no extra computational cost or adversarial learning. We illustrate that this general method consistently achieves a relative $L^{2}$ error of the order of $10^{-5}$ using standard optimizers on typical benchmark cases of the literature. Furthermore, by investigating the evolution of weights during training, we identify two distinct learning phases reminiscent of the fitting and diffusion phases proposed by the information bottleneck (IB) theory. Subsequent gradient analysis supports this hypothesis by aligning the transition from high to low signal-to-noise ratio (SNR) with the transition from fitting to diffusion regimes of the adopted weights. This novel correlation between PINNs and IB theory could open future possibilities for understanding the underlying mechanisms behind the training and stability of PINNs and, more broadly, of neural operators. △ Less

Submitted 1 July, 2023; originally announced July 2023.

Traps for pinning and scattering of antiferromagnetic skyrmions via magnetic properties engineering

Authors: D. Toscano, I. A. Santece, R. C. O. Guedes, H. S. Assis, A. L. S. Miranda, C. I. L. de Araujo, F. Sato, P. Z. Coura, S. A. Leonel

Abstract: Micromagnetic simulations have been performed to investigate the controllability of the skyrmion position in antiferromagnetic nanotracks with their magnetic properties modified spatially. In this study we have modeled magnetic defects as local variations on the material parameters, such as the exchange stiffness, saturation magnetization, perpendicular magnetocrystalline anisotropy and Dzyaloshin… ▽ More Micromagnetic simulations have been performed to investigate the controllability of the skyrmion position in antiferromagnetic nanotracks with their magnetic properties modified spatially. In this study we have modeled magnetic defects as local variations on the material parameters, such as the exchange stiffness, saturation magnetization, perpendicular magnetocrystalline anisotropy and Dzyaloshinskii-Moriya constant. Thus, we have observed not only pinning (potential well) but also scattering (potential barrier) of antiferromagnetic skyrmions, when adjusting either a local increase or a local reduction for each material parameter. In order to control of the skyrmion motion it is very important to impose certain positions along the nanotrack where the skyrmion can stop. Magnetic defects incorporated intentionally in antiferromagnetic racetracks can be useful for such purpose. In order to provide guidelines for experimental studies, we vary both material parameters and size of the modified region. The found results show that the efficiency of skyrmion trap depends on a suitable combination of magnetic defect parameters. Furthermore, we discuss the reason why skyrmions are either attracted or repelled by a region magnetically modified. △ Less

Submitted 16 May, 2020; v1 submitted 27 February, 2020; originally announced March 2020.

Comments: arXiv admin note: substantial text overlap with arXiv:1810.03754

Journal ref: Journal of Applied Physics 127, 193902 (2020)

Suppression of the skyrmion Hall effect in planar nanomagnets by the magnetic properties engineering: Skyrmion transport on nanotracks with magnetic strips

Authors: D. Toscano, J. P. A. Mendonça, A. L. S. Miranda, C. I. L. de Araujo, F. Sato, P. Z. Coura, S. A. Leonel

Abstract: Micromagnetic simulations have been performed to investigate the suppression of the skyrmion Hall effect in nanotracks with their magnetic properties strategically modified. In particular, we study two categories of magnetically modified nanotracks. One of them, repulsive edges have been inserted in the nanotrack and, in the other, an attractive strip has been placed exactly on the longest axis of… ▽ More Micromagnetic simulations have been performed to investigate the suppression of the skyrmion Hall effect in nanotracks with their magnetic properties strategically modified. In particular, we study two categories of magnetically modified nanotracks. One of them, repulsive edges have been inserted in the nanotrack and, in the other, an attractive strip has been placed exactly on the longest axis of the nanotrack. Attractive and repulsive interactions can be generated from the engineering of magnetic properties. For instance, it is known that the skyrmion can be attracted to a region where the exchange stiffness constant is decreased. On the other hand, the skyrmion can be repelled from a region characterized by a local increase in the exchange stiffness constant. In order to provide a background for experimental studies, we vary not only the magnetic material parameters (exchange stiffness, perpendicular magnetocrystalline anisotropy and the Dzyaloshinskii-Moriya constant) but also the width of the region magnetically modified, containing either a local reduction or a local increase for each one of these magnetic properties. In the numerical simulations, the skyrmion motion was induced by a spin-polarized current and the found results indicate that it is possible to transport skyrmions around the longest axis of the nanotrack. In practice, the skyrmion Hall effect can be completely suppressed in magnetic nanotracks with strategically modified magnetic properties. Furthermore, we discuss in detail 6 ways to suppress the skyrmion Hall effect by the usage of nanotracks with repulsive edges and nanotracks with an attractive strip. △ Less

Submitted 14 February, 2020; v1 submitted 6 December, 2019; originally announced December 2019.

Report number: MAGMA 166655

Investigation of domain wall pinning by square anti-notches and its applications in three terminals MRAM

Authors: C. I. L. de Araujo, J. C. S. Gomes, D. Toscano, E. L. M. Paixao, P. Z. Coura, F. Sato, D. V. P. Massote, S. A. Leonel

Abstract: In this work we perform investigations of the competition between domain-wall pinning and attraction by anti-notches and finite device borders. The conditions for optimal geometries, which can attain a stable domain-wall pinning, are presented. This allow us the proposition of a three-terminals device based on domain-wall pinning. We obtain, with very small pulses of current applied parallel to th… ▽ More In this work we perform investigations of the competition between domain-wall pinning and attraction by anti-notches and finite device borders. The conditions for optimal geometries, which can attain a stable domain-wall pinning, are presented. This allow us the proposition of a three-terminals device based on domain-wall pinning. We obtain, with very small pulses of current applied parallel to the nanotrack, a fast motion of the domain-wall between anti-notches. In addition to this, a swift stabilization of the pinned domain-wall is observed with a high percentage of orthogonal magnetization, enabling high magnetoresistive signal measurement. Thus, our proposed device is a promising magnetoresistive random access memories with good scalability, duration, and high speed information storage. △ Less

Submitted 11 May, 2019; originally announced May 2019.

Building traps for skyrmions by the incorporation of magnetic defects into nanomagnets: pinning and scattering traps by magnetic properties engineering

Authors: D. Toscano, S. A. Leonel, P. Z. Coura, F. Sato

Abstract: In this work we have used micromagnetic simulations to report four ways to build traps for magnetic skyrmions. Magnetic defects have been modeled as local variations in the material parameters, such as the exchange stiffness, saturation magnetization, magnetocrystalline anisotropy and Dzyaloshinskii-Moriya constant. We observe both pinning (potential well) and scattering (potential barrier) traps… ▽ More In this work we have used micromagnetic simulations to report four ways to build traps for magnetic skyrmions. Magnetic defects have been modeled as local variations in the material parameters, such as the exchange stiffness, saturation magnetization, magnetocrystalline anisotropy and Dzyaloshinskii-Moriya constant. We observe both pinning (potential well) and scattering (potential barrier) traps when tuning either a local increase or a local reduction for each one of these magnetic properties. It is found that the skyrmion-defect aspect ratio is a crucial parameter to build traps for skyrmions. In particular, the efficiency of the trap is compromised if the defect size is smaller than the skyrmion size, because they interact weakly. On the other hand, if the defect size is larger than the skyrmion diameter, the skyrmion-defect interaction becomes evident. Thus, the strength of the skyrmion-defect interaction can be tuned by the modification of the magnetic properties within a region with suitable size. Furthermore, the basic physics behind the mechanisms for pinning and for scattering is discussed. In particular, we discover that skyrmions move towards the magnetic region which tends to maximize its diameter; it enables the magnetic system to minimize its energy. Thus, we are able to explain why skyrmions are either attracted or repelled by a region with modified magnetic properties. Results here presented are of utmost significance for the development and realization of future spintronic devices, in which skyrmions will work as information carriers. △ Less

Submitted 26 October, 2018; v1 submitted 2 October, 2018; originally announced October 2018.

Report number: MAGMA 64994