Heterogeneous Peridynamic Neural Operators: Discover Biotissue Constitutive Law and Microstructure From Digital Image Correlation Measurements

Abstract.

Human tissues are highly organized structures with specific collagen fiber arrangements varying from point to point. The effects of such heterogeneity play an important role for tissue function, and hence it is of critical to discover and understand the distribution of such fiber orientations from experimental measurements, such as the digital image correlation data. To this end, we introduce the heterogeneous peridynamic neural operator (HeteroPNO) approach, for data-driven constitutive modeling of heterogeneous anisotropic materials. The goal is to learn both a nonlocal constitutive law together with the material microstructure, in the form of a heterogeneous fiber orientation field, from loading field-displacement field measurements. To this end, we propose a two-phase learning approach. Firstly, we learn a homogeneous constitutive law in the form of a neural network-based kernel function and a nonlocal bond force, to capture complex homogeneous material responses from data. Then, in the second phase we reinitialize the learnt bond force and the kernel function, and training them together with a fiber orientation field for each material point. Owing to the state-based peridynamic skeleton, our HeteroPNO-learned material models are objective and have the balance of linear and angular momentum guaranteed. Moreover, the effects from heterogeneity and nonlinear constitutive relationship are captured by the kernel function and the bond force respectively, enabling physical interpretability. As a result, our HeteroPNO architecture can learn a constitutive model for a biological tissue with anisotropic heterogeneous response undergoing large deformation regime. The anisotropy and heterogeneity of this tissue stems from collagen fibers with unknown natural orientation, resulting in a location-dependent anisotropy. To demonstrate the applicability of our approach, we apply the heterogeneous PNO in learning the material model and fiber orientation field from digital image correction (DIC) data containing the planar displacement field on the tissue and the reaction forces in a biaxial testing. We find the learnt fiber architecture consistent with observations from polarized spatial frequency domain imaging. Moreover, the framework is capable to provide displacement and stress field predictions for new and unseen loading instances.

Siavash Jafarzadeh¹, Stewart Silling², Lu Zhang¹, Colton Ross³,

Chung-Hao Lee^3,4, S. M. Rakibur Rahman⁵, Shuodao Wang⁵, and Yue Yu${}^{{\href mailto:[email protected]}*1}$

¹Department of Mathematics, Lehigh University, Bethlehem, PA 18015, USA

²Center for Computing Research, Sandia National Laboratories, Albuquerque, NM, USA

³School of Aerospace and Mechanical Engineering, The University of Oklahoma, Norman, OK 73019, USA

⁴Department of Bioengineering, University of California, Riverside, Riverside, CA 92521, USA

⁵School of Mechanical and Aerospace Engineering, Oklahoma State University, Stillwater, OK 74078, USA

1. Introduction

Biological tissues are highly organized structures with specific architectures that enable local functions. Tissues exhibit a unique ability to adapt, remodel, and self-heal in case of disease or injury [1, 2, 3, 4, 5, 6]. Many tissue structures are heterogeneous with specific collagen matrix arrangements varying from point to point within the same tissue, allowing it to respond to diverse local mechanical stimuli. Signaling pathways and chemical transduction processes in the underlying cells detect these stimuli, alter the composition and organization of tissue extracellular matrix (ECM), and orchestrate changes in tissue function in response [7, 8, 9, 10, 11, 12]. Heterogeneity plays an important role in tissue function and resilience.

For the purpose of enabling virtual screening to identify and optimize therapeutic interventions, much effort has been devoted to describing the complex fiber-matrix interactions within tissues. A classical computational approach is based on the development of constitutive relations that are needed for the governing equations. In [13], seminal phenomenological constitutive models were employed for the modeling of soft tissues, including the iris [14], cardiac heart valves [15, 16, 17], arterial vessels [18], and skin [19]. In [20, 7, 21], Lee et al. developed a full collagen–fiber mapped transversely isotropic model and employed it to characterize the in-vivo mechanical response of the mitral valve anterior leaflet through an inverse modeling approach. To model adaptations of the fiber architecture under stress, an ensemble fiber stress–strain relationship was proposed in [22, 23] and employed to study fiber reorientation and fiber recruitment observed in experiments. In all of these studies, a strain energy density function is predefined with a specific functional form when modeling the mechanical responses from experimental measurements. Within this predefined material model, the material parameters are calibrated through an inverse method or analytical stress–strain fitting.

Using such a predefined model creates two challenges that can affect its reliability. Firstly, the descriptive power of the models is often restricted to certain deformation modes and strain ranges, which can restrict its predictivity and generalizability [20, 24, 25]. For some tissues such as skin [26, 27], there is not a definitive material model available in the literature. Secondly, many of these models are developed for homogeneous materials, and hence a full description that accounts for the microstructure is still lacking. In reality, the mechanical properties and microstructure of biotissues vary not only for different tissue types, but also vary with position and can be different for different patients. As a result, there is a need for a constitutive model to capture the mechanical responses for the particular microstructure within a given sample.

To help overcome these limitations, data-driven computing has been considered in recent years as an alternative. In the context of biological tissue modeling [28, 24, 29, 30], data-driven constitutive laws directly integrate material identification with the modeling procedures, and hence do not require a predefined constitutive model form. In [31], the constitutive law for soft tissue damage was constructed by solving the system of linear equations consisting of coefficients of shape functions, rather than nonlinear fitting to a predefined model. A local convexity data-driven (LCDD) computational framework was developed in [24, 29]. This framework couples manifold learning with nonlinear elasticity and was applied to model the stress–strain response of the porcine mitral (heart) valve posterior leaflet. In [30], a neural network was developed to infer the relationship between the isochoric strain invariants and the value of strain energy. This was applied to learn the mechanical behavior of porcine and murine skin from biaxial testing data. Despite these advances, data-driven constitutive laws on soft tissue modeling have mostly focused on the identification of stress–strain and energy–strain relationships for a homogenized material model, and are thus unable to capture the effects of heterogeneity. Because of the importance of heterogeneity in tissue function and failure [32, 33], the assumption of homogeneity seriously restricts the capabilities of a constitutive model.

In an alternative approach, there has been significant progress in the development of deep neural networks (NNs) for heterogeneous material modeling [34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45]. Among these works, we focus on the neural operator learning approach [40, 41, 42, 43, 44, 45, 46, 47], which learns the maps between the function spaces. In comparison with classical NNs, the most notable advantage of neural operators is their generalizability to different input instances. As a result, measurements with different resolutions can be integrated to train the same model [48], and the trained model can be used to solve for new instances on a different grid [49]. Moreover, the neural operator is provably a universal approximator for function-to-function map**s [50, 51]. Therefore, it has the capability to capture both complex material responses and the heterogeneity within data. All of these properties make the neural operator particularly promising for learning complex material responses without predefined constitutive models and direct measurements of the microstructure. In the context of continuum mechanics, most of the current neural operators have assumed that the underlying mathematical description uses PDEs. The neural operator learns a surrogate map** from the loading field to the material response field, which can be seen as learning the solution operator of a hidden PDE. In [52, 53, 54], neural operators have been successfully applied to modeling the physical response of homogeneous materials. In [43, 44, 45, 55], neural operators were used as a solution surrogate for Darcy flow in a heterogeneous porous medium with a known microstructure field. In [40], an implicit neural operator architecture, namely the implicit Fourier neural operator (IFNO), was proposed to model material responses without using any predefined constitutive models or microstructure measurements. IFNOs were then applied to learn the tissue responses from digital image correlation (DIC) measurements. It was found that this neural operator approach is superior to other methods at capturing the heterogeneous features [33].

Despite these advances in neural solution operators, this approach has three limitations. Firstly, it is challenging to incorporate fundamental physical laws into solution operators. As a result, the solution operator needs to learn all physical laws from data, and therefore it cannot always guarantee the physical consistency of fundamental laws such as linear and angular momentum conservation and Galilean and frame invariances. Secondly, since the solution operator varies under different domain shapes and boundary conditions, the neural operators are not generalizable to different domain geometries and loading scenarios. This means, for example, that an operator learned on square-domain samples cannot be immediately applied to predict the material deformation on a circular domain. Last but not least, with solution operators the information about the constitutive law and microstructure is captured implicitly via neural network parameters. Therefore, the relation between the constitutive law and the microstructure is complex and largely hidden, making it challenging to interpret the learnt model. To overcome the first two challenges, in [56] we proposed the peridynamic neural operator model (PNO), a homogeneous nonlocal constitutive law from data. This neural operator provides a forward model in the form of state-based peridynamics, with Galilean invariance, frame invariance, and momentum balance laws guaranteed. The model was validated on a DIC displacement tracking dataset, and it was compared to baseline models that use predefined constitutive laws. Although it assumed homogeneity, this PNO achieved improved accuracy by learning the constitutive law from data. It also showed generalizability to different domain configurations, external loadings, and discretizations.

In the present work, we go one step further and develop a data-driven constitutive law for heterogeneous materials. We propose the heterogeneous peridynamic neural operator (HeteroPNO) and apply it to learn the material model together with fiber orientation field from DIC measurements on a representative tricuspid valve anterior leaflet (TVAL) specimen from a porcine heart. We design HeteroPNO such that the fiber orientation is captured through an anisotropic nonlocal kernel. The material nonlinearity is represented through a nonlinear dependence of the internal forces on the deformation. As a result, the effects of the material heterogeneity and the constitutive law are disentangled. The position dependence of the collagen fiber orientation can be captured by rotating the kernel to align its principal direction with the fiber. Based on this architecture, we propose a two-phase learning algorithm. Firstly, we learn a homogeneous PNO model to capture the kernel form and the constitutive law. Then, we fine-tune the model by learning a fiber orientation field as a function of position. To evaluate the performance of the HeteroPNO, we validate the discovered collagen fiber architecture by comparing it with measurements from polarized spatial frequency domain imaging (pSFDI)-biaxial testing system [57]. We assess the predictability of our model on predicting displacement and stress fields under unseen scenarios. To the best of our knowledge, the present work is the first time that neural operator learning approaches have been applied to discover both the constitutive law and microstructure from data.

The remainder of this paper is organized as follows. In Section 2, we introduce as background the peridynamic nonlocal mechanical theory, the nonlocal neural operators, and the homogeneous peridynamic neural operator (PNO). Then, our proposed architecture, which is based on PNO and its incorporation of heterogeneity through a pointwise fiber orientation field, is introduced in Section 3, followed by the corresponding two-phase machine learning algorithm to discover both the model and microstructure. To verify the performance of our model, in Section 4 we demonstrate the effectiveness of the learning technique on a synthetic dataset describing the deformation of a hyperelastic and anisotropic fiber-reinforced material. This demonstration shows the capability of the method in learning the heterogeneous fiber orientation field and reproducing displacement and stress fields consistent with ground-truth solutions. In Section 5, we combine the method with DIC experimental measurements on a representative TVAL specimen. For comparison with alternative methods, we compare the results of the HeteroPNO with modeling results using a fitted Fung-type homogeneous constitutive model. Finally, we provide a summary of our results and concluding remarks in Section 6.

2. Background

In this section, we briefly introduce the concept of peridynamic theory, nonlocal neural operators and our base model, the peridynamic neural operator for homogenized materials [56]. Throughout this paper, we use unbolded case letters to denote scalars/scalar-valued functions, bold letters to denote vectors/vector-valued functions, underlined unbolded letters for scalar-valued state functions, underlined bold letters for vector-valued state functions, and calligraphic letters for operators. For any vector $\bm{v}$, we use ${\left|\bm{v}\right|}$ to denote its $l^{2}$norm. For any function $\bm{f}(\bm{x})$, $\bm{x}\in{\Omega}\subseteq^{d}$, taking values at nodes $\chi:=\{\bm{x}_{1},\bm{x}_{2},\dots,\bm{x}_{M}\}$, ${\left|\left|\bm{f}\right|\right|}$ denotes its $l^{2}$ norm, i.e., ${\left|\left|\bm{f}\right|\right|}:=\sqrt{\sum_{i=1}^{M}{{\left|\bm{f}(\bm{x}_{i})\right|}^{2}}/M}$, which can be seen as an approximation to the $L^{2}({\Omega})$ norm of $\bm{f}$ (up to a constant). ^d represents the dimension-$d$ Euclidean space.

2.1. Peridynamic Theory

Peridynamics provides a description of continuum mechanics in terms of integral operators rather than classical differential operators [58, 59, 60, 61, 62, 63, 64]. These nonlocal models include a length scale $\delta$, referred to as the horizon, which denotes the extent of nonlocal interaction. Because a peridynamic model does not require smoothness of the deformation, it allows a natural description of processes requiring reduced regularity in the solution, such as fracture [65, 66, 67].

In peridynamics, considering a domain of interest, ${\Omega}\subset^{d}$, the equation of motion is given in terms of the displacement $\bm{u}$ as follows:

(2.1) $$\rho(\bm{x})\ddot{\bm{u}}(\bm{x},t)=\int_{{B_{\delta}(\bm{x})}}\bm{f}(\bm{u},\bm{q},\bm{x},t)\;d\bm{q}+\bm{b}(\bm{x},t)\text{ ,}\quad(\bm{x},t)\in{\Omega}\times[0,T]\text{ ,}$$

where $\bm{x}$ and $\bm{q}$ are material points in the reference (undeformed) configuration of the body. $\rho(\bm{x})$ is the mass density function. $B_{\delta}(\bm{x})$ is a ball centered at $\bm{x}$ of radius $\delta$. $\bm{b}(\bm{x},t)$ is the body force density (external loading), which is assumed to be prescribed. $\bm{f}(\bm{u},\bm{q},\bm{x},t)$ is the pairwise bond force density that $\bm{q}$ exerts on $\bm{x}$, satisfying $\bm{f}(\bm{u},\bm{q},\bm{x},t)=-\bm{f}(\bm{u},\bm{x},\bm{q},t).$ The pairwise bond force density is given by

(2.2) $$\bm{f}(\bm{u},\bm{q},\bm{x},t)={\underline{\bm{T}}}[\bm{u},\bm{x},t]\langle\bm{q}-\bm{x}\rangle-{\underline{\bm{T}}}[\bm{u},\bm{q},t]\langle\bm{x}-\bm{q}\rangle\text{ ,}$$

where the underlined symbols denote states. States are map**s from a bond $\bm{q}-\bm{x}$ to some other quantity, usually either a vector or a scalar. ${\underline{\bm{T}}}$ is called the force state, which contains the contribution of the material model at a point to the bond force density. The quantities square brackets, $[\bm{u},\bm{x},t]$, indicate that ${\underline{\bm{T}}}$ is defined at material point $\bm{x}$ and time $t$, and it is dependent on the displacement field $\bm{u}(\cdot,\cdot)$. The force states in the right hand side of (2.2) contain the contributions of the material models at both bond endpoints $\bm{x}$ and $\bm{q}$ to the pairwise bond force density. A material model $\hat{\underline{\bm{T}}}({\underline{\bm{Y}}})$ provides the force state ${\underline{\bm{T}}}$ as a function of the deformation state ${\underline{\bm{Y}}}$, which is defined by

(2.3) $${\underline{\bm{Y}}}[\bm{u},\bm{x},t]\langle\bm{q}-\bm{x}\rangle={\bm{\xi}}+{\bm{\eta}},\text{ where }{\bm{\xi}}:=\bm{q}-\bm{x},\;{\bm{\eta}}:=\bm{u}(\bm{q},t)-\bm{u}(\bm{x},t)\text{ .}$$

Thus, in the peridynamic setting, a material model is a state-valued function of a state, rather than a tensor-valued function of a tensor, as in the conventional theory of continuum mechanics. For a heterogeneous body, our goal is to determine the material model:

$${\underline{\bm{T}}}[\bm{u},\bm{x},t]=\hat{\underline{\bm{T}}}({\underline{\bm{Y}}}[\bm{u},\bm{x},t],\bm{x}),$$

where the relationship between the force state and the deformation state depends explicitly on the position $\bm{x}$ as well as the deformation state at that position, ${\underline{\bm{Y}}}[\bm{u},\bm{x},t]$.

Following [56], in this work we assume that the material model is ordinary, meaning that the bond force vectors in the force state are always parallel to the deformed bonds. As in [56], it is further assumed that the material model is mobile, which means that the magnitudes of the bond force vectors in ${\underline{\bm{T}}}$ depend only on the length changes of the bonds. Denoting the unit direction of the deformed bond as:

(2.4) $${\underline{\bm{M}}}[\bm{u},\bm{x},t]\langle\bm{q}-\bm{x}\rangle:=\dfrac{{\underline{\bm{Y}}}[\bm{u},\bm{x},t]\langle\bm{q}-\bm{x}\rangle}{{\left|{\underline{\bm{Y}}}[\bm{u},\bm{x},t]\langle\bm{q}-\bm{x}\rangle\right|}}=\dfrac{{\bm{\xi}}+{\bm{\eta}}}{{\left|{\bm{\xi}}+{\bm{\eta}}\right|}},$$

and the length changes of the bond as

(2.5) $${\underline{e}}[\bm{u},\bm{x},t]\langle\bm{q}-\bm{x}\rangle:={\left|{\bm{\xi}}+{\bm{\eta}}\right|}-{\left|{\bm{\xi}}\right|},$$

the material model for a heterogeneous body composed of ordinary, mobile material can be written as:

(2.6) $$\hat{\underline{\bm{T}}}({\underline{\bm{Y}}},\bm{x})={\underline{t}}({\underline{e}},\bm{x})\,{\underline{\bm{M}}}\text{ .}$$

As discussed in [68, 56], this formulation guarantees linear and angular momentum conservation, Galilean invariance, and frame invariance (objectivity). An important feature of (2.6) is that the force in a given bond $\bm{q}-\bm{x}$ can depend on the length changes in all the bonds in the family of $\bm{x}$; expressing this dependence precisely is the purpose of state-based peridynamic material models.

Combining (2.1), (2.2), and (2.6), we obtain the following peridynamic model:

(2.7) $$\rho(\bm{x})\ddot{\bm{u}}(\bm{x},t)=\int_{{B_{\delta}(\mathbf{0})}}\left({\underline{t}}[\bm{u},\bm{x},t]\langle{\bm{\xi}}\rangle+{\underline{t}}[\bm{u},\bm{x}+{\bm{\xi}},t]\langle-{\bm{\xi}}\rangle\right){\underline{\bm{M}}}[\bm{u},\bm{x},t]\langle{\bm{\xi}}\rangle\;d{\bm{\xi}}+\bm{b}(\bm{x},t),\\ \text{ for }(\bm{x},t)\in\Omega\times[0,T],$$

with boundary data supplied by

(2.8) $$\bm{u}(\bm{x},t)=\bm{u}_{BC}(\bm{x},t)(\bm{x},t)\in\Omega_{I}\times[0,T].$$

In (2.7), the identity ${\underline{\bm{M}}}[\bm{u},\bm{x}+{\bm{\xi}},t]\langle-{\bm{\xi}}\rangle=-{\underline{\bm{M}}}[\bm{u},\bm{x},t]\langle{\bm{\xi}}\rangle$ has been used. $\Omega_{I}:=\{\bm{x}|\bm{x}\in^{d}\backslash{\Omega},\,\text{dist}(\bm{x},{\Omega})<2\delta\}$ is the interaction region in which prescribed boundary data $\bm{u}_{BC}$ is prescribed. With the peridynamic governing equation of motion, in order to guarantee the existence of a unique solution $\bm{u}$ for any forcing term $\bm{b}$, nonlocal boundary conditions (“volume constraints”) must be prescribed on this interaction region.

The purpose of this work is to learn the peridynamic material model, in the form of ${\underline{t}}$, from training data in the form of loading/response function pairs $\{\bm{u}^{s},\bm{b}^{s}\}_{s=1}^{S}$. The material model involves neural operators and is not simply an algebraic expression. The learnt model (2.7) is applied to solve for the displacement field $\bm{u}(\bm{x},t)$ in new loading instances $\bm{b}(\bm{x},t)$ distinct from the training instances. In addition to the displacement field, the peridynamic model provides other quantities of interests, such as the stress field given by:

(2.9) $$\bm{P}(\bm{x},t)=\int_{{B_{\delta}(\mathbf{0})}}{\underline{\bm{T}}}[\bm{u},\bm{x},t]\langle{\bm{\xi}}\rangle\otimes{\bm{\xi}}d{\bm{\xi}}.$$

Such an evaluation of the stress is especially important while learning from experimental measurements using DIC: while DIC has the major advantage to provide for full displacement fields, one of its main issues is that the stress field in heterogeneous state configurations cannot be directly accessed. By learning a constitutive law (2.7) from DIC measurements, our method allows for estimating displacement, strain, and stress fields on different domain and loading instances.

2.2. Nonlocal Neural Operators

The general nonlocal neural operators [43, 44, 45, 49, 40] were developed for scientific computing applications, which parameterize function-to-function map** by incorporating the nonlocal operator [65] into neural network architectures. A prototypical instance is the case of seeking PDE solution operator in material modeling problems, where the initial input field (body load/boundary load) is mapped to the corresponding displacement field via a nonlinear parameterized map**. Under this solution operator context, several architectures were developed in neural operator based methods [43, 44, 45, 49, 33, 40, 53, 69, 70, 41, 71, 72, 73, 74, 72]. Compared to classical neural networks that operate between finite-dimensional Euclidean spaces, one of the most remarkable advantages of neural operators is the capability to learn map**s between infinite-dimensional function spaces [43, 44, 45, 49, 75, 41, 42, 53, 70]. As a result, neural operators feature resolution independence, which implies that the prediction accuracy is invariant to the resolution of input functions. Furthermore, in contrast to classical PDE-based approaches, neural operators can be trained directly from data, and hence require no domain knowledge or pre-assumed PDEs. All these advantages make neural operators a promising tool for learning complex material responses from experimental measurements [52, 53, 54, 33, 43, 44, 45, 55].

Formally, a nonlocal neural operator aims to construct a surrogate operator $\mathcal{G}:\mathbb{U}\rightarrow\mathbb{F}$ that maps the input function $\bm{u}(\bm{x})$ to the output function $\bm{b}(\bm{x})$. The resolution-independence property is realized by parameterizing the layer update, $\mathcal{J}$, as a nonlocal (integral) operator, given as:

(2.10) $$\bm{h}(\bm{x},l+1)=\mathcal{J}[\bm{h}(\cdot,l)](\bm{x}):=\sigma\left({\bm{R}}\bm{h}(\bm{x},l)+\int_{\Omega}\bm{K}(\bm{x},\bm{q};\bm{v})\bm{h}(\bm{q},l)d\bm{q}+\bm{c}\right)\text{ .}$$

Here, $\bm{h}(\cdot,l)$ denotes the feature function of the $l-$th layer, taking values in ${}^{d_{h}}$. $\sigma$ is an activation function, ${\bm{R}}\in^{d_{h}\times d_{h}}$, $\bm{c}\in^{d_{h}}$ are trainable tensors parameterizing a point-wise linear transformation, and $\bm{K}\in^{d_{h}\times d_{h}}$ is a tensor kernel function whose parameters $\bm{v}$ are to be learned. While in the original version of nonlocal neural operator the integral is extended to the whole set ${\Omega}$, for efficiency purposes, restrictions to a ball of radius $\delta$ centered at $\bm{x}$, i.e. $B_{\delta}(\bm{x})$, can also be considered. However, as expected, this choice might compromise the accuracy since the support of Green’s function generally spans the whole domain in PDE solving problems [43].

Despite the aforementioned advances of nonlocal solution operators, these approaches have limitations from physical consistency and domain/boundary condition generalizability. That means, to learn the basic physical laws and generalize the solution, they require a large corpus of paired datasets, which is prohibitively in experiments. Moreover, the learnt operator is not applicable to different domain shapes/boundary conditions. To resolve these limitations, in our previous work [56], the peridynamic neural operator (PNO) was proposed, where we parameterize the nonlocal neural operator architecture as a nonlocal constitutive law based on the peridynamics formulation (2.7). As such, the model preserves the fundamental physical laws and is generalizable to different domain geometries and loading scenarios. In this context, the nonlocal neural operator aims to construct a surrogate operator $\mathcal{G}:\mathbb{U}\rightarrow\mathbb{F}$ that maps the displacement function $\bm{u}(\bm{x})$ to the body load function $\bm{b}(\bm{x})$:

(2.11) $$\mathcal{G}[\bm{u}](\bm{x},t)\approx\rho(\bm{x})\ddot{\bm{u}}(\bm{x},t)-\bm{b}(\bm{x},t)\text{ ,}$$

where the operator $\mathcal{G}$ is formulated as:

$$\mathcal{G}[\bm{u}](\bm{x},t):=\int_{{B_{\delta}(\mathbf{0})}}\left({\underline{t}}[\bm{u},\bm{x},t]\langle{\bm{\xi}}\rangle+{\underline{t}}[\bm{u},\bm{x}+{\bm{\xi}},t]\langle-{\bm{\xi}}\rangle\right){\underline{\bm{M}}}[\bm{u},\bm{x},t]\langle{\bm{\xi}}\rangle\;d{\bm{\xi}}.$$

Although $\bm{b}$ was treated in Section 2.1 as prescribed, here it becomes an output of the solution operator representing the external load that would be needed to make the approximation in (2.11) exact. Herein, we parameterize the scalar force state ${\underline{t}}$ with neural networks:

(2.12) $${\underline{t}}[\bm{u},\bm{x},t]\langle{{\bm{\xi}}}\rangle:=\sigma^{NN}(\omega({\bm{\xi}}),\vartheta(\bm{x},t),{\underline{e}}[\bm{u},\bm{x},t]\langle{\bm{\xi}}\rangle,|{\bm{\xi}}|;\bm{v})\text{ ,}$$

where

(2.13) $$\omega({\bm{\xi}}):=\omega^{NN}({\bm{\xi}};\bm{w})\text{ ,}$$

(2.14) $$\vartheta(\bm{x},t):=\dfrac{\int_{B_{\delta}(\mathbf{0})}\omega^{NN}\left({\bm{\xi}};\bm{w}\right){\underline{e}}[\bm{u},\bm{x},t]\langle{\bm{\xi}}\rangle|{\bm{\xi}}|d{\bm{\xi}}}{\int_{B_{\delta}(\mathbf{0})}\omega^{NN}\left({\bm{\xi}};\bm{w}\right)|{\bm{\xi}}|^{2}d{\bm{\xi}}}\text{ .}$$

$\sigma^{NN}$ and $\omega^{NN}$ are scalar-valued functions that take the form of a (usually shallow) multi-layer perceptron (MLP) with learnable parameters $\bm{v}$ and $\bm{w}$, respectively. $\omega$ is the kernel function characterizing the influence state of neighboring material points, and $\vartheta$ is a nonlocal generalization of the dilatation function, which describes the change of material in volume per unit volume.

With the PNO architecture, one can model complex material mechanical responses from data. In particular, given a set of observations $\mathcal{D}=\{\bm{b}^{s},\bm{u}^{s}\}_{s=1}^{S}$ of the loading field $\bm{b}^{s}(\bm{x},t)$ and the corresponding displacement field $\bm{u}^{s}(\bm{x},t)$, where $s$ is the sample index, the set of parameters in the network architecture will be inferred by solving the following minimization problem

(2.15) $$\min_{\bm{v},\bm{w}}\mathbb{E}_{\bm{u}}[C(\mathcal{G}[\bm{u};\bm{v},\bm{w}],\mathcal{G}^{\dagger}[\bm{u}])]\approx\min_{\bm{v},\bm{w}}\dfrac{1}{S}\sum_{s=1}^{S}[C(\mathcal{G}[\bm{u}^{s};\bm{v},\bm{w}],\rho\ddot{\bm{u}}^{s}-\bm{b}^{s})],$$

where $C$ denotes a properly defined cost functional $C:\mathcal{U}\times\mathcal{U}\rightarrow$. Although $\bm{u}^{s}$ and $\bm{b}^{s}$ are (vector) functions defined on a continuum, with the purpose of doing numerical simulations, we assume that they are defined on a discretization of the domain defined as $\chi=\{\bm{x}_{1},\cdots,\bm{x}_{M}\}\subset{\Omega}$, and a temporal discretization as $t_{n}=n\Delta t$, $n=1,\cdots,N$. With such a discretization to establish learning governing laws, a popular choice of the cost functional $C$ is the mean square error, i.e.,

$$C(\mathcal{G}[\bm{u}^{s};\bm{v},\bm{w}],\rho\ddot{\bm{u}}^{s}-\bm{b}^{s}):=\dfrac{1}{NM}\sum_{n=1}^{N}\sum_{\bm{x}_{i}\in\chi}{\left|\left|C(\mathcal{G}[\bm{u}^{s};\bm{v},\bm{w}](\bm{x}_{i},t_{n})-\rho(\bm{x}_{i})\ddot{\bm{u}}^{s}(\bm{x}_{i},t_{n})+\bm{b}^{s}(\bm{x}_{i},t_{n}))\right|\right|}^{2}.$$

Once the constitutive law is obtained, for any new loading instance $\bm{b}^{\text{t}est}$, we solve for the displacement field $\bm{u}$ using an iterative nonlinear static solver. The stress field $\bm{P}$ can also be calculated following (2.9).

3. Heterogeneous Peridynamic Neural Operators

3.1. Mathematical Formulation

In this section we introduce a heterogeneous PNO (HeteroPNO) architecture for the data-driven constitutive modeling of materials with position-dependent anisotropy. The particular materials of interest are biological tissues. These materials are soft and sustain large deformation. The position-dependent anisotropy in these tissues stems from embedded collagen fibers oriented in a heterogeneous pattern of directions in the plane. Figure 1 (on the right) shows an example of such orientation field for a biological tissue.

The idea behind the new architecture is to assume that a fundamental influence (kernel) function $\omega({\bm{\xi}})$ for an ideal homogeneous version of the tissue with unidirectional fiber orientation exists. This influence function can depend on direction of the bond vector ${\bm{\xi}}$ as well as $|{\bm{\xi}}|$. Then, in the case of the heterogeneous tissue, that influence state can be rotated at each location such that the stiffer direction of the influence state aligns with the fiber orientation in that location. Figure 1 schematically illustrates this idea.

Figure 1. Schematic demonstration of the hetero-PNO mechanism, with a fundamental influence (kernel) function $\omega$ corresponding to a horizontal fiber orientation, which is rotated for each point independently, according to the local fiber angle.

To construct such model, we replace (2.13) with the following position-dependent influence function:

(3.1) $$\omega(\bm{x},{\bm{\xi}}):=\omega^{NN}({\bm{R}}(-\alpha(\bm{x})){\bm{\xi}};\bm{w})\text{ ,}$$

where

(3.2) $${\bm{R}}(\theta):=\begin{bmatrix}\mathrm{cos}\theta&-\mathrm{sin}\theta\\ \mathrm{sin}\theta&\mathrm{cos}\theta\end{bmatrix}$$

is the rotation matrix. The fundamental influence function has its stiffest direction along the $x$-axis ($\alpha=0$). To account for the orientation of fibers in other directions at different positions, each peridynamic bond ${\bm{\xi}}$ is rotated clockwise by the fiber angle before evaluation. This is equivalent to rotating the fundamental influence function through an angle $\alpha$ counterclockwise at each point to align with the actual fiber orientation on that point.

The position-dependent influence state is then used in HeteroPNO architecture as follows:

(3.3) $$\mathcal{G}[\bm{u}](\bm{x},t)=\int_{{B_{\delta}(\mathbf{0})}}\left({\underline{t}}[\bm{u},\bm{x},t]\langle{\bm{\xi}}\rangle+{\underline{t}}[\bm{u},\bm{x}+{\bm{\xi}},t]\langle-{\bm{\xi}}\rangle\right){\underline{\bm{M}}}[\bm{u},\bm{x},t]\langle{\bm{\xi}}\rangle\;d{\bm{\xi}}=\rho(\bm{x})\ddot{\bm{u}}(\bm{x},t)-\bm{b}(\bm{x},t),$$

where

(3.4) $${\underline{t}}[\bm{u},\bm{x},t]\langle{{\bm{\xi}}}\rangle:= \sigma^{NN}(\omega(\bm{x},{\bm{\xi}}),\vartheta(\bm{x},t),{\underline{e}}[\bm{u},\bm{x},t]\langle{\bm{\xi}}\rangle,|{\bm{\xi}}|;\bm{v})\;\text{ ,}$$

(3.5) $$\omega(\bm{x},{\bm{\xi}}):= \omega^{NN}({\bm{R}}(-\alpha(\bm{x})){\bm{\xi}};\bm{w})\text{ ,}$$

(3.6) $$\vartheta(\bm{x},t):= \dfrac{\int_{B_{\delta}(\mathbf{0})}\omega^{NN}\left(\bm{x},{\bm{\xi}};\bm{w}\right){\underline{e}}[\bm{u},\bm{x},t]\langle{\bm{\xi}}\rangle|{\bm{\xi}}|d{\bm{\xi}}}{\int_{B_{\delta}(\mathbf{0})}\omega^{NN}\left(\bm{x},{\bm{\xi}};\bm{w}\right)|{\bm{\xi}}|^{2}d{\bm{\xi}}}\text{ .}$$

Our goal is to learn the optimal influence (kernel) function $\omega^{NN}$, force state $\sigma^{NN}$, together with the pointwise fiber orientation function $\alpha(\bm{x})$ from data.

3.2. Machine learning algorithm

In this section we describe how to train the HeteroPNO models for two different scenarios: 1) learning the constitutive model from displacement field–force field data pairs, with the ground-truth fiber orientation field given. 2) Discovering both the constitutive law and the microstructure (fiber orientation field) from displacement-force field data.

Algorithm 1 Algorithm for HeteroPNOs.

1:Pre-processing Phase:

2:Read data and inputs: $\chi=\{\bm{x}_{i}\}_{i=1}^{M}$,

3:if Case I then

4:     $\mathcal{D}_{tr}:=\{\bm{u}^{s,tr}(\bm{x}_{i}),\bm{b}^{s,tr}(\bm{x}_{i})\}_{i=1,s=1}^{M,S_{tr}}$ for training, $\mathcal{D}_{val}:=\{\bm{u}^{s,val}(\bm{x}_{i}),\bm{b}^{s,val}(\bm{x}_{i})\}_{i=1,s=1}^{M,S_{val}}$ for validation sets.

5:if Case II then

6:     $\mathcal{D}_{tr}:=\{\bm{u}^{s,tr}(\bm{x}_{i})\}_{i=1,s=1}^{M,S_{tr}},\{\bm{P}^{s,tr}\}_{s=1}^{S_{tr}}$ for training, $\mathcal{D}_{val}:=\{\bm{u}^{s,val}(\bm{x}_{i})\}_{i=1,s=1}^{M,S_{val}},\{\bm{P}^{s,val}\}_{s=1}^{S_{val}}$ for validation sets.

7:Find and store the peridynamic neighbor node set $N(\bm{x}_{i})$ in the form of message passing graph structure:

8:for $i=1:M$ do

9:     Find all $\bm{x}_{j}\in\chi$ such that $|\bm{x}_{j}-\bm{x}_{i}|<\delta$.

10:Training Phase:

11:for $ep=1:epoch_{max}$ do

12:     Initialize error: $E_{tr}=0$

13:     for  each batch do

14:         Reset batch loss: $loss=0$

15:         if Case I then

16:              Compute batch loss following (3.7).

17:         else if Case II then

18:              For each training sample in this batch, run nonlinear solver to obtain $\mathcal{G}^{-1}[\bm{b}^{s}]$.

19:              Compute peridynamic stress for all training samples from (2.9) and get the averaged axial components $\hat{\bm{P}}={\hat{P_{11}},\hat{P_{22}}}$.

20:              Compute batch loss following (3.8).          

21:         Update HeteroPNO parameters $\bm{w}$, $\bm{v}$, and $\bm{\alpha}$ with the Adam optimizer.      

22:     Compute training error $E_{tr}$.

23:     Validation Phase:

24:     if training error decreases then

25:         Initialize validation error: $E_{val}=0$

26:         if Case I then

27:              Compute validation error following (3.7).

28:         else if Case II then

29:              For each validation sample, run nonlinear solver to obtain $\mathcal{G}^{-1}[\bm{b}^{s}]$.

30:              Compute peridynamic stress from (2.9) and get the averaged axial components $\hat{\bm{P}}={\hat{P_{11}},\hat{P_{22}}}$.

31:              Compute validation error following (3.8).          

32:         If validation error also decreases, save the current model.      

Constitutive model learning.

In this work we focus on (quasi)static, two-dimensional ($d=2$) applications, although the prescribed architecture is readily applicable to higher-dimensional domains and dynamic settings. We aim to learn the surrogate operator $\mathcal{G}$ such that

$$\mathcal{G}[\bm{u}](\bm{x})\approx-\bm{b}(\bm{x})\text{ ,}$$

where $\bm{b}$ is prescribed. Algorithm 1 provides the structure of the pseudo code used for this work to learn the constitutive model (and microstructure if applicable). The program is coded in Python and uses the PyTorch libraries for GPU computing and the Adam optimizer for minimizing the loss function. In all tests, the neural network functions for $\omega^{NN}$ and $\sigma^{NN}$ are multi-layer perceptrons (MLP) with three layers and ReLu as the activation function. The widths of the MLP layers are denoted by $(W_{in},W_{h1},W_{h2},W_{out})$, where $W_{in}$ is the input’s dimension, $W_{h1},W_{h2}$ are the widths of the hidden layers, and $W_{out}$ is the dimension of the output. For $\omega^{NN}$, $W_{in}=2$ and $W_{out}=1$ for all examples, although $W_{out}$ can take on larger values in other applications. From (3.4), since $|{\bm{\xi}}|$ and ${\underline{e}}$ are scalars and $\vartheta(\bm{x})$ has the same dimension as $\omega({\bm{\xi}})$ (see (2.14)), it follows that $W_{in}=4$ and $W_{out}=1$ in the $\sigma^{NN}$ MLP. $W_{h1},W_{h2}$ in both $\omega^{NN}$ and $\sigma^{NN}$ as well as the horizon size $\delta$ are hyperparameters of choice to tune.

Formally, we assume that measurements are available in the following format:

• Coordinates of nodes where measurements are taken: $\chi=\{\bm{x}_{i}\}_{i=1}^{M}$, where $M$ is the total number of nodes.

• External force-displacement function pairs at each node $\bm{x}_{i}$ and for each sample $s$: $\{\bm{u}^{s}_{i},\bm{b}^{s}_{i}\}_{i=1,s=1}^{M,S}$ where $S$ is the total number of samples.

• If the external forces are all zero (which is the case for the biaxial tension experiments described below), we can compare the area-averaged stress components given by (2.9) against the measured loads in the testing machine: $\{\bm{P}^{s}\}_{s=1}^{S}$ where $\bm{P}^{s}=\{P^{s}_{11},P^{s}_{22}\}$ and $S$ is the total number of samples.

For the purpose of training, validation and test, we split the available data into three sets: $S_{tr}$, $S_{val}$, $S_{test}$ denote the total number of samples in training, validation, and test sets, respectively. Depending on whether the data includes nonzero external forces (Case I) or not (Case II), either of the following two different loss functions is used.

In Case I, we follow the convention of nonlocal neural operator literature [43, 45] and consider the relative $L^{2}$ error of the output function, $\bm{b}$, as the loss function, which is given by

(3.7) $$\text{loss}_{b}=\frac{1}{S_{tr}}\sum_{s=1}^{S_{tr}}\dfrac{{\left|\left|\mathcal{G}[\bm{u}^{s}]+\bm{b}^{s}\right|\right|}}{{\left|\left|\bm{b}^{s}\right|\right|}}\text{ .}$$

Physically, the loss function in (3.7) measures the error in the internal force distribution on the nodes needed to equilibrate the system.

If external forces are zero (Case II), the loss function (3.7) fails since the denominator becomes zero. If the denominator is removed to avoid division by zero, the program leads to the trivial solution for the model parameters, such that with any deformation the model returns zero forces. To circumvent this problem, in Case II, we define the loss in terms of the displacement field and averaged axial Piola-Kirchhoff stress components:

(3.8) $$loss_{u}=\beta\left(\frac{1}{S_{tr}}\sum_{s=1}^{S_{tr}}\dfrac{{\left|\left|\mathcal{G}^{-1}[\bm{b}^{s}]-\bm{u}^{s}\right|\right|}}{{\left|\left|\bm{u}^{s}\right|\right|}}\right)+(1-\beta)\left(\frac{1}{S_{tr}}\sum_{s=1}^{S_{tr}}\dfrac{{\left|\hat{\bm{P}^{s}}-\bm{P}^{s}\right|}}{\bar{P}}\right)\text{ ,}$$

where $\hat{\bm{P}^{s}}$ denotes the spatial average of axial 1st Piola-Kirchhoff stresses for sample $s$, and $\bar{P}$ is the mean of axial ground truth stresses across all training samples. This results in a scaled absolute error measure for stress contribution to the loss. $\mathcal{G}^{-1}[\bm{b}^{s}]$ in (3.8) denotes the numerical solution of $\mathcal{G}[\bm{u}]=-\bm{b}^{s}$ using an iterative nonlinear static solver (see line 18 in Algorithm 1). In this work we use the Polak-Ribiere conjugate gradient method [76], due to its efficiency and robustness in peridynamics nonlinear problems [77]. Additional inputs needed for the solver are ${\bm{u}^{s}_{0}}$, ${\bm{u}^{s}_{BC}}$, $tol$, $itr_{max}$, which denote the initialization displacement (which may or may not be zero), Dirichlet boundary condition values on $\Omega_{I}$, convergence tolerance, and maximum allowed iterations, respectively. $\beta$ in (3.8) takes values between 0 and 1 and is a hyperparameter to tune. It determines how much the learning relies on stress versus the displacement predictions.

As described in [56], Case II is computationally more expensive and has significantly higher memory requirements due to the multi-iteration procedure per sample and per epoch. To make the Case II algorithm affordable, the following strategies are taken to reduce the number of solver’s iterations:

1. Avoiding the use of too small tolerances for convergence criterion.

2. Smart initialization of the solution. Here, in particular, we used the displacements from another data sample with a close temporal sequence as the initial guess for the solver.

3. Limiting the maximum number of iterations to the lowest possible value at which convergence for most samples is likely to be achieved.

As described in [56], in Case II, we found it helps to modify the architecture of the scalar force state given by (2.12) as follows:

(3.9) $${\underline{t}}[\bm{u},\bm{x}]\langle{{\bm{\xi}}}\rangle:=\sigma^{NN}(\omega(\bm{x},{\bm{\xi}}),\vartheta(\bm{x}),{\underline{e}}[\bm{u},\bm{x}]\langle{{\bm{\xi}}}\rangle,|{\bm{\xi}}|;v)\;e({\bm{\xi}},{\bm{\eta}})\text{ .}$$

which guarantees that a small deformation bond will be associated with a small force state, and it causes the larger deformation bonds to weigh more on the overall equilibrium deformation. Thus, the modified form (3.9) tends to improve the training efficiency.

Microstructure Discovery.

When the microstructure field, in the form of $\alpha(\bm{x})$ is not provided, we further treat $\bm{\alpha}:=\{{\alpha}(\bm{x}_{i})\}_{i=1}^{M}$ as trainable parameters to be found together with other HeteroPNO’s learnable MLPs’ weights and biases. As such, we discover the $\alpha$ field from displacement-force/stress data only. A two-stage training strategy is developed. First, we train a homogeneous PNO (HomoPNO). We then take the trained HomoPNO force state function ($\sigma^{NN}$) and the influence (kernel) function ($\omega^{NN}$) as initializations for $\sigma^{NN}$ and $\omega^{NN}$ in HeteroPNO, and train for $\sigma^{NN}$, $\omega^{NN}$, and $\alpha(\bm{x})$ simultaneously. For the purpose of verification, in our numerical experiments we consider two scenarios:

• Using actual fiber angles, $\alpha(\bm{x})$, in (3.1) as known information. This scenario will be denoted as HeteroPNO I.

• Treating $\alpha(\bm{x})$ as unknown to be learned from data. This scenario will be denoted as HeteroPNO II.

Stress field calibration.

Note that unlike Case II where stress information are directly incorporated in the loss function, Case I loss only enforces $\mathcal{G}[\bm{u}](\bm{x})=-\bm{b}(\bm{x})$, which makes the HeteroPNO invariant to shifted scalar force states. That means, given a force state function ${\underline{t}}$ satisfying (3.3) and denoting $\tilde{{\underline{t}}}:={\underline{t}}+C$ for a scalar constant $C$, we have

$$\tilde{\mathcal{G}}[\bm{u}](\bm{x}):= \int_{{B_{\delta}(\mathbf{0})}}\left(\tilde{{\underline{t}}}[\bm{u},\bm{x},t]\langle{\bm{\xi}}\rangle+\tilde{{\underline{t}}}[\bm{u},\bm{x}+{\bm{\xi}},t]\langle-{\bm{\xi}}\rangle\right){\underline{\bm{M}}}[\bm{u},\bm{x},t]\langle{\bm{\xi}}\rangle\;d{\bm{\xi}}$$

$$= \int_{{B_{\delta}(\mathbf{0})}}\left({{\underline{t}}}[\bm{u},\bm{x},t]\langle{\bm{\xi}}\rangle+C+{{\underline{t}}}[\bm{u},\bm{x}+{\bm{\xi}},t]\langle-{\bm{\xi}}\rangle+C\right){\underline{\bm{M}}}[\bm{u},\bm{x},t]\langle{\bm{\xi}}\rangle\;d{\bm{\xi}}$$

$$= \int_{{B_{\delta}(\mathbf{0})}}\left({{\underline{t}}}[\bm{u},\bm{x},t]\langle{\bm{\xi}}\rangle+{{\underline{t}}}[\bm{u},\bm{x}+{\bm{\xi}},t]\langle-{\bm{\xi}}\rangle\right){\underline{\bm{M}}}[\bm{u},\bm{x},t]\langle{\bm{\xi}}\rangle\;d{\bm{\xi}}+2C\int_{{B_{\delta}(\mathbf{0})}}{\underline{\bm{M}}}[\bm{u},\bm{x},t]\langle{\bm{\xi}}\rangle\;d{\bm{\xi}}$$

Here, we have used the fact

$$\int_{{B_{\delta}(\mathbf{0})}}{\underline{\bm{M}}}[\bm{u},\bm{x},t]\langle{\bm{\xi}}\rangle\;d{\bm{\xi}}\approx 0$$

due to radial symmetry of deformed $B_{\delta}(\mathbf{0})$ for sufficiently small horizon. In order to predict stress correctly using the HeteroPNO trained via Case I loss, we use stress from a training sample to calibrate the shift constant. In particular, we solve for $C$ via:

(3.10) $$\int_{{B_{\delta}(\mathbf{0})}}({{\underline{t}}}[\bm{u},\bm{x},t]\langle{\bm{\xi}}\rangle+C){\underline{\bm{M}}}[\bm{u},\bm{x},t]\langle{\bm{\xi}}\rangle\otimes{\bm{\xi}}d{\bm{\xi}}=\bm{P}(\bm{x}).$$

Then, the force state is calibrated as:

$$\tilde{{\underline{t}}}[\bm{u},\bm{x},t]\langle{\bm{\xi}}\rangle:={{\underline{t}}}[\bm{u},\bm{x},t]\langle{\bm{\xi}}\rangle+C$$

and employed to calculated the stress filed.

4. Verification on Synthetic Dataset

Figure 2. Demonstration of the heterogeneous synthetic data generation, under HGO constitutive law and heterogeneous fiber orientation field. Left: One instance of the random applied external forces and the resulting displacement field. Right: the data setting and prescribed collagen fiber orientation.

In this section, we illustrate the performance of the proposed HeteroPNO on a benchmark material modeling problem. In particular, we consider a synthetic dataset describing the deformation of a hyperelastic and anisotropic fiber-reinforced material. All our numerical experiments were performed on a machine with a single Nvidia RTX 3090 GPU.

4.1. Data preparation

To generate training and test samples, the Holzapfel-Gasser-Odgen (HGO) model [78] was employed to describe the constitutive behavior of the material in this example, with its strain energy density function given as:

$$\psi =\frac{E}{4(1+\nu)}(\overline{I}_{1}-2)-\frac{E}{2(1+\nu)}\ln(J)$$

$$+\frac{k_{1}}{2k_{2}}\left(\exp{(k_{2}\langle S(\alpha)\rangle^{2}})+\exp{(k_{2}\langle S(-\alpha)\rangle^{2}})-2\right)+\frac{E}{6(1-2\nu)}\left(\frac{J^{2}-1}{2}-\ln{J}\right).$$

Here, $\langle\cdot\rangle$ denotes the Macaulay bracket, and the fiber strain of the two fiber groups is defined as:

$$S(\alpha)=\frac{\overline{I}_{4}(\alpha)-1+|\overline{I}_{4}(\alpha)-1|}{2}.$$

where $k_{1}$ and $k_{2}$ are fiber modulus and the exponential coefficient, respectively, $E$ is the Young’s modulus for the non-fibrous ground matrix, and $\nu$ is the Poisson ratio. Moreover, $\overline{I}_{1}=\text{tr}(\mathbf{C})$ is the first invariant of the right Cauchy-Green tensor $\mathbf{C}=\mathbf{F}^{T}\mathbf{F}$, $\mathbf{F}:=\bm{F}(\bm{u})=\bm{I}+\dfrac{\partial\bm{u}}{\partial\bm{x}}$ is the deformation gradient tensor, and $J=\det\mathbf{F}$. For the fiber group with angle direction $\alpha$ from the reference direction, $\overline{I}_{4}(\alpha)=\mathbf{n}^{T}(\alpha)\mathbf{C}\mathbf{n}(\alpha)$ is the fourth invariant of the right Cauchy-Green tensor $\mathbf{C}$, where $\mathbf{n}(\alpha)=[\cos(\alpha),\sin(\alpha)]^{T}$. Here, $\alpha$ denotes the collagen fiber orientation angle.

In this problem, the specimen is assumed to be subject to different body loads $\bm{b}(\bm{x})$, and the goal is to find the corresponding displacement field $\bm{u}:\Omega\rightarrow^{2}$ under each body loading, where $\Omega:=[0,1]^{2}$. To generate the high-fidelity (ground-truth) dataset, we sampled $250$ different body loads $\bm{b}(\bm{x})$ from a random field, following the algorithm in [79, 54]. To include heterogeneity in the generated data, we assume that the left and the right half of the square domain have fibers along 110^o and 70^o directions respectively (see Figure 2). The external force, $\bm{b}(\bm{x})$ is taken as the restriction of a 2D random field, $\phi(\bm{x})=\mathcal{F}^{-1}(\gamma^{1/2}\mathcal{F}(\Gamma))(\bm{x})$. Here, $\Gamma(\bm{x})$ is a Gaussian white noise random field on ², $\gamma=(w_{1}^{2}+w^{2}_{2})^{-\frac{5}{4}}$ represents a correlation function, $w_{1}$, $w_{2}$ are the wave numbers on $x$ and $y$ directions, respectively, and $\mathcal{F}$, $\mathcal{F}^{-1}$ denote the Fourier transform and its inverse, respectively. Then, for each sampled traction loading, we solved the displacement field on the entire domain by solving the weak form of displacement formulation for the equilibrium: find the displacement field $\bm{u}(\bm{x})\in\bm{U}_{0}:=\{\bm{w}(\bm{x})\in[H^{1}([0,1]^{2})^{2}|\bm{w}(\bm{x})=0\text{ on }\partial\Omega]\}$ which satisfies:

$$\int_{\Omega}\bm{\sigma}:\dfrac{1}{2}\left(\left(\dfrac{\partial\bm{v}}{\partial\bm{x}}\right)^{T}\bm{F}(\bm{u})+\bm{F}(\bm{u})^{T}\left(\dfrac{\partial\bm{v}}{\partial\bm{x}}\right)\right)d\bm{x}-\int_{\Omega}\rho\bm{b}\cdot\bm{v}d\bm{x}=0,\bm{v}\in\bm{U}_{0},$$

where $\bm{\sigma}:=\dfrac{\partial\psi}{\partial\bm{E}}$ is the second Piola-Kirchhoff stress tensor, and $\bm{E}:=\dfrac{1}{2}\left(\left(\dfrac{\partial\bm{u}}{\partial\bm{x}}\right)^{T}+\left(\dfrac{\partial\bm{u}}{\partial\bm{x}}\right)+\left(\dfrac{\partial\bm{u}}{\partial\bm{x}}\right)^{T}\left(\dfrac{\partial\bm{u}}{\partial\bm{x}}\right)\right)$ is the Green Lagrange strain tensor. In particular, we use the finite element method implemented in FEniCS [80], with the displacement filed approximated by continuous piecewise linear finite elements with triangular mesh, and the grid size taken as $0.05$. Then, the finite element solution was interpolated onto $\chi$, a structured $21\times 21$ grid which will be employed as the discretization in our HeteroPNO.

Figure 2 shows the problem setting and the force and the displacement fields for one of the generated samples. After generating the samples we extend the domain on all four edges for each sample by $2\delta$ as our $\Omega_{I}$ where nonlocal Dirichlet boundary condition/volume constraints are prescribed using the mirror-based fictitious nodes method [81] for peridynamic computations. For more details of the extension and nonlocal boundary values on a similar data set see [56].

4.2. Learning the constitutive law and microstructure

error		training (200)	validation (25)	test(25)
force	HomoPNO	16.02%	17.05%	16.33%
	HeteroPNO I	8.98%	9.56%	8.75%
	HeteroPNO II	7.17%	7.67%	7.25%
displacement	HomoPNO	6.45%	6.80%	6.91%
	HeteroPNO I	1.77%	1.91%	1.81%
	HeteroPNO II	1.57%	1.83%	1.73%

Table 1. Averaged relative $L^{2}$-norm error for HomoPNO and HeteroPNO predictions on forces (given displacement) and on displacement (given boundary conditions).

Figure 3. HomoPNO and HeteroPNO predictions of displacement field (given extenal load and boundary conditions), and external forces (given displacement field) against the ground truth.

Figure 4. HomoPNO and HeteroPNO predictions of the first Piola-Kirchhoff stress tensor against the ground truth.

Figure 5. Discovered hidden fiber orientation by Hetero-PNO on the synthetic data set and comparison against the ground truth. The averaged absolute error is 9.30^o.

Figure 6. Learned influence (kernel) states for HomoPNO and the two HeteroPNO models.

We randomly split the 250 generated samples into training, validation, and test sets of size 200, 25, and 25, respectively. The widths of the MLPs for $\omega^{NN}$ and $\sigma^{NN}$ are (2, 256, 512, 1) and (4, 512, 512, 1) respectively, and the peridynamic horizon size is set as $\delta=3\Delta x$. In this example, since the data contains nonzero external forces, the setting falls under the category of Case I where (3.7) is minimized via Algorithm 1.

After training the PNO models, we report two different errors: 1) force error which reports the difference between predicted $-\mathcal{G}[\bm{u}](\bm{x})$ and the applied external force density. 2) displacement error which reports the difference between the ground truth displacements and HeteroPNO predicted displacements using the static solver to solve for equilibrium under applied external force and Dirichlet boundary condition. These errors for the HomoPNO and HeteroPNO models are reported in 1. Here, HeteroPNO I and II, respectively, refer to scenarios were microstructure/orientation field is considered as known input, or needs to be inferred from data. The predicted displacements and forces for one test sample are plotted in Figure 3. We also plot the HeteroPNO predicted 1st Piola-Kirchhoff stress components versus the ground truth for the same test sample in Figure 4. From Table 1 We observe that the HeteroPNO architecture improves the force predictions by $50\%$ and the displacement prediction by $75\%$ compared to the HomoPNO. The HeteroPNO II model has slightly lower error compared to the HeteroPNO I. Although HeteroPNO I uses the ground-truth fiber orienations, having $\alpha$ as tunable parameter in HeteroPNO II, makes it more expressive, allowing it to find the $\alpha$ that gives the minimum error regardless of what its physical purpose in the model is. Figure 5 shows the predicted fiber orientation field by HeteroPNO II versus the ground-truth fibers assumed during data generation. The left plot is the discovered microstructure, the middle displays the overlaid predicted microstructure and the ground-truth one for better visual comparison. The right image plots the absolute difference in angles normalized by 90^o:

(4.1) $$err_{\alpha}=\frac{|\hat{\alpha}-\alpha|}{90^{o}}$$

where $\hat{\alpha}$ denotes the predicted angles. In this definition, error value of 1.0, corresponds to the worst possible angle difference which is 90^o. As observed the microstructure prediction error is higher in the middle vertical line which is determined to have a sharp transition between 110^o and 70^o orientation. This can be attributed to the fact that PNO learns a smoother transition between the left and right subdomains.

Lastly, we plot the learned influence states for HomoPNO, HeteroPNO I and II in Figure 6. The kernel in HomoPNO with higher values along the vertical direction suggest that the material has a stiffer response in 90^o which is consistent with the averaged fiber orientation assumed in the generated data. The base influence states of HeteroPNO I and II which are corresponding to $\alpha=0$ fiber directions, both show stiffer material response in horizontal axis, which is expected. Note that these influence states are then rotated at each location by depending on $\alpha$ at that location to align with local fibers.

5. Application on DIC measurements of Bio-tissues

Having illustrated the performances of our learned neural operators on high-fidelity synthetic simulation datasets, we now consider a problem of learning the material response of a tissue sample from DIC displacement tracking measurements as a prototypical exemplar. The main objective of this section is to provide a proof-of-principle demonstration that the framework introduced thus far applies to discover the constitutive equations and material microstructure and to estimate the stress field, while the dataset has unavoidable measurement noise. In this application we further compare our proposed HeteroPNO against two conventional approaches that use constitutive modeling with parameter fitting to demonstrate the advantages of neural operator models and the importance of considering the heterogeneity of material microstructures.

5.1. Data Collection and Preparation

Figure 7. Experimental collagen fiber orientation field.

We have acquired biaxial mechanical testing data and collagen fiber microstructure from a representative porcine tricuspid valve anterior leaflet (TVAL) tissue, following our established experimental procedures [82, 83, 84]. In brief, one adult porcine heart ($n=1$, 120 kg, 1.5 years of age) was obtained from a USDA-approved abattoir (Animal Technologies, Inc., Tyler, TX, USA) within a day of animal slaughtering. Upon arrival at our laboratory, we then sectioned the TVAL tissue into a square specimen and measured the thickness at three locations using an optical measuring system (Keyence, Itasca, IL, USA) to obtain the average tissue thickness ($L_{z}=0.22mm$). We next introduced a speckling pattern to the tissue surface using an airbrush and black paint [33], that was later used in digital image correlation (DIC)-based tracking of tissue displacement/deformation. The painted specimen was then mounted to a commercial biaxial testing system (BioTester, CellScale, Waterloo, ON, Canada), resulting in an effective testing area of $L_{x}\times L_{y}=9\times 9mm$ for the subsequent tissue biaxial mechanical characterizations.

First of all, the sample was immersed in a phosphate-buffered saline (PBS) and underwent a preconditioning protocol that consists of 10 cycles of equi-biaxial tension loading and unloading, targeting a first Piola-Kirchhoff stress of 150 kPa for emulating the in vivo functioning conditions of the tricuspid valve [82]. Next, seven displacement-controlled biaxial tension protocols were performed, considering the following biaxial stresses: $P_{11}:P_{22}=1:1,1:0.75,1:0.5,1:0.25,0.75:1,0.5:1,$ and $0.25:1$. Here, $P_{11}=f_{x}\textfractionsolidus{L_{x}L_{z}}$ and $P_{22}=f_{y}{\textfractionsolidus L_{y}L_{z}}$ are the 1-1 and 2-2 first Piola-Kirchhoff (1st PK) stress components, respectively. Since the specimen was mounted with tissue’s circumferential direction aligned with the $x-$direction of the BioTester, they can also be viewed as the 1st Piola-Kirchhoff stress components in the $x-$ and $y-$directions. The corresponding stretches can also be computed as the ratio of the deformed to the undeformed tine distances. Each of the above seven stress ratios was performed for five loading/unloading cycles. From the last load and unloading cycle of each stress ratio, $1280\times 960$ resolution images of the specimen were captured by a CCD camera, and the load cell readings ($f_{x}$ and $f_{y}$) were recorded at 5 Hz, resulting in 1,318 data points.

Collagen Microstructural Imaging.

After mechanical testing, we further performed collagen microstructural imaging via an in-house polarized spatial frequency domain imaging (pSFDI) device to obtain the collagen fiber orientation map of the tissue specimen. Following the procedure of pSFDI-based collagen fiber quantifications, the incident spatial frequency light patterns were produced from an LED projector (Texas Instruments, Dallas, TX) with a wavelength of 490 nm (cyan). A 1.2-megapixel CCD camera (Basler, Germany) was used to capture the reflected light intensity responses through a rotating linear polarizer (Thorlabs Inc., Newton, NJ) at 37 distinct polarization states (i.e., 0° to 180°, 5° increments). This imaging procedure was repeated for three linear phase shifts (0°, 120°, and 240°) of the spatial frequency pattern. Image processing and data analyses were then completed via custom MATLAB (MathWorks, Natick, MA) programs to examine the pixel-by-pixel collagen fiber orientation ($\alpha$) of the tissue’s region of interest (ROI) at the post-preconditioning state that was chosen as the reference (undeformed) configuration. Here, the central $4.4\times 4.4$ $mm^{2}$ region was selected as the tissue ROI for our collagen fiber microstructure analysis and DIC-based displacement tracking (Figure 7 and 8). Please refer to more details of pSFDI data analysis in Goth et al. [85] and Jett et al [83]. The fiber orientation field is further smoothed using convolution with a square pulse of size $21$ pixels to reduce measurement noises. Figure 7 shows the measured fiber orientation field and its visualization after smoothing in the ROI.

Digital Image Correlation (DIC)-based Displacement Tracking.

An open-source DIC software Ncorr [86] was used in this study to extract the full-field displacements from the recorded images. A rectangular ROI was considered in the reference image (the colored regions in Figure 8), which is partitioned into subsets of smaller regions. A subset size of 40 pixels (0.878 mm) and a subset spacing of 2 pixels (0.0439 mm) were selected according to DIC processing principles [87] and the speckle patterns in this study. Each distinctively identified pair of corresponding subsets in the reference and deformed images are correlated based on the normalized cross-correlation (NCC) criteria [86] (difference vector norm cutoff 1e-6, iteration count cutoff 100). Since the TVAL specimen was subjected to large deformations during the equi-biaxial experiments, the reference image was updated multiple times during the DIC analysis to continuously keep track of the positions of largely deformed material regions. This was achieved by enabling the ‘high-strain analysis’, ‘seed propagation’, and ‘auto propagation’ functionalities in Ncorr. The full-field displacement components shown in Figure 8 was obtained in the software by the following algorithm:

$$\tilde{x}_{i}-x_{i}=u_{0}+\frac{\partial u}{\partial x}(x_{i}-x_{0})+\frac{\partial u}{\partial y}(y_{i}-y_{0})$$

$$\tilde{y}_{i}-y_{i}=v_{0}+\frac{\partial v}{\partial x}(x_{i}-x_{0})+\frac{\partial v}{\partial y}(y_{i}-y_{0})$$

where $(x_{0},y_{0})$ and $(u_{0},v_{0})$ are the coordinate and displacement components of a reference subset center. $(x_{i},y_{i})$ and $(u,v)$ are coordinate and displacement components of an initial reference subset point. Coordinates of that reference subset point in the deformed state is denoted by $(\tilde{x}_{i},\tilde{y}_{i})$. Afterwards the displacements in ROI was further smoothed by convolving with a square pulse of size $105\times 105$ pixels to reduce the DIC processing artifacts.

Figure 8. Demonstration of displacement field measurements via digital image correlation (DIC).

5.2. Learning material model and microstructure discovery

The 1318 collected data is split into training, validation and test sets as follows. The training set used for this problem consists of 100 samples equi-spaced temporally over all cycles. The validation set consists of 20 samples selected from the remainder of the data, such that instances with high, low, and mid-level stretches from all seven protocols are included. The remainder 1198 samples are used as the test set to evaluate the accuracy of the models.

To have a base model, first, we calibrate a popular classical hyperelastic constitutive law using the training set. Then we train a HomoPNO, HeteroPNO I (assuming fiber orientation field are given) and HeteroPNO II (unknown fiber angles to be discovered) in a similar fashion to the pervious example.

Baseline Constitutive Model.

For the baseline model, we use one of the popular classical constitutive laws, Fung model, which is frequently used for biological tissues. The Fung model here has the strain energy density of form:

$$\psi=\frac{c}{2}\left[exp(a_{1}E_{11}^{2}+a_{2}E_{22}^{2}+2a_{3}E_{11}E_{22})\right]\text{,}$$

where $c$, $a_{1}$, $a_{2}$, and $a_{3}$ are the model parameters to be determined, and $E_{11}$, $E_{22}$ are the principle Green–Lagrange strains in the $x-$ and $y-$directions, respectively. Here we have used two approaches to determine the model parameters from data. One approach, we we call it forward calibration, is to find the parameters by fitting the model predicted stretch-stress curves to those obtained from the DIC measurements. To this aim differential evolution optimization is employed which minimizes the residual mean squared errors in the stress between the experimental data (training set only) and model prediction [88].
The second calibration approach which we refer to it as inverse, is to define the optimization objective function on the finite-element-obtained displacement error instead of the stresses. This requires FE analysis for each optimization step on all training samples to provide nodal displacement predictions which is then passed onto the objective function to be compared with the DIC measurements.
Once the model parameters are obtain from either approaches, they are used in Abaqus/standard solver [89] to predict the displacements for samples in the test set. The FEM setting in inverse calibration and also in the testing, is to solve for equilibrium using the DIC displacement on the domain edges as Dirichlet boundary conditions.

model	training(100)	validation(20)	test(1198)
Fung (forward)	15.64 %	18.69 %	16.02 %
Fung (inverse)	15.17 %	18.01 %	15.39 %
HomoPNO	8.48 %	12.43 %	9.95 %
HeteroPNO I	7.65 %	11.42 %	8.50 %
HeterPNO II	7.38 %	10.71 %	7.43 %

Table 2. Averaged relative $l_{2}$-norm error of different models’ prediction for the displacement field given boundary conditions.

Figure 9. predicted displacement field by HomoPNO and HeteroPNO models versus ground truth (DIC measurements).

Figure 10. Prediction of collagen fiber orientation field by the hetero-PNO II model versus the ground truth (experimentally detected orientation). The averaged absolute error is 6.59^o.

Figure 11. Predicted averaged first Piola-Kirchhoff axial stresses by the HomoPNO and HeteroPNO models versus the measured values (ground truth) for the 1198 test samples.

Figure 12. Predicted stress field (first Piola-Kirchhoff) by the HomoPNO and HeteroPNO models. Legend values are in KPa.

Figure 13. Learned influence states for HomoPNO and the two HeteroPNO models for the bio-tissue.

Learning of homogenized and heterogeneous PNOs.

For training our PNO models, we use a similar approach as in the previous example which is to first train a HomoPNO model, and then use the trained $\sigma^{NN}$ MLP as the initialization for $\sigma^{NN}$ in the HeteroPNO I and II. For HeteroPNO II, fiber angles are initialized as $90^{o}$ with random perturbation, as the collagen fibers in the specimen are known to have a “preferred direction” of $90^{o}$ from the biology of the cardiovascular tissue where the specimen is cut out.

In this example, Since the external forces are all zero, the setting falls under the category of Case II where (3.8) is used to minimize the displacement error and averaged axial stress errors (see 1). As mentioned in machine learning section, Case II is computational more demanding. Therefore we use smaller MLPs for this problem: The widths of the MLPs for $\omega^{NN}$ and $\sigma^{NN}$ are respectively: (2, 32, 64, 1) and (4, 64, 64, 1). Similar to the previous example the peridynamic horizon size is set as $\delta=3\Delta x$. In hyperparameter tuning performed during the training, $\beta$ in the loss function is tried with three different values: 0.2, 0.5, 0.8.

After training the baseline (Fung model) and the PNOs, we report the displacement errors on training validation and test sets in Table 2. As observed HomoPNO outperforms the baseline classical model by $35\%$, while the heterogeneous architectures improved the accuracy by another $25\%$ showing the significance of considering heterogeneity in modeling bio-tissues. The displacements predicted by PNOs versus the ground truth (DIC measurements) for a test sample are plotted in Figure 9. The discovered microstructure via HeteroPNO II model is plotted against the experimentally measured ones in Figure 10. The plotted normalized absolute error on the right is computed using the error measure in (4.1). The averaged predicted axial Piola-Kirchhoff stresses by PNO models are plotted for all 1,198 test samples against the experimentally measured values in Figure 11. As another important contribution of this work, we are able to provide the spatial stress fields for DIC displacements. Figure 12 presents the stress fields provided from PNO models for a test sample’s experimental displacements. HeteroPNO stresses reveal locations with significantly higher than average stress concentrations while they are absent in the HomoPNO stress plots. This concentration sites will be susceptible to tear and rupture much earlier that the rest of the tissue which is undetectable by homogeneous models. By comparing the HeteroPNO stresses and the fiber orientation field (Figure 7), it is understood that locations with rapid collagen fiber orientation transitions are susceptible to high stress concentration and possibly damage initiation.

Finally, the learned influence states of the PNO models are plotted in Figure 13. While the preferred direction of the fibers in specimen are known to be about $90^{o}$, we see a $45^{o}$ angle in the influence state of the HomoPNO model. The reason for this phenomenon is that the DIC data only contains biaxial tension instances, which makes the data blind to $\pm 45^{o}$ anisotropy. Since the data is indifferent to existence of fibers along $\pm 45^{o}$ direction, HomoPNO influence states can be affected and have larger values along $\pm 45^{o}$ direction. The HeteroPNO influence states however are not affected, because they learn a fundamental influence state for the real fiber angles (or learned fiber angles that minimize the errors). As such, they show the stiffer direction along $x-$axis corresponding to $\alpha=0$ as expected by design.

6. Summary and future directions

In this work, we have developed a novel neural operator architecture, which we call the Heterogeneous Peridynamic Neural Operator (HeteroPNO), for modeling the mechanical response of a biological tissue specimen and discovering its fiber orientation field. By learning a nonlocal constitutive law within the framework of a heterogeneous ordinary state-based peridynamics, our HeteroPNO features 1) the guarantee of fundamental physical laws (momentum conservation laws and objectivity); 2) generalizability to different domains, loading and boundary conditions; and 3) the disentanglement of the physical effects of material nonlinear response and heterogeneity. The heart of the architecture is a neural operator with two shallow neural networks: a nonlocal kernel $\omega^{NN}$ to capture the fiber orientation, and a nonlocal model form $\sigma^{NN}$ for the peridynamic scalar force state to capture the complex constitutive behavior.

Based on the HeteroPNO architecture, a data-driven computing workflow has been proposed, which learns the material model together with fiber orientation field directly from displacement/loading data pairs. As such, we integrate material identification and microstructure inference into one unified learning framework, and the learnt model is applicable to unseen loading conditions. To verify the method on soft tissue samples with unknown spatial heterogeneity, measurement noise, fiber-reinforced and nonlinear behaviors, we successfully modeled a porcine heart TVAL specimen using DIC measurement data collected from biaxial and constrained uniaxial tension tests. By learning the material heterogeneity together with constitutive laws, our numerical studies have demonstrated improved accuracy of displacement predictions relative to alternative methods: HeteroPNO has outperformed the homogenized Fung-type model by 50% and a homogeneous PNO model by over 25%. The inferred microstructure and stress field show good consistency with experimental measurements. These results suggest that even in a small and noisy training data regime (100 samples), the HeteroPNO model could build from DIC data a parametric description for the heterogeneous constitutive model that predicts the displacement field, the microstructure, and the local stress.

Despite the encouraging results presented here, questions and potential applications require further investigations. A natural future extension is the generalization of the model to other specimens with different material properties and microstructures. In the current work, the HeteroPNO model does not account for the load-dependent reorientation and realignment of the collagen fibers [57]. To capture these effects, one possible approach is to extend the model using transfer-learning techniques such as the meta-learning methods proposed in [90]. Translating the current trained model to whole organ simulations would be another interesting generalization problem. An important future step is to accelerate the solution procedure of the learnt nonlocal constitutive model, since the computational treatment of integral equations in peridynamics is often slower than with the classical PDE solvers. To achieve speedup, we plan to investigate the combination of the neural constitutive model and the neural solution operator, possibly with an encoder/decoder architecture.