Physics-Guided State-Space Model Augmentation Using Weighted Regularized Neural Networks

Abstract

Physics-guided neural networks (PGNN) is an effective tool that combines the benefits of data-driven modeling with the interpretability and generalization of underlying physical information. However, for a classical PGNN, the penalization of the physics-guided part is at the output level, which leads to a conservative result as systems with highly similar state-transition functions, i.e. only slight differences in parameters, can have significantly different time-series outputs. Furthermore, the classical PGNN cost function regularizes the model estimate over the entire state space with a constant trade-off hyperparameter. In this paper, we introduce a novel model augmentation strategy for nonlinear state-space model identification based on PGNN, using a weighted function regularization (W-PGNN). The proposed approach can efficiently augment the prior physics-based state-space models based on measurement data. A new weighted regularization term is added to the cost function to penalize the difference between the state and output function of the baseline physics-based and final identified model. This ensures the estimated model follows the baseline physics model functions in regions where the data has low information content, while placing greater trust in the data when a high informativity is present. The effectiveness of the proposed strategy over the current PGNN method is demonstrated on a benchmark example.

1 Introduction

Model-based design plays a crucial role in achieving satisfactory performance for complex dynamic systems by providing an interpretable framework that facilitates a deep understanding of system behaviors, including nonlinearities such as dam** and friction. However, the accurate and comprehensive system dynamics that can be modeled by first principle laws are often costly to obtain.

Nonlinear system identification (Schoukens and Ljung, 2019) is a well-established topic and can be characterized by a wide range of model classes such as state-space models (Schön et al., 2011), block-oriented models (Schoukens and Tiels, 2017), NARMAX (Billings, 2013), etc. Wherein, extensive research (Verdult, 2002; Paduart et al., 2010; Schön et al., 2018) on identification with nonlinear state-space (NLSS) models has shown its flexibility for handling multi-variable systems with potentially fewer parameters. Estimation of state-space models is advantageous for the subsequent control design, given the dependency of many nonlinear control methods on such representation of the system behavior.

Artificial neural networks (ANNs) have long been a focus of interest in the field of nonlinear system identification because of their high expressiveness, flexibility, and capability of approximating functions with arbitrary accuracy (Scarselli and Tsoi, 1998). In Suykens et al. (1995), recurrent neural networks have already been employed to represent a nonlinear state-space model. This structure is referred to as state-space neural network (SS-NN) and has been further discussed in (Amoura et al., 2011; Forgione and Piga, 2020; Schoukens, 2021; Beintema et al., 2023). Recently, Beintema et al. (2023) have introduced a computationally efficient nonlinear system state-space identification algorithm based on a subspace-encoder network (SUBNET). Nonetheless, ANNs are typically black-box models that lack physical interpretation, and exhibit poor generalization capabilities outside the training dataset, especially when the training data is limited. Hence, even though the ANNs may exhibit improved accuracy compared with first-principle modeling, deploying such models in practice or the controllers that are designed for them is simply dangerous.

To address this issue, physics-guided neural network (PGNN) (Karpatne et al., 2017) has been introduced, also within the field of systems and control (Bolderman et al., 2024), that ensures the interpretability and generalization capabilities of the estimated models. Compared with the ANNs, a physics-based cost function is incorporated into the optimization objective of PGNN, ensuring that the learned model not only achieves high accuracy on the training dataset, but also shows consistency with known physics laws on the unseen region without the need for large amounts of ground truth data.

However, there are some open technical issues with using PGNNs in nonlinear state space model identification. First, the classical PGNN does not perform the model augmentation, i.e., the prior model is only used to compute the physics-based regularization term in the cost function. Second, the classical PGNN penalizes the difference between the physics model and the identified model at the output level, which can lead to conservative estimation results. This is because systems with highly similar state-transition functions can have significantly different time-series outputs. Furthermore, the physics-based term of the classical PGNN regularizes the model difference over the entire state space, which makes this approach lose some flexibility, especially when the assumed prior model is inaccurate.

Motivated by these facts, this paper proposes an innovative PGNN-based state-space modeling strategy for nonlinear system identification, namely, W-PGNN, to efficiently complete prior physics-based state-space models with a weighted regularized SS-NN. The main contributions are as follows:

1) A new weighted-regularization cost function is designed to penalize the difference between both the state and output functions of the baseline physics-based and final identified models in regions where measured data provides little information.

2) Compared to the classical PGNNs, the proposed identification approach makes more extensive use of the pre-existing approximate model. The learned dynamics are capable of adhering to the data in regions with high information content, and preserving the behavior of the baseline physics model outside this region. This significantly enhances the flexibility of the SS-NN model.

The remainder of this paper is organized as follows. Section 2 introduces the nonlinear model class and the identification method with a state-space neural network. The classical PGNN method is discussed in Section 3. The proposed W-PGNN method is detailed in Section 4. Numerical simulation results are provided in Section 5, followed by the conclusions in Section 6.

Notation: $\mathbb{R}$ and $\mathbb{Z}$ denote the sets of real numbers and integers, respectively. The 2-norm of a vector or a matrix is denoted as $\|\cdot\|$. $\mathrm{vec}({x}_{1},...,{x}_{n})=[{x}_{1}^{\top}\ \cdots\ {x}_{n}^{\top}]^{\top}$ denotes the column-wise composition of vectors. $\mathcal{N}(0,1)$ is the standard normal distribution, while $\mathcal{U}(a,b)$ represents a uniform distribution with a support from $a$ to $b$.

2 Problem Statement

2.1 Nonlinear Model Class

Consider the following discrete-time state-space model as the data-generating system:

$$x(k+1) =f(x(k),u(k)),$$ (1)

$$y_{0}(k) =g(x(k),u(k)),$$

where $u(k)\in\mathbb{R}^{n_{\mathrm{u}}}$ denotes the input, $x(k)\in\mathbb{R}^{n_{\mathrm{x}}}$ is the state, $y(k)\in\mathbb{R}^{n_{\mathrm{y}}}$ is the output, and $k\in\mathbb{Z}$ represents the discrete time. Additionally, ${f}:\mathbb{R}^{n_{\mathrm{x}}\times n_{\mathrm{u}}}\to\mathbb{R}^{n_{\mathrm{x}}}$ and ${g}:\mathbb{R}^{n_{\mathrm{x}}\times n_{\mathrm{u}}}\to\mathbb{R}^{n_{\mathrm{y}}}$ are bounded deterministic vector functions. The training dataset $\mathcal{D}={\{({y}(k),{u}(k))\}_{k=1}^{N}}$ contains $N$ noisy outputs $y(k)=y_{0}(k)+v(k)$, collected from an experiment on (1), where the noise $v(k)$ is assumed to be a zero-mean random signal with finite variance independent from the input $u(k)$.

Assume that we only have access to an a priori known state-space model:

$$\tilde{x}(k+1) =\tilde{f}\left(\tilde{x}(k),u(k)\right),$$ (2)

$$\tilde{y}_{0}(k) =\tilde{g}\left(\tilde{x}(k),u(k)\right),$$

with the state and output $\tilde{x}(k)\in\mathbb{R}^{n_{\mathrm{x}}}$ and $\tilde{y}(k)\in\mathbb{R}^{n_{\mathrm{y}}}$ that has the same model order as (1). Note that the functions $\tilde{f}(\cdot)$ and $\tilde{g}(\cdot)$ constitute the physically well-interpretable and a priori know dynamics of the system (1), i.e., the nominal model. However, the prior model (2) does not accurately capture the true dynamics (1). For instance, there may exist local nonlinearities in certain regions, which are not able to be obtained by a rough identification or modeling based on first principles. Hence, it is essential to augment this a priori known model using newly measured data through nonlinear system identification.

2.2 State-Space Neural Network Identification

To this end, we consider the following nonlinear discrete-time state-space model of (1), which has the following structure:

$$\hat{x}(k+1) =\tilde{f}\left(\hat{x}(k),u(k)\right)+f_{\theta}\left(\hat{x}(k),u(k)\right),$$ (3)

$$\hat{y}(k) =\tilde{g}\left(\hat{x}(k),u(k)\right)+g_{\theta}\left(\hat{x}(k),u(k)\right),$$

where $f_{\theta}(\cdot)$ and $g_{\theta}(\cdot)$ are the completion functions that model the dynamics that cannot be captured reliably by the idealistic model (2), and are represented by fully connected feedforward neural networks with one hidden layer containing $n_{n}$ neurons and a linear output layer:

$$f_{\theta}(x(k),u(k))=W_{x}\phi\left(\left[W_{fx}W_{fu}\right]\left[\begin{array}[]{l}x(k)\\ u(k)\end{array}\right]+b_{f}\right)+b_{x}$$ (4)

where $\phi(\cdot)\in\mathbb{R}^{n_{\mathrm{u}}\times 1}$ denotes the activation function, $W_{x}$, $W_{fx}$, $W_{fu}$, $b_{f}$ and $b_{x}$ represent the weight and bias parameters of the neural network with proper dimensions, respectively. A similar representation is used for $g_{\theta}(x(k),u(k))$.

As discussed in Suykens et al. (1995) and Schoukens (2021), the state-space model (3) can be written in a specific form of a recurrent neural network, i.e., an SS-NN. The parameters $\theta$ for the SS-NN can be trained by optimizing the data-based cost function over $N$ samples:

$$V_{\mathrm{Data}}(\theta,N) =\frac{1}{N}\sum_{k=1}^{N}\left\|y(k)-\hat{y}(k|\theta)\right\|^{2}$$ (5)

$$\hat{\theta} =\arg\min_{\theta}V_{\mathrm{Data}}(\theta,N)$$

where $\hat{y}(k|\theta)$ is the simulated output of the model (3) given the parameter vector $\theta$. More detailed discussions of SS-NN are provided in Section 4.

From (5), it is obvious that the ANN simply learns the map** between system input and output data without considering any prior knowledge about the underlying physics. This makes it difficult for ANN to have good generalization performance outside of the training region, especially when the dataset is limited.

3 Classical PGNN for System Identification

In this section, we briefly introduce the concept of classical PGNN. Compared with the baseline ANN approach, there is an additional regularization term in the cost function to force the learnt model to follow the prior model even outside the training region.

The classical PGNN is trained by minimizing the following cost function:

$$V(\theta,N,\bar{N}) =V_{\mathrm{Data}}(\theta,N)+\gamma V_{\mathrm{Phy}}(\theta,\bar{N})$$ (6)

$$\hat{\theta} =\arg\min_{\theta}V(\theta,N,\bar{N})$$

where $V_{\mathrm{Data}}(\theta,N)$ is given by (5), and the physics-based penalized term $V_{\mathrm{Phy}}(\theta,\bar{N})$ is given by:

$$V_{\mathrm{Phy}}(\theta,\bar{N})=\frac{1}{\bar{N}}\sum_{k=1}^{\bar{N}}\left\|\tilde{\bar{y}}(k)-\hat{\bar{y}}(k|\theta)\right\|^{2}$$ (7)

where $\tilde{\bar{y}}(k)$ and $\hat{\bar{y}}(k|\theta)$ are the output response of the a priori known model (2) and the simulated model (3) given the regularization input signal $\bar{u}(k)$, respectively, and $\gamma\in\mathbb{R}_{>0}$ is the constant trade-off hyperparameter that balances between the data-fitting term and the regularization term in the overall cost. In this way, the prior model (2) is embedded in the trained ANN. Note that the physics-based cost $V_{\mathrm{Phy}}(\theta,\bar{N})$ does not rely on the measurement $y(k)$ from system (1). It is evaluated over a separate regularization dataset $\mathcal{D}_{\mathrm{Reg}}=\{\tilde{\bar{y}}(k),\bar{u}(k)\}_{k=1}^{\bar{N}}$ generated by the user using the baseline physics model (2).

As can be seen in (6), the penalization of the physics-guided part is at the output level. However, this can lead to a conservative result as systems with highly similar state-transition functions, i.e. only slight differences in parameters, can have significantly different time-series outputs. A good example of this is given by two mass spring damper systems with slightly different resonance frequencies, as shown in Fig. 1. Furthermore, the classical PGNN cost function regularizes the model estimate for the whole state space, which means that the a priori known model is assumed to hold equally for any unseen region. Though this feature enables the classic PGNN to have better generalization performance than the baseline NN, but in an ideal setting, we would like to trust the information in the data when a high informativity is present, while we would like to follow the prior model in regions where the data provides little information, i.e., to preserve the behavior of the a priori known model.

Figure 1: Frequency and time responses of two mass spring damper systems with only $10\%$ uncertainties in the system parameters. It can be observed that with a slight shift of resonance frequencies, the outputs are significantly different.

4 Weighted PGNN Method

4.1 Weighted function regularization

Unlike other model augmentation strategies, for instance, (Hoekstra et al., 2024), our approach aims to regularize state-space neural network estimation using a reference model and penalize the difference between physics and identified model at both the state and output levels. Moreover, the regularization should only be active in the regions where no data is present, i.e. the reference model prescribes the dynamics that the learned model should fall back to outside the training area.

The proposed approach starts by generating a surrogate input sequence $\bar{u}$ of length $\bar{N}$. It is worth noting that $\bar{u}$ is not applied to the true system during optimization to acquire output measurements, but plays a role in the regularization of the proposed approach. Ideally, $\bar{u}$ should cover the full range of operation of the system. Then, the model estimate is evaluated both applying $u$ and $\bar{u}$ on system (3), where the second input sequence results in the state sequence $\hat{\bar{x}}$.

Figure 2: Model structure of the physics-based SS-NN.

The cost function for the proposed W-PGNN is given by:

$$V(\theta,N,\bar{N})=V_{\mathrm{Data}}(\theta,N)+V_{\mathrm{Reg}}(\theta,w,\bar{N})$$ (8)

where the novel weighed regularization term is given by:

$$V_{\mathrm{Reg}}(\theta,w,\bar{N})\triangleq\frac{1}{\bar{N}}\sum_{j=1}^{\bar{N}}w_{j}\left(\gamma_{x}e^{x}_{j}+\gamma_{y}e^{y}_{j}\right)$$ (9)

where $e^{x}_{j}\triangleq\left\|f_{\theta}(\hat{\bar{x}}(j),\bar{u}(j))\right\|^{2}$, $e^{y}_{j}\triangleq\left\|g_{\theta}(\hat{\bar{x}}(j),\bar{u}(j))\right\|^{2}$. The weight vector $w\in\mathbb{R}^{\bar{N}\times 1}$ is defined as:

$$w_{j} \triangleq\frac{1}{\sum_{k=1}^{{N}}h_{k}(\bar{z}_{j})+\epsilon},$$ (10)

$$h_{k}(\bar{z}_{j}) =\exp\left(-\frac{\|\hat{z}_{k}-\bar{z}_{j}\|^{2}}{2\sigma^{2}}\right),$$

with the state-input pairs $\hat{z}_{k}=(\hat{x}(k),{u}(k))$ and $\bar{z}_{j}=(\hat{\bar{x}}(j),\bar{u}(j))$. Furthermore, $\hat{x}(k)$ and $\hat{\bar{x}}(j)$ denote the responses of the estimated model (3) to the training input $u$ and regularization input $\bar{u}$, respectively. $\sigma$ represents the center width of $h_{k}(\cdot)$, and $\epsilon$ is a small constant. Additionally, it is clear that if the weight $w_{j}$ is set to $1$ for all $j=1,...,\bar{N}$, then it is a classical PGNN with state-level regularization; if the weight $w_{j}$ is set to $0$ for all $j=1,...,\bar{N}$, then the cost function (8) will completely fall back to the baseline (5).

One can observe from (10) that, the further away the current regularization state-input pair $(\hat{\bar{x}}(j),\bar{u}(j))$ is from the training dataset, the larger the cost $V_{\mathrm{Reg}}(\theta,w,\bar{N})$ will be. This pushes $f_{\theta}$ and $g_{\theta}$ towards zero, consequently bringing the identified model closer to the prior model in that region of the joint input-state space. Additionally, the terms $e^{x}_{j}$ and $e^{y}_{j}$ share the same weight because both the state and the output function estimate depend on the same state-input pair. Note that the proposed cost function does not penalize the difference between the output of the a priori known model and the estimated model, but it rather penalizes the difference between both the state and output function of both models in regions where little information is provided by the measured data. As a consequence, the regularization state-input pair $(\hat{\bar{x}}(j),\bar{u}(j))$ should cover the full intended range of operation of the completed model for an effective regularized model augmentation through the proposed approach.

4.2 Implementation

The whole structure of the physics-based SS-NN is illustrated in Fig. 2. Specifically, the SS-NN architecture mainly comprises two components, namely the physics state/output layer and the state/output completion layer, where the physics state/output layer is employed to represent the prior known model $\tilde{f}(\cdot)$ and $\tilde{g}(\cdot)$ given in (2), and the state/output completion layer is utilized for estimating the unknown dynamics $f_{\theta}(\cdot)$ and $g_{\theta}(\cdot)$ given in (3). It is worth mentioning that the prior model (2) should have the same state dimension as the estimated model (3), which is a limitation of the proposed approach.

Training: The hyperparameters of the classical PGNN, $\gamma$, and the proposed W-PGNN, $\gamma_{x}$, $\gamma_{y}$, $\sigma$, $\epsilon$, are determined by grid searching on a validation dataset. Specifically, the selection of $\sigma$ depends on the density of data distribution, for instance, sparsely distributed data can necessitate choosing a larger $\sigma$. The weights and bias parameters $\theta={\mathrm{vec}}(W_{x},W_{fx},W_{fu}$ $,W_{y},W_{gx},W_{gu},b_{f},b_{x},b_{g},b_{y})$ of the SS-NN are trained by minimizing the cost function (8) via gradient-based approaches. Several optimization algorithms have been proposed to solve this problem, such as quasi-Newton (Fletcher and Powell, 1963) and conjugate gradients (Fletcher and Reeves, 1964) methods. In this paper, the Levenberg-Marquardt algorithm (Levenberg, 1944) is employed to find the minimum of (8). All the algorithms are implemented in the Matlab Deep Learning Toolbox and the Matlab Optimization Toolbox.

Initialization of model completion layer: Due to the use of the Levenberg-Marquardt optimization algorithm, an initial guess of the parameter values is required. We adopt the method in Schoukens (2021) to intuitively initialize the weight and bias parameters of the model completion layer, i.e., an explicit linear approximation is introduced:

$$f_{\theta}(x(k),u (k))=A_{\theta}x(k)+B_{\theta}u(k)$$ (11)

$$+\tilde{W}_{x}\phi\left(\left[\tilde{W}_{fx}~{}\tilde{W}_{fu}\right]\left[\begin{array}[]{l}x(k)\\ u(k)\end{array}\right]+\tilde{b}_{f}\right)+\tilde{b}_{x}$$

$$g_{\theta}(x(k),u (k))=C_{\theta}x(k)+D_{\theta}u(k)$$ (12)

$$~{}+\tilde{W}_{y}\phi\left(\left[\tilde{W}_{gx}~{}\tilde{W}_{gu}\right]\left[\begin{array}[]{l}x(k)\\ u(k)\end{array}\right]+\tilde{b}_{g}\right)+\tilde{b}_{y}$$

which leaves quite some flexibility in initializing the weights and biases of the nonlinear layers. Then, the weights and biases of the linear layer are initialized as $A_{\theta}=B_{\theta}=C_{\theta}=D_{\theta}=0$ and $\tilde{b}_{x}=\tilde{b}_{y}=0$. Additionally, the weights and biases of the nonlinear layer are initialized as $\tilde{W}_{x}=\tilde{W}_{y}=0$, and $\tilde{W}_{fx},\tilde{W}_{fu},\tilde{W}_{gx},\tilde{W}_{gu},\tilde{b}_{f},\tilde{b}_{g}$ are randomly initialized by $\mathcal{U}(-1,1)$. This chosen parameter initialization ensures that the initial model behaves like the a priori provided physics model. During the optimization, the weights $\tilde{W}_{x}=\tilde{W}_{y}$ will become nonzero, and this will activate the model completion part of the model.

5 Simulation Study

In this section, simulation results are presented to illustrate the effectiveness of our proposed W-PGNN approach. A 1-D example is conducted to validate the superior learning performance of the proposed W-PGNN approach over the baseline and classical PGNN approaches. Consider a SISO system:

$$x(k+1) =ax(k)+bu(k)+\Delta(x(k)),$$ (13)

$$y_{0}(k) =x(k)$$

with $a=0.8187$ and $b=0.1813$. The function $\Delta(x(k))$ is defined as:

$$\Delta(x)=0.2\left(e^{-\frac{x^{2}}{l^{2}}}-e^{-\frac{(x-c)^{2}}{l^{2}}}\right)$$ (14)

with $c=-0.3$, $l=0.2$, which represents the local nonlinearity that is not able to be expressed by the given baseline physics model. Then, the augmentation structure (3) is given in terms of the prior physics model $\tilde{f}(x(k),u(k))=ax(k)+bu(k)$, $\tilde{g}(x(k),u(k))=x(k)$ and the completion function $f_{\theta}(x)$ aimed to identify $\Delta(x(k))$ while $g_{\theta}=0$ in this case. Thus, the goal is to augment the prior model $\tilde{f}(x(k),u(k))$ with a well-estimated $f_{\theta}$ based on the proposed W-PGNN approach.

Figure 3: Illustration of the training, regularization, and test input and output signals used in the simulation.

Figure 4: Estimation results under the considered approaches in terms of $f_{\theta}(x)+ax$ (top), and the absolute value of estimation error of $f_{\theta}(x)$ over $x$ (bottom).

With this SISO system (13), the training input is selected as $u_{\mathrm{train}}(k)=\sin(0.15k)-0.2$ with $N=200$ samples to generate the training dataset $\mathcal{D}$. Furthermore, the regularization input signal is designed as a concatenation of signals $\bar{u}(k)=8\sin(0.2k)$ and $\bar{u}(k)=(8+(2/500)k)\mathcal{N}(0,1)$, each with a length of 500, respectively, leading to a $\mathcal{D}_{\mathrm{Reg}}$ with size $\bar{N}=1000$. In addition, the test input signal is selected as $u_{\mathrm{test}}(k)=\sin(0.01k+0.5)+\sin(0.02k-0.1)-2\sin(0.03k+0.2)$ with $N=500$ samples, which will explore a much larger region of input-output space than the training dataset. It is worth noting that only noise at the output of the system is present with SNR$\approx$40dB. The aforementioned signals are visualized in Fig. 3, which implies that the training dataset is significantly less informative than the test dataset. This is in line with the model augmentation philosophy of this work: as an adequate prior model is already in place, we only would like to augment this model using a simple dataset dedicated to a particular region.

To construct the NN model, the activation function is chosen as the radial basis function because of its universal approximation capability. A total of 20 neurons ($n_{n}=20$) are used in the state/output completion layer. Moreover, to determine the most suitable hyperparameters for classical PGNN and the proposed W-PGNN, a grid search is conducted on the validation dataset $\mathcal{D}_{\mathrm{Val}}$, which is generated by validation input signal $u_{\mathrm{Val}}(k)=1.08\sin(0.15k)-0.2$ for the classical PGNN, and $u_{\mathrm{Val}}(k)=u_{\mathrm{Train}}(k)$ for the W-PGNN. Both of them are 500 samples long. The results of the hyperparameter search are: $\gamma=10^{-3}$, $\gamma_{x}=\gamma_{y}=10^{-4}$, $\sigma=\sqrt{0.001}$, and $\epsilon=0.1$. Then all three approaches are trained on the obtained dataset $\mathcal{D}$ and $\mathcal{D}_{\mathrm{Reg}}$, of which the parameters $\hat{\theta}$ are optimized by the Levenberg-Marquardt algorithm, as mentioned in Section 4.2.

Figure 5: Test results for the augmented state-space model by the three approaches on the test dataset. The proposed W-PGNN shows the best estimation results both within and outside the training region.

The estimation results in terms of $f_{\theta}(x)+ax$, and the absolute value of estimation error $\Delta(x)-f_{\theta}(x)$ are depicted in Fig. 4, where the shaded area indicates the training data region and the black dots represent the linear prior model. It is clear that all three approaches are capable of capturing the true model well inside the training region, however, the baseline NN approach has poor generalization performance with the unseen data. Moreover, both the classical PGNN and the proposed W-PGNN approaches show good learning results outside the training region. However, the performance of the proposed W-PGNN approach is approximately 20% better compared to the classical PGNN (see also Table 1), mainly resulting from the novel weighted-regularization physics-based term in the cost function, which enables the learned model to follow the ground truth within the range of the training data, and in turn, be forced toward the linear prior model within the low-informative data area. This can also be seen in Fig. 5, where the zoom-in sub-figures show the estimation trajectories inside and outside the training region, respectively. It can be observed that despite the test dataset being much larger than the training dataset the proposed W-PGNN still has the capability of identifying the system in the whole state space with the highest estimation accuracy.

Furthermore, a Monte Carlo simulation with 10 runs under random initial parameters is conducted to compare the estimation error of the three approaches. To assess the simulation performance of the identified models, the following root mean squared error (RMSE) on the test dataset is utilized:

$$e_{\mathrm{RMSE}}=\sqrt{\frac{1}{N}\sum_{k=1}^{N}(y(k)-\hat{y}(k|\theta))^{2}}$$ (15)

Table 1: Quantitative evaluation of the performance of the three approaches in terms of their RMSE on the training and test dataset.

Approach	$e_{\mathrm{RMSE}}$($\times 10^{-2}$)
	Training set	Test set
Baseline	0.241$\pm$0.027	199.7$\pm$ 170.3
Classical PGNN	0.206$\pm$0.024	5.791$\pm$2.810
W-PGNN	0.209$\pm$0.021	$\mathbf{4.303\pm 1.503}$

Table 1 quantifies the RMSE and its variability of the three considered approaches on the training and test dataset over 10 runs. One can see that the achieved RMSE of the proposed W-PGNN significantly improves and shows better generalization performance on the unseen dataset compared to the baseline NN and classical PGNN.

6 Conclusion

A novel PGNN-based model completion strategy is proposed in this paper for nonlinear state-space model identification. Specifically, we enhance the interpretability and generalization performance of classical PGNN by introducing a weighted function regularization strategy, i.e., the W-PGNN. A new weighted regularization cost function is presented to penalize the difference between the physics and identified model at both the state and output levels in regions with low information content. The proposed strategy provides new perspectives into the fusion of physics-guided and black-box data-driven modeling approaches, especially in cases where the available data is limited. The effectiveness of W-PGNN has been analyzed and demonstrated by numerical simulations and compared with some classical ANN modeling methods. Future work will focus on extending the application scenarios of the proposed W-PGNN method to more complex and larger benchmarks.