BERP: A Blind Estimator of Room Acoustic and Physical Parameters for Single-Channel Noisy Speech Signals

Intelligibility parameters, including the STI and $\%\rm{AL}_{\rm{cons}}$, are used to predict speech intelligibility and assess verbal comprehension in a sound field.

Speech intelligibility index. The STI is employed to predict speech intelligibility and the corresponding listening difficulty in noisy surroundings. Houtgast and Steeneken initially defined the STI based on the modulation transfer function (MTF) [47, 48]. The higher the STI is, the more intelligible a sound field is. The STI can be calculated from the RIR as follows, which was standardized by IEC 60286-16:2020 [10].

First, an RIR $h(t)$ passes through seven octave-band filters to obtain the MTFs at 14 specific modulation frequencies from the corresponding temporal envelopes as:

where $i=1,2,...,7.$, $k=1,...,7$, $h_{i,\rm{oct}}$ denotes the RIR at each octave band, and $m_{k}(f_{m,i})$ represents the MTF for the $i$-th octave band at the $k$-th specific modulation frequency. The 14 specific modulation frequencies are $f_{m,i}(\rm{Hz})=\{0.63,\ 0.80,\ 1.00,\ 1.25,\ 1.60,\ 2.00,\ 2.50,\ 3.15,\ 4.00,\ 5.00,\\ \ 6.30,\ 8.00,\ 10.00,\ 12.50\}$. Then, the modulation distortion ratio is calculated as follows:

The transmission index at each octave band is normalized to the unit scale by limiting the range of $N_{k,i}$ relative to $15$ dB, which is determined as:

Finally, the STI is calculated as the weighted sum of $T(k,i)$:

where $\rm{w}=\{0.129,\ 0.143,\ 0.114,\ 0.114,\ 0.186,\ 0.171,\ 0.143\}$.

Percentage articulation loss of consonants. $\%\rm{AL}_{\rm{cons}}$ accounts for the measurement of incorrectly understood consonants, and this measure was originally introduced by Peutz and Kelin [11]. Since the STI does not account for the way in which a listener’s proficiency and linguistic knowledge affect the intelligibility, $\%\rm{A}_{\rm{cons}}$ assumes that consonants are harder to comprehend than vowels in a room. The utility of $\%\rm{A}_{\rm{cons}}$ extends the limitations of the STI by not discounting significant intelligibility-related information and considering linguistic proficiency. Moreover, its robustness against contamination from guessing makes it a great indicator of speech intelligibility [6]. Thus, $\%\rm{AL}_{\rm{cons}}$ emerges as an indispensable complement to the STI for comprehensively evaluating speech intelligibility within various room settings. $\%\rm{AL}_{\rm{cons}}$ can be steadily calculated from the STI according to Farrell Becker’s empirical formula [49]:

The reverberation time ($T_{60}$) and early decay time (EDT) are pertinent to the reverberation and quantify the subjective impression of the vivacity of a sound field. $T_{60}$ is the most essential RAP since it characterizes the physical properties of the RACs for which the reverberation energy is distributed within -$60$ dB. The EDT represents the decay time for the initial -$10$ dB to emphasize the more important contribution of early reflection to the perceived reverberation. $T_{60}$ is the 60-dB decay time calculated by line-fitting to the proportion of the energy decay curve (EDC) of the RIR from $-5$ dB to $-35$ dB and linearly extrapolating it to $-60$ dB. Similarly, the EDT is to line-fit the initial $-10$ dB of the EDC and to extrapolate to -$60$ dB.

Clarity ($C_{50}$ and $C_{80}$), definition ($D_{50}$), and center time ($T_{s}$) are the energy parameters used to measure the energy ratio of the RIR between the energy contributed from early reflections and late reverberation. They are strongly related to the impression of transparency.

Clarity. $C_{80}$ and $C_{50}$ express the logarithmetic ratio of the energy within the first $50$ ms for speech and that within the first $80$ ms for music to the remaining RIR, thereby characterizing the clarity perception of a speech or music signal traversing within a room. Clarity can be defined as:

where $t_{e}$ denotes $50$ or $80$ ms, respectively.

Definition. $D_{50}$ indicates the subjective intelligibility of speech in a room, which is defined as the ratio of the energy received within $50$ ms to the total energy of the RIR.

Center time. $T_{s}$ refers to “the center of gravity time”, characterizing the balance between clarity and reverberation that is related to speech intelligibility. $T_{s}$ is given by:

The geometric room volume $V$ is a position-independent parameter for modeling the attributes of a room. $V$ is strongly related to the estimation of the critical distance ($D_{c}$), which is the distance from the sound source at which the energy density of the reverberant signal is equal to that of the direct signal [27]. $D_{c}$ can be approximated using Sabine’s formula:

where $\varrho$ signifies the source directivity factor, and A represents the equivalent absorption area of a room. $D_{c}$ is vital for determining whether a virtual sound source should be rendered with reverberation, thereby serving as a key distance cue for the perception of reverberation by the listener [27, 33].

Furthermore, the mixing time used in AR rendering applications [5] can be determined from $V$ as $t_{m}=\sqrt{V}$. Jot et al. [5] identified room volume as the reverberation fingerprint to characterize rooms for spatial AR rendering. $V$ also plays an important role in the speech intelligibility [6]. The critical distance of intelligibility ($D_{ci}$), which acts as a distance cue for perceived intelligibility, is derived from $V$ as:

$\%\rm{AL}_{\rm{cons}}$ also exhibits a strong relationship with $V$, which can be alternatively expressed as[6]:

$D$ is the sound source distance, and $\mathfrak{c}$ is the correction factor.

The sound source distance $D$ contributes significantly to complementing the sound source localization (SSL) by integrating it with the DOA of the sound source [29]. The SSL is widely used in applications such as sound source separation [3], audio-oriented and navigational systems [8], speech-related applications [30], and human-robot interaction [26]. Furthermore, $D$ is intimately related to the perception of speech intelligibility, particularly in terms of $\%\rm{AL}_{\rm{cons}}$, as elaborated in Eq. (11).

As mentioned in Section II-B2, the DOA is a crucial component of SSL [30], and it has several applications in sound source separation [4], speech recognition [20], speech enhancement [9], and room acoustical analysis [7]. In this work, the DOA is represented by a pair including an azimuth ($\theta$) and elevation ($\psi$) and is denoted as $\rm{DOA}\coloneqq\{\theta,\psi\}$.

The detection of the instantaneous occupancy level of room $N$ around a listener’s location is highly useful for several applications. It is commonly known that the number of occupants affects the reverberation [50], thus affecting the efficacy of demand-driven hearing aid systems and speech enhancement methods. Additionally, the interference speeches generated by the occupants around the listener affect the target signals that the listener intends to receive. Knowing the occupancy level helps control interference to achieve intelligible and clear transmission.

In the context of smart homes, the estimated number of occupants can optimize the control of demand-driven heating, ventilation, and air conditioning (HVAC) operations in the local space to significantly reduce the cost of building operations for sustainable smart buildings [51, 52]. In the XR and AR scenarios, the local occupancy level, as an important factor in environmental information factor, is fundamental for ensuring safe interaction in real-world scenes, especially in public spaces populated by others.

The observed noisy reverberant signal, as perceived by a listener while transmitting from a speaker within a room and subject to the influence of the background environmental noise, can be formulated as:

where $y_{\rm{nr}}(t)$ denotes the noisy reverberant signal as perceived by the listener, $h(t)$ denotes the RIR, and $n(t)$ represents the background noise that is prevalent in the listener’s local surroundings. The symbol $``*"$ denotes the convolution operation.

The $y_{\rm{nr}}$ encapsulates the RIR information that fully characterizes the RACs in the listener’s local space, including RAPs and room volume. In addition, it contains information pertaining to sound source localization, such as the distance, azimuth, and elevation. Therefore, this signal model is instrumental for parameterizing the listener’s local acoustic space, encompassing aspects such as the volume, the distance and DOAs of the sound source, and RAPs. The noisy reverberant signal model is employed to model the real-world scenarios in which a listener interacts with a single speaker in the presence of environmental noise, as illustrated in Fig. 2.

Currently, the research domain lacks a comprehensive reverberant speech corpus for crowded environments that can enable the estimation of the occupancy level around the listener, which encompasses complete meta-information, including the number of speakers, the spatial geometry of speaker distribution relative to the listener, and the local RACs where the listener occupies. We introduce a novel signal model that incorporates these detailed meta-information to address this gap, as shown in Fig 2.

This signal model can be expressed as:

where $y_{\rm{cr}}$ signifies the crowded reverberant speech signal, $x_{i}(t)$ represents the speech signal originating from the $i$-th speaker proximal to the listener, and $d_{i}$ denotes the distance between the $i$-th speaker and the listener, which adheres to a Gaussian distribution. $A_{0}$ represents the baseline amplitude observed at a distance of $d_{0}$ from the listener, and $h(t)$ denotes the RIR that delineates the acoustic characteristics of the local room. Here, $d_{0}$ is assumed to be equal to 1. $N$ represents the total count of speakers, i.e., the occupancy level, according to a gamma distribution, which is well-suited for modeling real-life events that yield only positive results.

When develo** this speech signal model, a set of fundamental assumptions are postulated. These assumptions are instrumental for enabling an approximation that closely mirrors real-world scenarios while effectively mitigating the intricacies embedded within the observed speech signal, thereby devising a theoretically sound and practically viable model.

Within the scope of dynamic blind RAPs and RPPs estimation, our access is confined to an observed noisy reverberant signal. Hence, we model the observed signal as in Eq. (12). The ill-posed nature of blind estimation necessitates an RIR model to approximate an unknown RIR for serving as a bridge between the sound source signal and the perceived noisy reverberant signal.

Moreover, to reduce the computational complexity and to facilitate their simultaneous estimation, it is more efficient to model the RIR and subsequently estimate the parameters of this RIR model. This approach enables the simultaneous derivation of the RAPs from the modeled RIR instead of directly estimating them from the noisy reverberant signal.

The RIR can be categorized as the isolated (early reflections) and dense room modes (late reverberation), respectively, by applying modal theory to the room frequency response [55], i.e., the Fourier transform of the RIR, by using Schroeder’s frequency [56].

Badeau [55] introduced a unified mathematical framework for stochastically modeling the RIR, according to the image-source principle [27], as shown in Fig. 3. This work reported that the image sources (i.e., reflections) are distributed according a uniform Poisson distribution. The author further demonstrated that this stochastic distribution of image sources remains invariant regardless of the sound source’s and the receiver’s locations. Alternatively, based on billiard theory, Polack [57] showed that the Poisson distribution of image sources is also independent of the room geometry. Additionally, Traer and McDermott [40] analyzed the RIR statistics. They found that, during the time interval of dense room modes, the RIR exhibits a Gaussian distribution; this was in stark contrast to the time interval of isolated room modes, which exhibited a non-Gaussian distribution.

Drawing inspirations of [57, 40, 55], we present a novel stochastic RIR model, namely, the sparse stochastic impulse response (SSIR) model. This model combines the different stochastic properties of the isolated and dense room modes of the RIR. Specifically, the time interval of the isolated room modes is dominated by uniform Poisson-distributed image sources with their sparsity proportional to the room volume as $h_{\rm{i}}(t)\sim\boldsymbol{P}(\lambda\lvert V\rvert)$. Conversely, the time interval related to the dense room modes presents a Gaussian distribution as $h_{\rm{d}}(t)\sim\boldsymbol{N}(0,1)$. Here, $h_{\rm{i}}(t)$ and $h_{\rm{d}}(t)$ represent the isolated and dense room modes of the RIR, respectively, and $V$ denotes the room volume. Fig. 4 shows the fitting of the proposed SSIR model to the realistic RIR.

The SSIR model can be defined as:

where $T_{i}$ and $T_{d}$ are two parameters that control the exponentially ascending and descending temporal envelopes of the RIR, respectively. The constant $\alpha=6.9$ is known as Schroeder’s coefficient [58], and $\lambda$, which is equal to $\mu$, signifies the average of $T_{i}$ across the sample set. Here, $\mu$ is empirically determined to be $0.0399$.

We aggregated five extensive realistic RIR datasets to construct a composite RIR dataset for representing a wide range of room geometries and RACs. These datasets are the Arni RIR dataset [59], the Motus dataset [60], the BUT ReverbDB [61], the ACE corpus of RIRs [37], and the OpenAIR dataset [62]. Each dataset comprises monochanneled and omnidirectionally recorded RIRs. We resampled all RIRs to $16$ kHz.

To replicate the background environmental noise encountered in real-world scenarios, instead of using synthetic white Gaussian noise, we employ the actual noise samples from real-world daily life circumstances. We integrate noise signals from the DEMAND [63] and BUT[61] noise datasets, both of which are collected in real-world daily life environments and resampled to $16$ kHz.

We use the LibriSpeech corpus [64] for sampling the sound source speech signals when synthesizing the observed reverberant signals. Specifically, we select a 360-hours clean subset. This subset is composed of more than 100,000 unique clips articulated by 921 speakers with completely distinct linguistic contents. The deployment of this dataset ensures a broad spectrum of diverse speech signals, enhancing the robustness and generalizability of our synthesized signals.

In the composited RIR dataset with detailed annotations of $T_{i}$, $T_{d}$; the parameters of the SSIR RIR model; and metrics related to the room volume $V$, the sound source distance $D$ and DOA {$\theta$, $\psi$} of the sound source, we further employ data augmentation strategy. The strategy involves data upsampling and downsampling techniques to modulate the distribution of the labels, which mitigates potential biases in the data distributions for obtaining more natural distributions. The degrees of upsampling and downsampling are calibrated based on the relative rarity of the values of each label. After the data augmentation process is applied to the RIR dataset, a comprehensive collection of $47,430$ realistic RIRs is successfully compiled. This RIR dataset contains a wide range of RIRs, for which the corresponding $T_{60}$ spans from $0.18$ to $8.00$ sec.

Then, we randomly sampled 47,430 clips from the LibriSpeech corpus, choosing clips with the most common length (from $12$ to $17$ sec.) to be the sound source speech signals, regardless of the speaker information and linguistic content that they contain. In parallel, noise signals are randomly sampled, following an independent and identically distributed (I.I.D.) pattern, from the DEMAND and BUT datasets. Then, in accordance with Eq. (12), we synthesize the noisy reverberant speech signals. To enhance the robustness and efficacy of the model across diverse noisiness environments, the signal-to-noise ratio (SNR) between the reverberant and noise signals is uniformly varied by adjusting the SNR at five different levels, ranging from 0 dB to 20 dB in 5 dB increments, including a scenario with no noise (Inf). Given the uniqueness of each clip, we guarantee that every synthesized speech signal maintains its individuality in terms of both its waveform and linguistic content, further augmenting the diversity and richness of the synthesized dataset.

Initially, we apply voice activity detection to the LibriSpeech corpus to segment and annotate the timestamps corresponding to speech and silence segments. This process underlies the annotations of the synthesized crowded reverberant signals. Then, the gamma distribution of the occupancy levels is modeled. Explicitly, in rooms of varying volumes, the occupancy level $N$ follows a Gaussian distribution $N\sim\textit{{N}}(N,1)$, accompanying with the real-world principle that larger spaces typically accommodate more occupants, while smaller spaces accommodate fewer occupants.

The Gaussian mixture distribution is used to approximate the gamma distribution of the occupancy level, as detailed in Eq. (13). Furthermore, the distribution of the distance $d_{i}$ from the $i$-th occupant speaker to the listener is governed by a Gaussian distribution $d_{i}\sim\textit{{N}}(\mu_{d},1)$, where $\mu_{d}$ represents the mean of the maximum and minimum distances. In accordance with Assumption 1 (Sec. III-A2), $\mu_{d}$ is set to 2.5.

Using Eq. (13), we synthesize the crowded reverberant speech signals by superimposing the speech signals uniformly sampled from the LibriSpeech corpus, aligning with the annotated speech and silence segmentation. The initiation index for each overlap** speech signal is determined based on an I.I.D. pattern. Additionally, to authentically replicate the local room acoustics, the room volumes in the RIRs are precisely matched with their corresponding RIRs, thereby ensuring a realistic acoustic environment. Finally, we obtain a dataset comprising 47,430 samples of crowded reverberant speech signals, ranging from 10 to 25 sec. Fig. 6 shows an example of a crowded reverberant speech signal and its corresponding occupancy level according to Eq. (13) and the aforementioned synthesis strategy.

We use the three types of featurization methods to represent the observed input signals, including Gammatonegram, MFCC, and mel spectrogram. A Gammatonegram emphasizes the importance of low-frequency sections while a signal propagates within a room [65, 34]. While the MFCC characterizes the shape of the spectral envelope of a reverberant signal, closely related to the MTF of room acoustics [58]. The mel spectrogram rather mimicks human subjective perceptions to the RACs.

We use a room feature encoder (RFE) to learn room feature representations.

Room Feature Encoder. The RFE is structured into eight blocks, each block comprising four components. It incorporates a half-residual feedforward network, a multiheaded self attention, a convolutional network, and another half-residual feedforward network [66].

This encoder integrates the CNNs and transformer models, both of which account for gras** local and global acoustic features, respectively, since Wang et al. [28] showed that the acoustical information spreads the overall frequency components of the reverberant signal. Such integration makes it particularly well suited for learning the sophisticated map**s between the noisy and crowded reverberant speech signals with complex waveforms and the desired room parameters.

The signal flow from the input feature representation $\mathbcal{x}_{i}$ to the latent variable output $\mathbcal{y}_{i}$ across each block is mathematically expressed as:

where FFN denotes the feedforward network, MHSA denotes the multiheaded self attention, Conv denotes the convolutional network, and LayerNorm represents the layer normalization operation, respectively.

Feedforward network. The feedforward network is composed of a layernorm layer, linear layers with 2048 hidden and 512 embedding dimensions with the swish activation function [67], and a dropout layer of $0.1$ dropout rate. Fig. 7 visualizes the architecture of this module.

Multiheaded self attention. The multiheaded self attention with extrapolatable relative positional encoding (xPos) enhances the ability of the model to grasp the global comprehensive acoustical information encapsulated in feature representations [68]. Fig. 8 shows the corresponding architecture. The xPos encoding strategy has been empirically validated to augment the stabilization and robustness of the self attention mechanism, particularly for sequences with various length.

The xPos-based relative self attention can be formulated as follows:

where $\boldsymbol{Q}_{\mathbcal{x},\rm{xPos}}=(\textbf{W}_{q}\mathbcal{C}+\mathfrak{R}_{\boldsymbol{Q}}\mathbcal{S})\mathbcal{T}$, $\boldsymbol{K}_{\mathbcal{x},\rm{xPos}}=(\textbf{W}_{k}\mathbcal{C}+\mathfrak{R}_{\boldsymbol{K}}\mathbcal{S})\mathbcal{T}^{-1}$, and $\boldsymbol{V}_{\mathbcal{x}}=\textbf{W}_{v}\mathbcal{x}$. $\mathbcal{C}$ is equal to $\cos(m\vartheta_{i})$ and $\mathbcal{S}$ is equal to $\sin(m\vartheta_{i})$, which are the cosine and sine positions at the embedding dimension $i$ and the time slice $m$, respectively. $\mathfrak{R}$ corresponds to the rotary matrix of $\boldsymbol{Q}$ and $\boldsymbol{K}$. $\mathbcal{D}_{h}$ is the head dimension of the attention mechanism. “T” denotes transposition. $\mathbcal{T}=\varsigma_{m,i}$. The $\varsigma_{i}$ is given by:

where $\beta$ is the optimal setting and $\vartheta_{i}=10000^{-2i/\mathbcal{D}_{h}}$. $\textbf{W}_{q}$, $\textbf{W}_{k}$, $\textbf{W}_{v}$ and $\boldsymbol{M}$ are trainable weighting matrices of query, key, and value of the attention mechanism, respectively.

Convolutional network. The convolutional network functions in capturing the local acoustic features and reinforcing the temporal causality of the feature representation. This module leverages prenorm residual connection with gating mechanisms to distill the important acoustical characteristics via pointwise and depthwise convolutional and a gated linear unit (GLU) layers [69], as illustrated in Fig. 9.

For the regression task of room parameter estimation, we employ a parametric predictor (PP) solely for $T_{i}$, $T_{d}$, $V$, $D$, and utilize both PP and an acoustical bias corrector (ABC) for $\theta$, and $\psi$.

Parametric Predictor. The PP employs several convolutional layers with ReLU activation functions to compose a nonlinear regression function, allowing us to utilize the encoded representations within the latent space to predict the physically-meaningful room parameters. These parameters includes the two parameters of the SSIR model $\hat{T}_{i}$ and $\hat{T}_{d}$, the room volume $\hat{V}$, the sound source distance $\hat{D}$, and DOA of the sound source $\hat{\theta}$ and $\hat{\psi}$. The behavior of the predictor can be mathematically determined as follows:

where $\boldsymbol{\gamma}$ denotes the room parameters output from the PP, which is a constant function alongside the time axis. During inference, we sum up and average the values over the time axis to obtain a single predicted room parameter. The overall architecture of the PP is presented in Fig. 10.

Acoustical Bias Corrector. The ABC acts as a gating mechanism to differentiate between the biased and unbiased data encoded within the latent space, thereby directing the optimal signal flow into the PP and ensuring that the PP can learn from the unbiased data distribution. Additionally, we leak some biased data into the PP to make it robust to biases. The necessity of such a mechanism arises in the context of room parameters such as the sound source azimuth $\theta$ and elevation $\psi$, whose distributions exhibit the substantial inherent biases that are difficult to mitigate through conventional data augmentation techniques, as shown in Fig. 11. The biases often lead to the regression of trivial results, i.e., the mean of the whole distribution.

The ABC comprises a sandwich structure, characterizing the rotary-positional self attention[70] as a feature enhancer to assign the different attention weights to all latent spectro-temporal feature representation in a frame-by-frame manner. The rotary-position encoding approach also contributes to stabilizing the training process. It is also adaptable to variable latent input lengths without the need for alignment. The output bias probability $p_{\rm{abc}}(\mathbcal{\hat{y}})$ of the ABC can be mathematically expressed as follows:

Fig. 12 depicts the entire architecture of the ABC. “GELU” and “Sigmoid” denote the GELU [71] and sigmoid activation functions, respectively.

A FC layer is engineered to regress the encoded feature representations to physics-informed instantaneous occupancy levels as a time sequence derived from the observed crowded signals. The pivot of this architecture is substantiated by Assumption 2 (Section III-A2), which facilitates forecasting a time series from a regression task to a classification task, thus simplifying the complexity of the task while improves the robustness of the prediction. Considering that the occupancy level exhibits no significant temporal dependence, we instead simply adopt a linear layer with log-softmax activation function to predict the occupancy level rather than recurrent or autoregressive structures. The resolution of estimation process is about $62.5$ Hz, i.e., $16$ ms per frame, for predicting the occupancy level around the listener’s location.

We employ the Huber loss [72] to optimize the PPs across each targeted room parameter. The Huber loss of the PPs is defined as:

where $\gamma$ is the targeted room parameter and $\delta$ is set to 1. The symbol $\mathcal{N}$ denotes the batch size, and $\mathcal{K}$ represents the time frame length. The Huber loss possesses the dual sensitivity of the minimum-variance estimation by the $\mathcal{L}$2 loss and the robustness of the median-aware estimation against outliers by the $\mathcal{L}$1 loss. It also circumvents the convergence problem of the $\mathcal{L}$1 loss on a small scale [73] and contributes to preventing exploding gradients by clip** gradients exceeding $\delta$.

Considering the prediction task of the ABC is binary, i.e., distinguishing between unbiased and biased data, we adopt the binary cross-entropy (BCE) for optimization, which is defined as follows:

where $y^{\rm{abc}}_{n}$ denotes the ground-truth label presenting acoustical bias, and $p_{\rm{abc}}(\hat{y}_{n})$ denotes the predicted bias probability output from the ABC.

The occupancy module utilizes the cross-entropy (CE), reflecting the multiclass nature of the occupancy level estimation task. This loss function is determined as follows:

and $\mathcal{C}$, the upper bound of the occupancy level, is set to 12 according to Assumption 2 (detailed in Section III-A2). Here, $\hat{y}^{crowd}_{n,c}$ represents the logits output from the FC, $p_{\rm{crowd}}$ denotes the output probability after softmax function, and $N_{n,c}$ denotes the ground-truth instantaneous occupancy level.

When estimating the sound source azimuth and elevation, the ABC is deployed to counteract significant bias inherent within the data distribution. Nevertheless, a critical issue arises from the disparate gradient descent rates of the two employed loss functions (the Huber and BCE losses). The gradient descent rate for the BCE loss is significantly faster than that for the Huber loss, which causes the training instability, specifically when the BCE loss approaches overfitting whereas the Huber loss still remains underfitting.

Therefore, we introduce a new loss function, namely, the see-saw loss, to solve this disparity. This loss function can adaptively balance the gradient descent rates of BCE and Huber losses, thus stabilizing the training process. The see-saw loss function devised for DOA estimation is formulated as follows:

where $\mathcal{L}_{\rm{see-saw}}(\theta;\psi,\hat{\theta};\hat{\psi})$ denotes the total loss. The components $\mathcal{L}^{\rm{az}}_{\rm{corr}}$ and $\mathcal{L}^{\rm{elev}}_{\rm{corr}}$ represent the BCE losses of the azimuth and elevation through the ABC, respectively. $\mathcal{L}_{\rm{pred}}(\theta,\hat{\theta})$ and $\mathcal{L}_{\rm{pred}}(\psi,\hat{\psi})$ correspond to the Huber losses for the azimuth and elevation using PPs. $\mathfrak{w}^{\prime}_{\rm{corr}}$ and $\mathfrak{w}^{\prime\prime}_{\rm{corr}}$ are the weight coefficients of the bias correctors. $\mathfrak{w}_{\rm{pred}}$, $\mathfrak{w}^{\prime}_{\rm{pred}}$, and $\mathfrak{w}^{\prime\prime}_{\rm{pred}}$ are the corresponding weight coefficients of the predictors.

We introduce a loss function that combines polynomial losses with see-saw loss for the joint estimation framework.

The polynomial see-saw loss $\mathcal{L}_{\rm{unified}}$ is formulated as follows:

where $\mathfrak{w}_{T_{i}}$, $\mathfrak{w}_{T_{d}}$, $\mathfrak{w}_{V}$, and $\mathfrak{w}_{D}$ are weighting coefficients for losses $\mathcal{L}_{\rm{pred}}(T_{i},\hat{T}_{i})$, $\mathcal{L}_{\rm{pred}}(T_{d},\hat{T}_{d})$, $\mathcal{L}_{\rm{pred}}(V,\hat{V})$, and $\mathcal{L}_{\rm{pred}}(D,\hat{D})$, respectively. The weighting ratio is arranged as: $\mathfrak{w}_{T_{i}}:\mathfrak{w}_{T_{d}}:\mathfrak{w}_{V}:\mathfrak{w}_{D}:\mathfrak{w}^{\prime}_{\rm{corr}}:\mathfrak{w}^{\prime\prime}_{\rm{corr}}:\mathfrak{w}_{\rm{pred}}:\mathfrak{w}^{\prime}_{\rm{pred}}:\mathfrak{w}^{\prime\prime}_{\rm{pred}}=5.0:5.0:5.0:5.0:0.1:0.1:0.5:10.0:1.0$.