Quantifying Spuriousness of Biased Datasets Using
Partial Information Decomposition

Abstract

Spurious patterns refer to a mathematical association between two or more variables in a dataset that are not causally related. However, this notion of spuriousness, which is usually introduced due to sampling biases in the dataset, has classically lacked a formal definition. To address this gap, this work presents the first information-theoretic formalization of spuriousness in a dataset (given a split of spurious and core features) using a mathematical framework called Partial Information Decomposition (PID). Specifically, we disentangle the joint information content that the spurious and core features share about another target variable (e.g., the prediction label) into distinct components, namely unique, redundant, and synergistic information. We propose the use of unique information, with roots in Blackwell Sufficiency, as a novel metric to formally quantify dataset spuriousness and derive its desirable properties. We empirically demonstrate how higher unique information in the spurious features in a dataset could lead a model into choosing the spurious features over the core features for inference, often having low worst-group-accuracy. We also propose a novel autoencoder-based estimator for computing unique information that is able to handle high-dimensional image data. Finally, we also show how this unique information in the spurious feature is reduced across several dataset-based spurious-pattern-mitigation techniques such as data reweighting and varying levels of background mixing, demonstrating a novel tradeoff between unique information (spuriousness) and worst-group-accuracy.

¹ University of Maryland College Park   ² Google Research   ³ Princeton University

1 Introduction

Spurious patterns (Haig, 2003) arise when two or more variables are correlated in a dataset even though they do not have any causal relationship. For example, in the Waterbird dataset (Wah et al., 2011), most waterbirds have water backgrounds, and landbirds have land backgrounds (see Fig. 1). This correlation in the dataset essentially misleads a machine learning model into creating a spurious link between background and bird type, since it often finds the background to be “more informative” than the foreground for predicting the bird type. Learning such spurious links from the data may result in high performance on the training and in-distribution datasets, but results in reduced performance on out-of-distribution datasets and affects worst-group-accuracy (Lynch et al., 2023; Sagawa et al., 2019), i.e., the accuracy on the minority groups like waterbirds with land background or vice versa.

Several existing works (Kirichenko et al., 2022; Izmailov et al., 2022; Wu et al., 2023; Ye et al., 2023; Liu et al., 2023) focus on different dataset-based and model-training-based approaches to mitigate spurious patterns and evaluate the empirical performance over out-of-distribution datasets (or, to improve worst-group-accuracy). However, this notion of spuriousness in any given dataset lacks a formal definition. This work addresses this gap by asking the question: Given a split between core and spurious features, how do we formally quantify the spuriousness in any given dataset?

To answer this question, we present an information-theoretic formalization of spurious patterns, by leveraging a body of work in information theory called Partial Information Decomposition (PID) (Bertschinger et al., 2014; Banerjee et al., 2018). We note that classical information-theoretic measures such as mutual information (Cover & Thomas, 2012) captures the entire statistical dependency between two random variables but fail to capture how this dependency is distributed among those variables, i.e., the structure of the multivariate information. Partial Information Decomposition (PID) addresses this nuanced issue by providing a formal way of disentangling the joint information content between the core and spurious features into non-negative terms, namely, unique, redundant, or synergistic information (see (2.1) in Section 2.1).

Figure 1: Spuriousness in the dataset due to sampling bias.

Our proposition is to use the unique information about the target variable $Y$ in the spurious features $B$ that is not in the core features $F$ as a measure of spuriousness in the dataset (often denoted as $\mathrm{Uni}({Y{:}B|F})$. To justify our proposition, we discuss how unique information is connected to Blackwell Sufficiency (Blackwell, 1953), a notable concept in statistical decision theory. Blackwell Sufficiency provides a partial ordering on when one random variable can be more “informative” (less noisy) than another for inference. Unique information captures the departure from Blackwell Sufficiency, which goes to zero if and only if one random variable is Blackwell Sufficient over another for a prediction task (see Theorem 1). Thus, unique information intuitively quantifies when one variable can be more informative than another, which we leverage to explain when the spurious feature $B$ can be more informative than the core feature $F$ for the model prediction. Additionally, we also show several desirable properties of unique information as a measure of spuriousness in the dataset in Theorem 2. Though Partial Information Decomposition (PID) has recently been applied to few other areas in machine learning (Tax et al., 2017; Dutta et al., 2020, 2021; Hamman & Dutta, 2024a; Liang et al., 2023; Dutta & Hamman, 2023) (also see Related Works), we are pioneering its use to decompose information in spurious and core features and quantify spuriousness, supported by desirable properties and empirical validation. Our main contributions can be concisely listed as follows:

• Novel information-theoretic formalization to explain spurious patterns: Though many works attempt to prevent a model from learning spurious patterns, there is a lack of a theoretical understanding of the “amount” of spuriousness in a dataset, and how do we quantify and measure it given a split of spurious and core features. Novel to this work, we investigate spuriousness through the lens of partial information decomposition (PID) and provide a fundamental understanding of when a model finds the spurious features to be “more informative” than the core features. We leverage PID to disentangle the joint information content between the core and spurious features into unique, redundant, and synergistic information.

• Demystifying unique information as a measure of spuriousness: Next, we propose unique information in the spurious features $\mathrm{Uni}({Y{:}B|F})$ as a measure of the spuriousness in a dataset. To justify our proposition, we first establish how unique information $\mathrm{Uni}({Y{:}B|F})$ quantifies the informativeness of a random variable $B$ compared to $F$ for predicting $Y$ (see Theorem 1 for a motivation from Blackwell Sufficiency). Depending on the increasing or decreasing nature of the unique information $\mathrm{Uni}({Y{:}B|F})$, one can then anticipate to what extent is a model going to leverage $B$ over $F$ for prediction. Additionally, we also show several desirable properties of unique information $\mathrm{Uni}({Y{:}B|F})$ as a measure of spuriousness in Theorem 2. Our measure can identify which features are more likely to be predictive for a classification task, paving a pathway for dataset quality assessment and explaining feature-based informativeness.

• Spuriousness Disentangler: An autoencoder-based estimator for computing unique information: We propose a novel autoencoder-based framework that we call – Spuriousness Disentangler – to compute the PID values for high dimensional image data. The estimator consists of mainly three main parts: (i) First, an autoencoder reduces the dimension of the image data and gives an one-dimensional array of clusters which serves as a lower-dimensional, discrete feature representation for the image data. Along the lines of (Guo et al., 2017), the dimensionality reduction and clustering are efficiently performed through minimization of a joint loss function; (ii) Next, the computation of the joint probability distribution of this lower-dimensional representation is performed; and (iii) Finally, the partial information decomposition (PID) values are calculated by solving a convex optimization problem using the Discrete Information Theory (DIT) package (James et al., 2018).

• Experimental Results and Novel Tradeoff: Our experimental results are in agreement with our theoretical postulations, demonstrating an empirical tradeoff between our proposed measure of spuriousness, i.e., $\mathrm{Uni}({Y{:}B|F})$ and empirical evaluation metrics known to be affected by spurious patterns, i.e., worst group accuracy. We show that for real-world unbalanced datasets, e.g., the Waterbirds dataset (Wah et al., 2011), the unique information in the spurious feature $\mathrm{Uni}({Y{:}B|F})$ is the most prominent and is significantly higher than any information in the core features. This helps explain why a model trained on this dataset readily uses the spurious feature rather than the core feature for prediction. Additionally, when a dataset-based spurious-correlation-mitigation method such as data-reweighting is applied, the unique information in the spurious features $\mathrm{Uni}({Y{:}B|F})$ reduces drastically (again explaining why a model might now be more likely to use the core feature $F$). We also observe a novel tradeoff between unique information $\mathrm{Uni}({Y{:}B|F})$ (proposed measure of spuriousness) and worst-group-accuracy for varying degrees of background mixing (a form of noise), i.e., the worst-group-accuracy improves with the decreasing unique information in the spurious features pointing to a novel tradeoff. We also study Grad-CAM (Selvaraju et al., 2017) (a technique to generate ’visual explanations’ for decisions made by Convolutional Neural Network (CNN)-based models) visualizations for many of the trained models to further confirm when the core or spurious feature is actually being emphasized by the model for different experimental setups.

Related Works: There are several perspectives on spurious correlation (see Haig (2003); Kirichenko et al. (2022); Izmailov et al. (2022); Wu et al. (2023); Ye et al. (2023); Liu et al. (2023); Stromberg et al. (2024); Singla & Feizi (2021); Moayeri et al. (2023) and the references therein; also see surveys (Ye et al., 2024; Srivastava, 2023; Ghouse et al., 2024)). Spuriousness mitigation techniques are broadly divided into two groups: (i) Dataset-based techniques (Kirichenko et al., 2022; Wu et al., 2023) and (ii) Learning-based techniques (Liu et al., 2023; Yang et al., 2023; Ye et al., 2023). Kirichenko et al. (2022) shows that last-layer fine-tuning of a pre-trained model with a group-balanced subset of data is sufficient to mitigate spurious correlation. Wu et al. (2023) proposes a concept-aware spurious correlation mitigation technique. Ye et al. (2023) introduces a Freeze and Train approach to learn salient features in an unsupervised way and freezes them before training the rest of the features via supervised learning. Yang et al. (2023) explores different regularization techniques to see the effect on the spurious correlation and  Liu et al. (2023) examines a logit correction loss. Our novelty lies in formalizing the spuriousness of datasets using the PID framework, and explaining how effective a dataset-based spurious-correlation mitigation will be for regular model training.

Partial information decomposition (PID) (Williams & Beer, 2010; Bertschinger et al., 2014; Dutta et al., 2021; Venkatesh & Schamberg, 2022) is an active area of research. PID measures are beginning to be used in different domains of neuroscience and machine learning (Tax et al., 2017; Dutta et al., 2020, 2021; Hamman & Dutta, 2024a; Ehrlich et al., 2022; Liang et al., 2024; Wollstadt et al., 2023; Mohamadi et al., 2023; Venkatesh et al., 2024; Hamman & Dutta, 2024b). However, examining spurious correlation through the lens of PID and observing novel empirical tradeoffs between the spurious pattern and worst-group-accuracy is unexplored. Additionally, there is limited work on calculating PID values for high dimensional multivariate continuous data. Some existing works (Dutta et al., 2021; Venkatesh & Schamberg, 2022; Venkatesh et al., 2024) handle continuous data with Gaussian assumptions while (Pakman et al., 2021) considers one-dimensional multivariate case. Hence, estimating PID for high-dimensional data by proper dimensionality reduction and discretization is unexplored.

For dimensionality reduction, different learning based methods exist (Hotelling, 1933; Law & Jain, 2006; Lee & Verleysen, 2005; Wang et al., 2015, 2014). Similarly, for discretization, different clustering algorithms exist, e.g., k-means clustering (MacQueen et al., 1967; Bradley et al., 2000), deep embedded clustering (Xie et al., 2016). Along the lines of an autoencoder-based clustering setup in (Guo et al., 2017), our proposed Spuriousness Disentangler trains a network to jointly learn a good representation of the input image data in a self-supervised way ensuring low representation error while also clustering simultaneously to deal with the challenge of high dimensional and continuous image data.

2 Preliminaries and Background

Let $X=(X_{1},X_{2},\ldots,X_{d})$ be the random variable denoting the input (e.g., an image) where each $X_{i}\in\mathcal{X}$ which denotes a finite set of values. The core features (e.g., the foreground) will be denoted by $F\subseteq X$, and the spurious features (e.g., the background) will be denoted by $B=X\backslash F$. We typically use the notation $\mathcal{B}$ and $\mathcal{F}$ to denote the range of values for the spurious and core features. Let $Y$ denote the target random variable, e.g., the true labels which lie in the set $\mathcal{Y}$, and the model predictions are given by $\hat{Y}=m_{\theta}(X)$ (parameterized by $\theta$). Generally, we use the notation $P_{A}$ to denote the distribution of random variable $A$, and $P_{A|B}$ to denote the conditional distribution of random variable $A$ conditioned on $B$. Depending on the context, we also use more than one random variable as sub-script, e.g., $P_{ABY}$ denotes the joint distribution of $(A,B,Y)$. Whenever necessary, we also use the notation $Q_{A}$ to denote an alternate distribution on the random variable $A$ that is different from $P_{A}$. We also use the notation $P_{A|B}\circ P_{B|C}$ to denote a composition of two conditional distributions given by: $P_{A|B}\circ P_{B|C}(a|c)=\sum_{b\in\mathcal{B}}P_{A|B}(a|b)P_{B|C}(b|c)\ \forall a\in\mathcal{A},\ c\in\mathcal{C},$ where $\mathcal{A}$, $\mathcal{B}$ and $\mathcal{C}$ denote the range of values that can be taken by random variables $A$, $B$, and $C$.

2.1 Background on Partial Information Decomposition

We provide a brief background on PID that would be relevant for the rest of the paper. The classical information-theoretic quantification of the total information that two random variables $A$ and $B$ together hold about $Y$ is given by the mutual information $\mathrm{I}({Y;A,B})$ (see (Cover & Thomas, 2012) for a background on mutual information). Mutual information $\mathrm{I}({Y;A,B})$ is defined as the KL divergence (Cover & Thomas, 2012) between the joint distribution $P_{YAB}$ and the product of the marginal distributions $P_{Y}\otimes P_{AB}$ and would go to zero if and only if $(A,B)$ is independent of $Y$. Intuitively, this mutual information captures the total predictive power about $Y$ that is present jointly in $(A,B)$ together, i.e., how well can one learn $Y$ from $(A,B)$ together. However, $\mathrm{I}({Y;A,B})$ only captures the total information content about $Y$ jointly in $(A,B)$ and does not unravel anything about what is unique and what is shared between $A$ and $B$.

Figure 2: PID of $\mathrm{I}({Y;A,B})$: $\mathrm{I}({Y;A,B})$ is decomposed into four nonnegative terms, namely, unique information in $A$ ($\mathrm{Uni}({Y{:}A|B})$), unique information in $B$ ($\mathrm{Uni}({Y{:}B|A})$), redundant information in both ($\mathrm{Red}({Y{:}A,B})$), and synergistic information in both ($\mathrm{Syn}({Y{:}A,B})$).

PID (Bertschinger et al., 2014; Banerjee et al., 2018) provides a mathematical framework that decomposes the total information content $\mathrm{I}({Y;A,B})$ into four nonnegative terms (also see Fig. 2):

$$\mathrm{I}({Y;A,B}) =\mathrm{Uni}({Y{:}B|A})+\mathrm{Uni}({Y{:}A|B})$$

$$+\mathrm{Red}({Y{:}A,B})+\mathrm{Syn}({Y{:}A,B}).$$ (1)

Here, $\mathrm{Uni}({Y{:}B|A})$ denotes the unique information about $Y$ that is only in $B$ but not in $A$. Next, $\mathrm{Red}({Y{:}A,B})$ denotes redundant information (common knowledge) about $Y$ in both $A$ and $B$. Lastly, $\mathrm{Syn}({Y{:}A,B})$ is an interesting term that denotes the synergistic information that is present only jointly in ${A,B}$ but not in any one of them individually, e.g., a public and private key can jointly reveal information not in any one of them alone.

Motivational Example. Let $Z{=}(Z_{1},Z_{2},Z_{3})$ with each $Z_{i}{\sim}$ i.i.d. Bern(1/2). Let $A=(Z_{1},Z_{2},Z_{3}\oplus N)$, $B=(Z_{2},N)$, and $N\sim$ Bern(1/2) which is independent of $Z$. Here, $\mathrm{I}(Z;A,B)=3$ bits. The unique information about $Z$ that is contained only in $A$ and not in $B$ is effectively in $Z_{1}$, and is given by $\mathrm{Uni}({Z{:}A|B})=\mathrm{I}({Z;Z_{1}})=1$ bit. The redundant information about $Z$ that is contained in both $A$ and $B$ is effectively in $Z_{2}$ and is given by $\mathrm{Red}({Z{:}A,B})=\mathrm{I}(Z;Z_{2})=1$ bit. Lastly, the synergistic information about $Z$ that is not contained in either $A$ or $B$ alone, but is contained in both of them together is effectively in the tuple $(Z_{3}\oplus N,N)$, and is given by $\mathrm{Syn}({Z{:}A,B}){=}\mathrm{I}({Z;(Z_{3}\oplus N,N)})=1$ bit. This accounts for the $3$ bits in $\mathrm{I}({Z;A,B})$.

We also note that defining any one of the PID terms suffices for obtaining the others. This is because of another relationship among the PID terms as follows (Bertschinger et al., 2014): $\mathrm{I}({Y;A})=\mathrm{Uni}({Y{:}A|B})+\mathrm{Red}({Y{:}A,B})$. Essentially $\mathrm{Red}({Y{:}A,B})$ is viewed as the sub-volume between $\mathrm{I}({Y;A})$ and $\mathrm{I}({Y;B})$ (see Fig. 2). Hence, $\mathrm{Red}({Y{:}A,B})=\mathrm{I}({Y;A})-\mathrm{Uni}({Y{:}A|B})$. Lastly, $\mathrm{Syn}({Y{:}A,B})=\mathrm{I}(Y;A,B)-\mathrm{Uni}({Y{:}A|B})-\mathrm{Uni}({Y{:}B|A})-\mathrm{Red}({Y{:}A,B})$ (can be obtained from (2.1) once both unique and redundant information has been defined). Here, we include a popular definition of $\mathrm{Uni}({Y{:}A|B})$ from (Bertschinger et al., 2014) which is computable using convex optimization.

Definition 1 (Unique Information (Bertschinger et al., 2014)).

Let $\Delta$ be the set of all joint distributions on $(Y,A,B)$ and $\Delta_{P}$ be the set of joint distributions with the same marginals on $(Y,A)$ and $(Y,B)$ as the true distribution $P_{YAB}$, i.e., $\Delta_{P}=\{Q_{YAB}{\in}\Delta$: $Q_{YA}=P_{YA}$ and $Q_{YB}=P_{YB}\}$. Then,

$$\mathrm{Uni}({Y{:}A|B})=\min_{Q\in\Delta_{P}}\mathrm{I}_{Q}({Y;A|B}).$$

Here $\mathrm{I}_{Q}({Y;A|B})$ denotes the conditional mutual information when $(Y,A,B)$ have joint distribution $Q_{YAB}$ instead of $P_{YAB}$.

3 Main Results

In this work, we first present an information-theoretic formalization of spurious patterns using the mathematical framework of Partial Information Decomposition (PID).

Proposition 1 (Unique Information as a Measure of Spuriousness).

For a given data distribution, the unique information $\mathrm{Uni}({Y{:}B|F})$ is a measure of spuriousness given a split of the spurious features $B$ and core features $F$.

Figure 3: Blackwell Sufficiency

To justify our proposition, we first establish that unique information is a measure of informativeness of the spurious feature $B$ over core feature $F$. We draw upon a concept in statistical decision theory called Blackwell Sufficiency (Blackwell, 1953) which investigates when a random variable is “more informative” (or “less noisy”) than another for inference (also relates to stochastic degradation of channels (Venkatesh et al., 2023; Raginsky, 2011)). Let us first discuss this notion intuitively when trying to infer $Y$ using two random variables $F$ and $B$. Suppose, there exists a transformation on $F$ to give a new random variable $B^{\prime}$ which is always equivalent to $B$ for predicting $Y$. We note that $B^{\prime}$ and $B$ do not necessarily have to be the same since we only care about inferring $Y$. In fact, $B$ and $B^{\prime}$ can have additional irrelevant information that do not pertain to $Y$, but solely for the purpose of inferring $Y$, they need to be equivalent. Then, feature set $F$ will be regarded as “sufficient” with respect to $B$ for predicting $Y$ since $F$ can itself provide all the information that $B$ has about $Y$ (see Fig. 3). This intuition is formalized as:

Definition 2 (Blackwell Sufficiency (Blackwell, 1953)).

A conditional distribution $P_{F|Y}$ is Blackwell sufficient with respect to another conditional distribution $P_{B|Y}$ if and only if there exists a stochastic transformation (equivalently another conditional distribution $P_{B^{\prime}|F}$ with both $B$ and $B^{\prime}\in\mathcal{B}$) such that $P_{B^{\prime}|F}\circ P_{F|Y}=P_{B|Y}$.

Figure 4: Spuriousness Disentangler: We propose autoencoder-based estimator for PID to handle high dimensional continuous image data. The left side denotes the clustering part where soft-level $q$ (clusters) is optimized by training a deep neural network consisting of encoder-decoder part with an objective to minimize loss $L$. The right side denotes the segmentation of one image into background (spurious features) and foreground (core features) followed by the clustering. Then, the joint distribution is estimated which is used to have the final estimation of PID values.

Now we demonstrate how our proposed unique information is closely tethered to Blackwell Sufficiency, thus justifying our Proposition 1. In fact, the unique information $\mathrm{Uni}({Y{:}B|F})$ is $0$ if and only if $P_{F|Y}$ is Blackwell sufficient with respect to $P_{B|Y}$ (see Theorem 1).

Theorem 1 (Spuriousness and Blackwell Sufficiency).

The $\mathrm{Uni}({Y{:}B|F})=0$ if and only if the conditional distribution $P_{F|Y}$ is Blackwell sufficient with respect to $P_{B|Y}$.

Since spuriousness (unique information) $\mathrm{Uni}({Y{:}B|F})=0$ if and only if $P_{F|Y}$ is Blackwell Sufficient with respect to $P_{B|Y}$, we note that $\mathrm{Uni}({Y{:}B|F})>0$ captures the “departure” from Blackwell Sufficiency, and thus quantifies relative informativeness. Intuitively, what this means is that for the given data distribution, there is no such transformation on core feature $F$ that is equivalent to the spurious feature $B$ for the purpose of predicting $Y$. This essentially makes spurious feature $B$ indispensable to the model for predicting $Y$, forcing the model to use or emphasize it in decision-making.

Next, we discuss some desirable properties of unique information $\mathrm{Uni}({Y{:}B|F})$.

Theorem 2.

The measure $\mathrm{Uni}({Y{:}B|F})$ satisfies the following desirable properties:

• $\mathrm{Uni}({Y{:}B|F})\leq\mathrm{I}({Y;B})$ and is $0$ if $\mathrm{I}({Y;B})=0$ (spurious feature $B$ has no information about $Y$).

• $\mathrm{Uni}({Y{:}B|F})$ is non-decreasing if more features are added to $B$, i.e., if the set of spurious features grows, so does its unique information over core features.

• $\mathrm{Uni}({Y{:}B|F})$ is non-increasing if more features are added to $F$, i.e., if the set of core features grow, the unique information in the spurious features reduce.

Spuriousness Disentangler (Autoencoder-based estimator): Next, we propose an autoencoder-based estimation framework – that we call Spuriousness Disentangler – to calculate the PID values. The motivation to use this estimator is that since the model learns the features to reconstruct the input image, the encoding of the image should have minimal information loss and hence should be a good low-dimensional representation of the input image. The framework mainly consists of three aspects: clustering, estimation of joint distribution and estimation of PID.

Since we are dealing with high dimensional data, dimesionality reduction is a necessary first step (Bellman, 1966). Traditionally, the clustering step is done by PCA followed by k-means clustering. However, in our setting, we can do these two steps together using an autoencoder, which is a deep neural network consisting of an encoder and a decoder, as shown in Fig. 4. The output of the encoder is the embedding for the input image, a low dimensional representation of the input images. The weights of this output layer, defined as the clustering layer, are used as the clusters centers initialized by k-means clustering algorithm. The clustering layer is optimized using the weighted sum of representation loss $L_{r}$ and clustering loss $L_{c}$. The overall loss function is defined as $L=L_{r}+\gamma L_{c}$ where $\gamma$ is a non-negative constant. The clustering loss $L_{c}$ is the KL divergence which measures the dissimilarity between different distributions  (Xie et al., 2016; Guo et al., 2017). For cluster centers ${\{\mu_{j}}\}^{K}_{1}$ and embedded point $z_{i}$ (output of the encoder), $q_{ij}$ is defined as follows (Van der Maaten & Hinton, 2008):

$$q_{ij}=\frac{(1+\|z_{i}-\mu_{j}\|^{2})^{-1}}{\sum_{j}(1+\|z_{i}-\mu_{j}\|^{2})^{-1}}$$ (2)

where $q_{ij}$ is the $j$th entry of the soft label $q_{i}$, denoting the probability of $z_{i}$ belonging to cluster $\mu_{j}$. The loss $L_{c}=KL(P||Q)=\sum_{i}\sum_{j}p_{ij}log\frac{p_{ij}}{q_{ij}}\quad\text{and}\quad p_{ij}=\frac{\frac{q^{2}_{ij}}{\sum_{i}q_{ij}}}{\sum_{j}(\frac{q^{2}_{ij}}{\sum_{i}q_{ij}})}$ where $P$ is the target distribution.

Figure 5: The first two columns show a significant drop in spuriousness, i.e., unique information in background (Uniq-B) when the dataset is changed from unbalanced to balanced form (balancing removes the spurious correlation between background and label). The last column depicts improvement in the worst-group (W.G.) and mean accuracy (%) when the dataset is balanced. Here the ’teal’ color is for unbalanced (WG-Un, Mean-Un) and ’coral’ colored bar is for balanced (UG-Ba, Mean-Ba) dataset.

The representation loss is the mean square error between the input of the encoder $x$ and output of the decoder $x^{\prime}$ defined as $L_{r}=\|x-x^{\prime}\|^{2}_{2}$.

The next step is to estimate the PID values. For this, the joint distribution of three random variables (e.g. the clusters of foreground, background and the binary label) is calculated using histograms, and then the PID values are obtained from the DIT package (James et al., 2018).

4 Experiments

We demonstrate experimental results to provide evidence in support of Proposition 1 for different experimental setups, i.e., unbalanced, balanced, and mixed background datasets. We illustrate how unique information in the spurious features has a tradeoff with the worst-group-accuracy, thus justifying its use as a measure of the spuriousness of a dataset. We also show a comparative analysis for PCA-based and autoencoder-based PID measurements.

Datasets: We conduct experiments on two datasets: Waterbird (Wah et al., 2011) and Dominoes (Shah et al., 2020), both framed as binary classification tasks.

Waterbird dataset (Wah et al., 2011) is the popular spurious correlation benchmark. The task is to classify the type of the birds (waterbird $=1$, landbird $=0$). However, there exists spurious correlation between the backgrounds (water $=1$, land $=0$) and the labels (bird type). Group$00$, Group$01$,Group$10$, and Group$11$ indicate the group of images where landbirds are in the land backgrounds, landbirds are in the water backgrounds, waterbirds are in the land backgrounds and waterbirds are in the water backgrounds respectively. We call the bird as the foreground of the image.

Dominoes is a synthetic dataset created by combining handwritten digits (zero and one) from MNIST (Deng, 2012) and images of cars and trucks from CIFAR$10$ (Krizhevsky et al., 2009) (digit $0$ or $1$ at the top, car ($=0$) or truck ($=1$) at the bottom of an image). We make two version of this synthetic dataset namely Dominoes $1.0$ and Dominoes $2.0$ inducing different degrees of bias. The task is to classify whether the image contains a car or truck hence the car or truck corresponds to the core features (foreground). On the other hand, the digits are considered as the spurious features (background). Group$00$, Group$01$,Group$10$, and Group$11$ illustrate the group of images where the top half is a zero and bottom half is a car, the top half is a one and bottom half is a car, the top half is a zero and bottom half is a truck, the top half is a one and bottom half is a truck, respectively.

4.1 Comparison between group-balanced and unbalanced datasets

We observe the relationship between the PID values and worst-group-accuracy for (i) an unbalanced dataset (which has spurious correlations) and (ii) a balanced dataset where the spurious correlation with the background is removed through sampling (balancing).

Problem Setup: We use group-balanced and unbalanced data for this part of the experiment. The balanced-unbalanced scenario arises from the four different groups that are present in the dataset, where the majority groups consist of the waterbirds with water backgrounds and landbirds with land backgrounds and other two combinations are the minority groups for the waterbird dataset. Similarly, in the Dominoes dataset, cars with digit $0$ and trucks with digit $1$ are the majority groups and the other two combinations are the minority groups. Worst-group-accuracy refers to the accuracy for the minority group which is generally the lowest for the model that is trained with biased dataset namely unbalanced dataset. The group-balanced dataset has equal number of samples in each group resulting in unbiased model training. We begin with using our autoencoder-based estimator, namely Spuriousness Disentangler, on both dataset and estimate the PID values separately for the background and foreground. This separation is done by using the segmentation mask of the foreground for the waterbird dataset. Next, we fine-tune the pre-trained ResNet-$50$ (He et al., 2016) model and calculate the worst-group-accuracy and mean accuracy over all groups.

Figure 6: Examples of Grad-CAM images: (a),(c) for the model trained with group-unbalanced dataset and (b),(d) for the model trained with group-balanced dataset. Observe that for the unbalanced dataset, the model adds more emphasis (red regions) to the background while in the balanced case, the foreground is more emphasized.

Figure 7: Samples of background mixed dataset (first row is for addition and second row corresponds to concatenation).

Figure 8: This bar-plot shows the distribution of the redundant information (R), unique information in background (Uniq-B) and foreground (Uniq-F) and Synergistic information (Syn) for the unbalanced, concatenation and addition dataset. Observe that the Uniq-B decrease for both addition and concatenation dataset compared to that of unbalanced dataset (Note that the scales are different for the visibility of small values).

Observations: Fig. 5 shows our findings regarding PID values and the worst-ground accuracy for three datasets. Firstly, we can observe that the unique information in background is generally much higher than the other PID values namely unique information in foreground, redundancy and synergy. Secondly, from the first two columns, it is obvious that there is a significant reduction of the unique information in background i.e., reduction in spuriousness when the dataset is balanced (having equal number of samples in all groups reducing the bias in dataset) and all other PID values are now in the same order. Next, from the last column of Fig. 5, we find out that the worst-group-accuracies are lower for the unbalanced case and these values significantly improve when the datasets become balanced which implies low spuriousness in the dataset. Finally, Fig. 6 shows through the Grad-CAM (Selvaraju et al., 2017) images that when the dataset is balanced, the model emphasizes more on the core features namely, waterbird or landbird for waterbird dataset and car or truck for the Dominoes dataset (the red regions) while in the unbalanced dataset the background is more highlighted which results in poor worst-group-accuracy.

4.2 Tradeoffs for varying levels of background mixing

Next, we look into the datasets for varying levels of background mixing to observe the tradeoffs between the spuriousness and the worst-group-accuracy.

Figure 9: Showing the tradeoffs between spuriousness i.e., unique information in background (Uniq-B) and worst-group-accuracy (%) for varying levels of background mixing. The worst-group-accuracy is decreasing with the Uniq-B.

Problem Setup: Starting with the dataset creation, we add two backgrounds at different levels. We consider two cases: (i) half of a land background is concatenated with half of a water background (named as concatenation); and (ii) the whole image of a land background is summed with a water background (named as addition). Similar techniques are applied for background mixing for Dominoes dataset (see Fig. 7). Then, the foreground is superimposed on the background. Next, the PID values are calculated for the mixed background and the foreground using our estimator. We train the pre-trained ResNet-$50$ (He et al., 2016) with the mixed background with foreground (the whole image) and evaluate the model with the normal test dataset (without any modification). One motivation of mixing the backgrounds is to remove the group bias that is generated due to the correlation between the background and the label in the dataset, that should help mitigate the spurious correlation since the background is no longer different for different groups.

Observations: Firstly, in Fig. 8 we can observe that unique information in the background is prominent in the unbalanced case and it decreases for both addition and concatenation scenarios which indicates spuriousness reduction while using addition and concatenation datasets. Next, we observe a trend in Fig. 9 between the unique information in background i.e., spuriousness and the worst-group-accuracy: with increasing unique information in background i.e., spuriousness, the worst-group-accuracy decreases. This trend is obtained for unbalanced, addition and concatenation datasets (lowest W.G. Acc. for unbalanced and highest W.G. Acc. for concatenation).

Table 1: Worst-Group-Accuracy(%) for different datasets

Dataset	Unbalanced	Balanced	Addition	Concatenation
Waterbird	29.75	86.60	89.88	92.99
Dominoes 1.0	90.73	91.42	94.665	96.45
Dominoes 2.0	79.79	86.94	85.51	87.35

Table 2: PID values for different variations of datasets

Dataset	Unbalanced				Balanced
	Redundancy	Uniq- B	Uniq - F	Synergy	Redundancy	Uniq - B	Uniq - F	Synergy
Waterbird	0.005677	0.166927	7.75E-07	0.018373	0.002635	0.000127	0.00909	0.02325
Dominoes 1.0	0.015414	0.172779	3.18E-09	0.006822	0.000296	0.000213	0.001261	0.01548
Dominoes 2.0	0.029422	0.56187	6.00E-06	0.006134	0.014792	0.246192	9.81E-07	0.022527
Dataset	Concatenation				Addition
	Redundancy	Uniq -B	Uniq -F	Synergy	Redundancy	Uniq- B	Uniq- F	Synergy
Waterbird	0.000162	0.000069	0.005487	0.014036	0.000375	0.000096	0.005302	0.015623
Dominoes 1.0	0.000949	0.000006	0.01443	0.006292	0.000933	0.000036	0.020326	0.007635
Dominoes 2.0	0.000103	0.000003	0.047737	0.009618	0.000141	0.000014	0.042616	0.007464

In the Table 2, the PID values i.e. redundant information, unique information in the background (Uniq-B), unique information in the foreground (Uniq-F) and synergistic information are demonstrated for all three datasets and all variants of the datasets: Unbalanced, balanced, concatenation and addition. Table 1 shows the worst-group-accuracy for all types of datasets. Observe that, the worst-group-accuracy is minimum for the unbalanced dataset and maximum for the concatenation dataset.

5 Conclusion

Quantifying and explaining spuriousness of a dataset can provide an efficient way to assess dataset quality rather than training a model for hours. In this work, we theoretically quantify spuriousness in a dataset with unique information, leveraging the mathematical tool of Partial information decomposition (PID). We demonstrate (with empirical validation) that unique information in the background can measure spuriousness and relate it to the worst-group-accuracy for various spurious correlation mitigation techniques. We also propose a novel autoencoder-based estimator for high-dimensional continuous image data, showing its superiority over classical estimators. However, there are some limitations: firstly to estimate the unique information, at first one has to identify the spurious features and core features of a given dataset which is not always straightforward. Moreover, the estimation is highly data-dependent. A small change in the dataset can greatly affect the PID values. Nonetheless, formally quantifying spuriousness can lead to more effective bias mitigation strategies.

Appendix A Proof of Theorem 1

As a proof sketch, we first derive the following lemma.

Lemma 1.

$\mathrm{Uni}({Y{:}B|F})=0$ if and only if there exists a row-stochastic matrix $T\in[0,1]^{|\mathcal{F}|\times|\mathcal{B}|}$ such that: $P_{YB}(Y=y,B=b)=\sum_{f\in\mathcal{F}}P_{YF}(Y=y,F=f)T(f,b)$ for all $y\in\mathcal{Y}$ and $b\in\mathcal{B}$.

Proof.

If $\mathrm{Uni}({Y{:}B|F})=0$, then we have: $\min_{Q\in\Delta_{P}}\mathrm{I}_{Q}({Y;B|F})=0$ where $\Delta_{P}=\{Q{\in}\Delta$ : $Q_{YF}(Y=y,F=f)=P_{YF}(Y=y,F=f)$ and $Q_{YB}(Y=y,B=b)=P_{YB}(Y=y,B=b)\}$. Thus, there exists a distribution $Q\in\Delta_{P}$ such that $Y$ and $B$ are independent given $F$ under the joint distribution $Q$. Then, we have

$$P_{YB}(Y=y,B=b) =Q_{YB}(Y=y,B=b)$$ (3)

$$=\sum_{f\in\mathcal{F}}Q_{YFB}(Y=y,F=f,B=b)$$ (4)

$$=\sum_{f\in\mathcal{F}}Q_{B|YF}(B=b|Y=y,F=f)Q_{YF}(Y=y,F=f)$$ (5)

$$\overset{(a)}{=}\sum_{f\in\mathcal{F}}Q_{B|YF}(B=b|Y=y,F=f)P_{YF}(Y=y,F=f)$$ (6)

$$\overset{(b)}{=}\sum_{f\in\mathcal{F}}Q_{B|F}(B=b|F=f)P_{YF}(Y=y,F=f)$$ (7)

$$\overset{(c)}{=}\sum_{f\in\mathcal{F}}T(f,b)P_{YF}(Y=y,F=f).$$ (8)

Here, (a) holds because $P_{YF}=Q_{YF}$ for all $Q\in\Delta_{P}$, (b) holds because under joint distribution $Q$, variables $Y$ and $B$ are independent given $F$, and (c) simply chooses $T(f,b)=Q_{B|F}(B=b|F=f)$ which is a function of $(f,b)$ and will lead to a row-stochastic matrix $T$ since $\sum_{b\in\mathcal{B}}T(f,b)=\sum_{b\in\mathcal{B}}Q_{B|F}(B=b|F=f)=1.$

Next, we prove the converse. Suppose, such a row-stochastic matrix $T$ exists such that:

$$P_{YB}(Y=y,B=b)=\sum_{f\in\mathcal{F}}T(f,b)P_{YF}(Y=y,F=f).$$

Now, we can define a joint distribution $Q^{*}$ such that:

$$Q^{*}(Y=y,F=f,B=b)=P_{YF}(Y=y,F=f)T(f,b).$$ (9)

We can show that $Q^{*}$ is a valid probability distribution since $T$ is row stochastic.

$$\sum_{y\in\mathcal{Y}}\sum_{b\in\mathcal{B}}\sum_{f\in\mathcal{F}}Q^{*}(Y=y,F=f,B=b) =\sum_{y\in\mathcal{Y}}\sum_{b\in\mathcal{B}}\sum_{f\in\mathcal{F}}P_{YF}(Y=y,F=f)T(f,b)$$

$$=\sum_{y\in\mathcal{Y}}\sum_{f\in\mathcal{F}}P_{YF}(Y=y,F=f)\left(\sum_{b\in\mathcal{B}}T(f,b)\right)$$

$$=\sum_{y\in\mathcal{Y}}\sum_{f\in\mathcal{F}}P_{YF}(Y=y,F=f)=1.$$ (10)

Also, we can show that $Q^{*}\in\Delta_{P}$ since:

$$Q^{*}_{YB}(Y=y,B=b)=\sum_{f\in\mathcal{F}}P_{YF}(Y=y,F=f)T(f,b)=P_{YB}(Y=y,B=b),$$ (11)

which holds since such a row-stochastic matrix $T$ exists. Also, we have:

$$Q^{*}_{YF}(Y=y,F=f)=\sum_{b\in\mathcal{B}}P_{YF}(Y=y,F=f)T(f,b)=P_{YF}(Y=y,F=f),$$ (12)

which holds since $T$ is row-stochastic.

Then, $\mathrm{Uni}({Y{:}B|F})=\min_{Q\in\Delta_{P}}\mathrm{I}_{Q}({Y;B|F})\leq\mathrm{I}_{Q^{*}}({Y;B|F})=0.$

∎

Next, it can be shown that the existence of such a row-stochastic matrix is equivalent to Blackwell Sufficiency as per Definition 2 from (Blackwell, 1953).

Appendix B Appendix to Experiments

This section includes additional results and figures for a more comprehensive understanding of our work.

B.1 Data

We consider the waterbird (Wah et al., 2011) and Dominoes dataset. For a summary of the datasets, we refer the readers to Tables 3, 4 and  5.

Table 3: Summary of the Waterbird Dataset

Waterbird	Group00	Group01	Group10	Group11
Train	3498	184	56	1057
Validation	467	466	133	133
Test	2255	2255	642	642
Total	6220	2905	831	1832

Table 4: Summary of the Dominoes 1.0 Dataset

Dominoes 1.0	Group00	Group01	Group10	Group11
Train	3750	1250	1250	3750
Test	473	507	507	473
Total	4223	1772	1757	4208

Table 5: Summary of the Dominoes 2.0 Dataset

Dominoes 2.0	Group00	Group01	Group10	Group11
Train	3000	500	1250	3000
Test	245	490	245	490
Total	3245	990	1495	3490

B.2 Experimental Setup

Table 6: Architecture details of autoencoder for Dominoes Dataset

Sl. No.	Layer	Filter No.	Kernel Size	Stride	Padding	Output Padding	Output Shape	Param No.
1	Conv2d	32	5	2	2	-	(32,16,16)	2432
2	LeakyReLU	-	-	-	-	-	(32,16,16)	0
3	BatchNorm2d	-	-	-	-	-	(32,16,16)	64
4	Conv2d	64	5	2	2	-	(64,8,8)	51264
5	LeakyReLU	-	-	-	-	-	(64,8,8)	0
6	BatchNorm2d	-	-	-	-	-	(64,8,8)	128
7	Conv2d	128	3	2	0	-	(128,3,3)	73856
8	LeakyReLU	-	-	-	-	-	(128,3,3)	0
9	Flatten	-	-	-	-	-	1152	0
10	Linear (embedding)	-	-	-	-	-	10	11530
11	Clustering Layer	-	-	-	-	-	10	100
12	Linear(deembedding)	-	-	-	-	-	1152	12672
13	LeakyReLU	-	-	-	-	-	1152	0
14	ConvTranspose2d	64	3	2	0	1	(64, 8, 8)	73,792
15	LeakyReLU	-	-	-	-	-	(64, 8, 8)	0
16	BatchNorm2d	-	-	-	-	-	(64, 8, 8)	128
17	ConvTranspose2d	32	5	2	2	1	(32, 16, 16)	51,232
18	LeakyReLU	-	-	-	-	-	(32, 16, 16)	0
19	BatchNorm2d	-	-	-	-	-	(32, 16, 16)	64
20	ConvTranspose2d	3	5	2	2	1	(3, 32, 32)	2403

Figure 10: Architecture of the proposed autoencoder for the waterbird dataset. Here, BN stands for Batch normalization.

Calculating PIDs: Calculation of PIDs: redundancy, unique information and synergy involves mainly three steps. First of all, the clusters for the given input images are estimated. This step requires the autoencoder. As shown in Fig.4, a given image is separated into two images: one contains the core features (foreground) and other contains the spurious features (background). For Dominoes dataset, the core features are formed of the images of cars or trucks and the spurious features are the images of zeros and ones. For each set of features, the clusters are computed. The architecture details of the autoencoder for Dominoes dataset are shown in Table 6. The output of the clustering layer is the desired clusters. For waterbird dataset, the architecture details are given in Fig. 10. The complexity of the autoencoder for waterbird is increased in order to handle the more challenging nature of this dataset as compared to the Dominoes one. The architecture is proposed inspired by (Sadeghi & Armanfard, 2023). To obtain the clusters, the model is pretrained with only mean square error loss function (MSEloss). Then, the model is again trained with weighted loss function which is a weighted sum of MSEloss and KL divergence loss. The weights of the clustering layer are initialized with the cluster centers obtained by k-means clustering. For the Dominoes dataset, hyperparameters are as follows: batch size $8$, learning rate $0.001$, CosineAnnealingLR scheduler, Adam optimizer with weight decay $0.0001$, pretraining epochs $100$ and later training is for $50$ epochs. The later training process is terminated if the change of label assignments between two consecutive updates for target distribution is less than $0.001$. For the waterbird dataset, hyperparameters are as follows: batch size $64$, learning rate $0.001$, CosineAnnealingLR scheduler, Adam optimizer with weight decay $0.0001$, pretraining epochs $150$ and later training is for $50$ epochs. Next, the clusters obtained for the foreground and the background and the binary labels are used to estimate the joint distribution using histograms followed by the PID estimation with DIT (James et al., 2018) package.

Calculating Accuracies: To calculate the worst-group accuracy for the different variations of different datasets we do fine tuning of the pre-trained ResNet-$50$ (He et al., 2016) model. The worst-group-accuracy is defined as the accuracy of the minority group having the lowest number of training sample (see Table 3 and  5. For waterbird dataset, group$10$ has minimum training samples and for Dominoes 2.0 dataset, group$01$ has the lowest minority group samples.). For Dominoes 1.0 dataset, since group$01$ and group$10$ have the same number of training and test samples, the worst-group-accuracy is calculated by taking the average of the accuracies of these two groups. For the Dominoes dataset, hyperparameters are as follows: batch size $8$, learning rate $0.0001$, CosineAnnealingLR scheduler, stochastic gradient descent (SGD) optimizer with weight decay $0.0001$, loss function binary cross-entropy and epochs $100$. The train dataset is split into two subsets, i.e., $70\%$ for training split and $30\%$ for validation split. For waterbird dataset, the batch size is $64$ and the other parameters are same as Dominoes. For addition and concatenation dataset the number of sample images in train and test dataset are distributed as in Table 3, 4 and 5 which are created accordingly. For balanced dataset, we use weighted random sampler where weights are selected as the proportion of the groups. All the experiments are executed on NVIDIA RTX A4500.

B.3 Additional Results

Fig.11 shows the Grad-CAM (Selvaraju et al., 2017) variations for different models trained with unbalanced, balanced, addition and concatenation dataset (from left ’a’: unbalanced, ’b,c’: balanced, ’d’: addition and ’e’: concatenation). Observe that for the dataset based mitigation techniques, the model is focusing on the foreground (red region) while on the unbalanced case the model is emphasising in the background. There are cases where model does not give any importance to any portion of the image (see Fig.11b).

Figure 11: Grad-CAM visualization for different variations of models trained with different datasets.