GROOD: GRadient-aware Out-Of-Distribution detection in interpolated manifolds

Abstract

DNN often fail silently with over-confident predictions on out-of-distribution (OOD) samples, posing risks in real-world deployments. Existing techniques predominantly emphasize either the feature representation space or the gradient norms computed with respect to deep neural network (DNN) parameters, yet they overlook the intricate gradient distribution and the topology of classification regions. To address this gap, we introduce GRadient-aware Out-Of-Distribution detection in interpolated manifolds (GROOD), a novel post-hoc method that relies on the discriminative power of gradient space to distinguish between in-distribution (ID) and OOD samples. To build this space, GROOD relies on class prototypes together with a prototype that specifically captures OOD characteristics. Uniquely, our approach incorporates a targeted mix-up operation at an early intermediate layer of the DNN to refine the separation of gradient spaces between ID and OOD samples. For a new sample, we ultimately estimate its deviation from the training distribution by comparing our computed gradient to the gradients observed on the training set, using nearest neighbor distance. Experimental evaluations substantiate that the introduction of targeted input mix-up amplifies the separation between ID and OOD in the gradient space, yielding impressive results across diverse datasets. Notably, when benchmarked on ImageNet-1k, GROOD surpasses the established robustness of state-of-the-art baselines. Through this work, we establish the utility of leveraging gradient spaces and class prototypes for enhanced OOD detection for DNNs in image classification.

1 Introduction

DNN have revolutionized a multitude of disciplines, ranging from computer vision and natural language processing to robotics and control systems [14, 32]. These models, trained with empirical risk minimization (ERM) [62], excel in their capacity to generalize learned representations from training datasets to test instances that are independent and identically distributed (iid) [30, 52, 18, 6, 67].

Nevertheless, the iid assumption, so foundational in traditional ERM, often does not hold in real-world settings [19, 35], which can lead to models performing admirably on samples that are iid with respect to the training data, called in-distribution (ID), but make over-confident and erroneous predictions when confronted with samples that deviate from the training distribution, called out-of-distribution (OOD) [44, 59].

Figure 1: Performance of Post-hoc OOD methods applied to a pretrained ImageNet-1k classifier using ViT-B-16 [9]. Reported AUROC values (%).

Figure 2: Diagram of GROOD showing steps to produce our metric used to measure the OOD score. Initially, input mix-up is performed on the early-layer features with the corresponding class prototype (Sec. 4.3). Subsequently, gradients of the softmax loss built upon the nearest prototype distance as logits are computed (Sec. 4.1). Then, a simple nearest neighbor search is used on gradients of the training data to produce a distance metric to the nearest ID gradient (Sec. 4.4).

This phenomenon is particularly problematic in safety-critical applications like healthcare and autonomous driving where both ID and OOD scenarios are prevalent [38, 4]. Furthermore, we increasingly deploy DNNs in open-world contexts [19], where the test data could substantially deviate from the training distribution. In such scenarios, it becomes imperative for the model to exhibit self-awareness about its own limitations [13]. Conventional approaches that focus solely on minimizing the training loss are often ill-equipped to cope with OOD samples, thereby jeopardizing the safe and reliable deployment of deep learning systems [10, 1, 50, 39].

Consequently, several active lines of research work toward equip** DNNs with the capability to detect unknown or OOD samples effectively. Among them, the post-hoc OOD detection approach, which designs algorithms that leverage representations of already-trained DNNs to identify OOD data, is the most convenient because it does not involve a new training procedure and can be applied to any neural network. Some experimental studies on large benchmarks have even emphasized that this approach may be the most performant [65, 69], stimulating the appearance of dedicated libraries [27, 45].

This rich domain is populated by different families of methods. Gradient-based methods [24, 26, 54, 34] build gradients of quantities based on the DNNs parameters. Distance-based approaches operate in a different regime, focusing on feature-space proximity to differentiate between samples [36, 58, 11]. Other methodologies, not reliant on gradients or feature space, direct their attention to the output space of the model. These works often analyze the distribution of output logits or energy scores as a basis for OOD detection [19, 64, 7].

In this work, we introduce a new method that we call GROOD that builds upon three observations:

1. The neural collapse property [46] suggests that at the end of neural network training, within-class variability tends to zero for sample representation in the feature space. This motivates the use of nearest centroid classification based on the distance to class centroids, also called prototypes, defined as means of samples of each class in the feature space. Inspired by this property, we construct logits out of the distance with respect to class prototypes.

2. As evidenced in Fig. 3 (upper panel) and  Sun et al. [57], the distribution of OOD points in the feature space is more spread out. However, we find that an OOD prototype can serve as an anchor point for computing gradients, enabling us to formulate our ID/OOD classification problem in a gradient space (Fig. 3, lower panel) that proves to be more suitable.

3. As we shall see in Sec. 4.3, performing mixup [61] in the early layers space affects OOD samples more than ID samples. Therefore, we perform guided interpolation of input samples with class prototypes in the manifold defined by early layers before computing their logits.

In contrast with previous post-hoc methods, we combine both gradient and distance-based methods by deriving the gradients of the cross-entropy loss applied to the constructed logits with respect to the newly introduced OOD prototype. Then, for a new input point, we find the nearest neighbour among the gradients of ID data points and compute the distance to this nearest neighbor to serve as our OOD score.

The principal contribution of this work can be encapsulated in the following premise:

Leveraging insights on neural collapse, feature space distribution differences between ID and OOD samples, OOD proxy information and mixup techniques, GROOD combines gradient and distance-based approaches to compute OOD scores, enabling more effective detection of OOD data without requiring additional training.

We conduct an extensive empirical study following the recent methodology introduced in OpenOOD Benchmark [69], but we also evaluate our method on other recent architectures. The following results highlight the following key advantages of GROOD:

• GROOD provides effective detection in both the near and the far OOD regimes with AUROC performance either on par with or above state-of-the-art post-hoc methods across all setups.

• On ImageNet-1K [18, 30], GROOD consistently shows superior performance for various architectures, including ViT-B-16 [9].

2 Related Work

Neural network properties

Prior research has underscored the significance of linear interpolation in manifold space, with applications ranging from word embeddings [43] to machine translation [16]. Verma et al. [61] introduced manifold mixup (MM) as a means to mitigate over-confident predictions near ID data by smoothing decision boundaries. This idea aligns with the observations by Papyan et al. [46] and Kothapalli et al. [29], who investigated the neural collapse (NC) phenomenon, highlighting how training data tends to cluster around class-specific prototypes.

Our work is also informed by insights into the structure of classification regions in deep neural networks. While Fawzi et al. [12] provided an understanding of these regions, our approach primarily leverages the neural collapse phenomenon and the concept of prototype/centroid-based classification. We exploit the sensitivity of feature extractor layers to differentiate between ID and OOD samples, employing guided perturbations at an initial layer to enhance this distinction.

Output-based methods

A prevalent approach to develo** OOD scores involves solely utilizing the model’s output. Hendrycks and Gimpel [20] employ the model’s maximum softmax probability as the OOD score. Guo et al. [15] scale the logits to enhance model calibration, subsequently using the maximum softmax probability as the OOD score. Liang et al. [37] apply temperature scaling and introduce minor perturbations to inputs to distinguish ID and OOD samples, while still using the maximum softmax probability as the OOD score. Hendrycks et al. [21] opt for logits over softmax as the OOD score. They also propose another OOD score which measures the KL divergence between softmax output and mean class conditional distributions.

In contrast to directly using the model output, several works devise OOD scores by calculating a different score over the model outputs. Liu et al. [40] propose the use of energy scores over the logits for OOD detection. Sun et al. [56] suggest truncating high activations to reduce model overconfidence, then using the energy score over the obtained logits as the OOD score. Sun and Li [55] propose pruning activations from less important weights and using the energy score over the obtained logits as the OOD score. Song et al. [53] suggest removing the largest singular value and associated singular vectors from the features to reduce the model’s overconfidence, especially on OOD samples, and use the energy score over the obtained logits as the OOD score. Djurisic et al. [8] propose removing a large portion of low activations and using the energy score over the obtained logits as the OOD score. Liu et al. [41] propose computing generalized entropy instead of cross entropy over the logits and using it as an OOD score.

Feature-based methods

Several other works devise OOD scores using the feature space of the network. Lee et al. [36] use the Mahalanobis distance from class centroids of features as the OOD score. Another work splits the image into two parts, foreground and background, and then calculates the Mahalanobis distance ratio between the two spaces [48]. Sastry and Oore [49] use anomalies found in activity patterns given by Gram matrices of features as the OOD score. Wang et al. [63] construct an additional virtual logit using features and existing logits to measure the probability of a given sample coming from an OOD class and use the softmax score of this logit as the OOD score. Sun et al. [57] use the distance from the k-th nearest neighbor sample as the OOD score. Zhang et al. [68] propose using modern Hopfield networks [47, 31] to measure the discrepancy between an OOD sample and class conditional features of ID samples as an OOD score. Various works propose an OOD score based on the cosine similarity between test sample features and class features [60, 5]. [66] showed that the first singular vector is more suitable for cosine similarity-based detection.

Gradient-based methods

rely on gradient information across different layers instead of relying solely on intermediate layers or network outputs. The seminal work of the out-of-distribution detector for neural networks (ODIN) [22] pioneered the utilization of gradient-based information to discern OOD samples, incorporating gradient-informed input perturbations. Such perturbations are designed to intensify the softmax confidence scores for ID samples, increasing the gap between ID and OOD. This technique is also in use in [35] to improve the discriminative power of the Mahalanobis distance between ID and OOD. While these techniques only use a gradient to perturb the input, Lee and AlRegib [33] proposed using gradients with respect to a neural network’s weight as a proxy to predict uncertainties and hence detect OOD samples having high uncertainties. Another recent work [54] proposed to separate ID and OOD samples by computing the Mahalanobis distance between input samples’ gradients and ID/OOD gradients guided by a self-supervised binary classifier. Recently, Huang et al. [24] proposed GradNorm, which derives a scoring function from the gradient space. GradNorm uses a vector norm of gradients back-propagated using the KL divergence between a softmax output and uniform probability distribution. The key idea is that the magnitude of gradients is higher for ID data than for OOD data. However, GROOD is the first to explore the use of gradients in the penultimate layer space built with ID and OOD information to disentangle ID from OOD.

3 Preliminaries and Notation

Before diving into the specifics of GROOD, we introduce the foundational problem of OOD detection and establish the notations.

3.1 Context and Notations

We consider a supervised classification problem. Let $X$ be the input space, and $Y=\{1,2,\ldots,C\}$ represent the label space. The training set $\mathcal{D}_{\text{in}}=\{(x_{i},y_{i})\}_{i=1}^{n}$ is assumed to be drawn iid from a joint data distribution $P_{XY}$. We denote $P_{\text{in}}$ as the marginal distribution on $X$.

We assume access to a neural network $f:X\to\mathbb{R}^{|Y|}$ trained on samples from $P_{XY}$ to produce a logit vector, which is subsequently used for label prediction. The architecture of our network is systematically decomposed as follows:

$$f=f^{\text{clf}}\circ f^{\text{pen}},\quad\text{where}\quad f^{\text{pen}}=f^{\text{mid}}\circ f^{\text{early}}.$$ (1)

$f^{\text{early}}$ represents the early feature extraction layer, responsible for initial processing of input images to generate low-level features. This is followed by the mid-level feature extractor $f^{\text{mid}}$, and subsequently, the penultimate feature extraction layer $f^{\text{pen}}$. The final stage of the network is the classification module $f^{\text{clf}}$, which outputs the prediction.

3.2 Problem Setting: Out-of-Distribution Detection

When deploying a machine learning model in practice, the classifier should not only be accurate on ID samples but should also identify any OOD inputs as “unknown”.

Formally, OOD detection can be viewed as a binary classification task. During testing, the task is to determine whether a sample $x\in X$ comes from $P_{\text{in}}$ (ID) or not (OOD). The decision can be framed as a level set estimation:

$$G_{\lambda}(x)=\begin{cases}\text{ID},&\text{if }S(x)\leq\tau,\\ \text{OOD},&\text{if }S(x)>\tau,\end{cases}$$

where $S:X\to\mathbb{R}$ is a score function, and $\tau$ is a threshold. Typically, OOD is evaluated on distribution simulating unknowns that might be encountered during deployment. Such samples are from a distribution whose label set has no intersection with $Y$.

4 GROOD Methodology

Overview

In this section, we introduce GRadient-aware Out-Of-Distribution detection in interpolated manifolds (GROOD), which is also summarized in Figure 2. As we show, GROOD is a new and effective way to use a distance-based measure with gradient information for post-hoc OOD detection.

The first step involves computing the class prototypes [46] at the early and penultimate layers of our network, as well as a OOD prototype, as described in Sec. 4.1. Subsequently, in Sec. 4.2, we show how we construct gradients out of these prototypes based on a modified cross-entropy loss built out of the distances to the penultimate class and OOD prototypes. Next, in Sec. 4.3, we introduce the concept of guided prototype interpolation, where we create interpolated samples guided by class prototypes at the early layer of the DNN. This interpolation accentuates the discriminative power of the gradients. Finally, we describe how we build our OOD score, on which our final decision between ID and OOD is based, using the distance to the nearest neighbor in the gradients space, as shown in Sec. 4.4.

4.1 Prototype Computation

The class prototypes are calculated as the averages of feature vectors at their respective layers, but exclusively for training instances that have been correctly classified. This method ensures that the prototypes effectively capture class-discriminative patterns. We note these prototypes $(\mathbf{p}^{\text{early}}_{y})_{y=1}^{C}$ and $(\mathbf{p}^{\text{pen}}_{y})_{y=1}^{C}$ and compute them as:

$$\mathbf{p}^{l}_{y}=\frac{1}{|X_{y}|}\sum_{x\in X_{y}}f^{l}(x),\quad l\in\{\text{early},\text{pen}\}$$ (2)

where $X_{y}$ is the set of correctly classified training instances in class $y$. Next, we propose to encapsulate the representation of our network in an OOD $\mathbf{p}^{\text{pen}}_{\text{ood}}{}$. Crucially, this special prototype is computed by averaging penultimate layer representations on instances that have not been used for training. For this paper, we consider a mixture of uniform noise with an OOD validation dataset:

$$\mathbf{p}^{\text{pen}}_{\text{ood}}=\frac{1}{n_{\text{Unif}}+n_{\text{ood}}}\bigg{(}\sum_{i=1}^{n_{\text{Unif}}}f^{\text{pen}}(u_{i})+\sum_{i=1}^{n_{\text{ood}}}f^{\text{pen}}(o_{i})\bigg{)}.$$ (3)

In constructing $\mathbf{p}^{\text{pen}}_{\text{ood}}$, we selected 100 data points from an OOD validation dataset sourced from the OpenOOD framework [69, 65] with no overlap with the test set and used in some exisitng post-hoc methods [35, 36, 28]. Alongside this, we incorporated 100 iid samples of uniform noise ($u_{i}\sim\mathcal{U}\{0,255\}^{\text{CxHxW}}$), culminating in a total of 200 data points for the OOD prototype computation. By combining real-world OOD data with synthetic noise, we achieve a more robust representation of the OOD prototype. We further illustrate the importance of using this combination in Sec. 5.2 (experiment (5)).

4.2 Gradients Computation

Inspired by the neural collapse property [46] and prototype-based open set recognition [51], we use the previously defined prototypes to construct logits with the Euclidean distances to each class prototype. Crucially, we also consider an additional logit corresponding to our OOD prototype. For any vector $h$ in the penultimate latent space, we therefore define our logit vector as

$$L(h)=-[\|h-\mathbf{p}^{\text{pen}}_{1}\|_{2},\ldots,\|h-\mathbf{p}^{\text{pen}}_{C}\|_{2},\|h-\mathbf{p}^{\text{pen}}_{C+1}\|],$$ (4)

where $\mathbf{p}^{\text{pen}}_{C+1}$ is equivalent to $\mathbf{p}^{\text{pen}}_{\text{ood}}$. The corresponding softmax values are:

$$p(h)_{i}=\frac{\exp(L_{i}(h))}{\sum_{j=1}^{C+1}\exp(L_{j}(h))},\quad i=1,\ldots,C,C+1,$$ (5)

where $p(h)_{C+1}=p(h)_{\text{ood}}$ denotes the softmax value corresponding to the OOD prototype. By considering the cross-entropy loss on this newly introduced set of probits, we are able to quantify how optimal our OOD prototype would be if $h$ were to represent an ID sample. This loss is given by:

$$H(h)=-\log p(h)_{y},$$ (6)

where $y\in\{1,\ldots,C\}$ is the class predicted by $f$, i.e. we are only interested in the case where $y$ is an ID class label. Our score is based on the gradient of this loss with respect to the OOD prototype $\mathbf{p}^{\text{pen}}_{\text{ood}}$. This expression has a closed form given by

$$\nabla H(h)=p(h)_{\text{ood}}\frac{h-\mathbf{p}^{\text{pen}}_{\text{ood}}}{\left\|h-\mathbf{p}^{\text{pen}}_{\text{ood}}\right\|_{2}},$$ (7)

where we denote $\nabla H(h)=\nabla_{\mathbf{p}^{\text{pen}}_{\text{ood}}}H(h)$ for conciseness. Note that this expression does not depend on the label $y$, since it is always an ID label. Eq. (7) shows that the gradient norm is equal to the softmax probability of the OOD class. When $h$ represents an ID sample, $p(h)_{\text{ood}}$ tends to be smaller than for OOD samples, which corresponds to a smaller norm of the gradient with respect to the OOD prototype. While this scalar information already enjoys a discriminative power, our gradient also contains the rich information given by the unit vector in the direction $h-\mathbf{p}^{\text{pen}}_{\text{ood}}$ pointing from $\mathbf{p}^{\text{pen}}_{\text{ood}}$ towards $h$. In summary, we expect that the hypothetical update of the OOD prototype represented by $\nabla H(h)$ will be larger in norm and/or different in orientation when $h$ represents an OOD sample compared to the case where it represents an ID sample.

In Fig. 3, we produce a t-SNE visualization of the penultimate feature space vs the gradient space to show how different these spaces can be, thereby potentially encompassing additional information. Specifically, we examine the t-SNE representations of in-distribution (ID) samples alongside out-of-distribution (OOD) test samples. We will see in Sec. 5.2 (experiment (2)) that this information is key to GROOD’s performance, and justifies evaluating the gradients of $H$ rather than simply comparing the logits.

Figure 3: The upper panel is a t-SNE visualization, capturing the distribution of ID and OOD samples within the feature representation space. The lower panel is a t-SNE visualization on the gradient distributions associated with both ID and OOD samples. These gradients are derived from a ResNet-18 model, which has been pre-trained on the CIFAR-10 dataset.

4.3 Guided Prototype Interpolation

We perform guided prototype perturbation by mixing-up samples towards the predicted class prototype at an early layer. The interpolation and the penultimate representation of the interpolated sample $\hat{h}(x)$ can be formalized as:

$$\hat{h}(x)=f^{\text{mid}}(\lambda f^{\text{early}}(x)+(1-\lambda)\mathbf{p}^{\text{early}}_{y}),$$ (8)

where $\lambda$ is a hyper-parameter and $y=\arg\max_{i\in Y}(f(x))$ is the predicted class of $x$. Note that computing $\hat{h}(x)$ requires to perform a forward pass to get $y$ and therefore the corresponding prototype.

Our intuition is that mix-up at an early layer with $\mathbf{p}^{\text{early}}_{y}$ alters the penultimate representation of OOD data more than ID data. In Fig. 4, we illustrate this intuition by comparing the distributions of distances $\|\hat{h}(x)-f^{\text{pen}}(x)\|$ on OOD and ID data points. We can see that OOD penultimate representations are more affected by the mixup than ID ones. We also dedicate Sec. 5.1 to an analysis of the choice of $\lambda$’s value, that we consequently set as $\lambda=0.9$ in all our experiments. The results of Sec. 5.2 (experiment (1)) show that this mixup strategy has a positive impact on the final OOD score.

Figure 4: Effect of guided interpolation: distance between mixed and non-mixed samples for ID and OOD. We compare the distances using CIFAR-10 as our ID set and CIFAR-100 as our OOD set.

Algorithm 1 GROOD initialization: Compute prototypes and gradients for the training set

1:Training set $\mathcal{D}_{\text{in}}$, trained model $f$

2:mixup parameter $\lambda$

3:Compute class prototypes $\mathbf{P}_{\text{early}}$ and $\mathbf{P}_{\text{pen}}$ $\triangleright$ Eq. 2

4:Compute OOD prototype $\mathbf{p}^{\text{pen}}_{\text{ood}}$ $\triangleright$ Eq. 3

5:function gpi($x,(\mathbf{p}^{\text{early}}_{y})_{y=1}^{C}$)

6:     $y=\arg\!\max_{i\in C}{f(x)}$ $\triangleright$ compute pseudo-label

7:     compute $\hat{h}(x)$ $\triangleright$ Eq. 8, using $\mathbf{p}^{\text{early}}_{y}$

8:     return $\hat{h}(x)$

9:end function

10:function comp_grad($h,(\mathbf{p}^{\text{pen}}_{y})_{y=1}^{C},\mathbf{p}^{\text{pen}}_{\text{ood}}$)

11:     compute $\nabla H(h)$ $\triangleright$ Eq. 7 using $(\mathbf{p}^{\text{pen}}_{y})_{y=1}^{C}$, $\mathbf{p}^{\text{pen}}_{\text{ood}}$

12:     return $\nabla H(h)$

13:end function

14:for each $x\in\mathcal{D}_{\text{in}}$ do

15:     $\hat{h}(x)=$ gpi $(x,(\mathbf{p}^{\text{early}}_{y})_{y=1}^{C},\lambda)$

16:     $\nabla(x)=$ comp_grad $(\hat{h}(x),(\mathbf{p}^{\text{pen}}_{y})_{y=1}^{C},\mathbf{p}^{\text{pen}}_{\text{ood}})$

17:end for

18:return $\{\nabla(x)\}_{x\in\mathcal{D}_{\text{in}}}$, $(\mathbf{p}^{\text{early}}_{y})_{y=1}^{C}$, $(\mathbf{p}^{\text{pen}}_{y})_{y=1}^{C}$, $\mathbf{p}^{\text{pen}}_{\text{ood}}$

4.4 Distance Measurement

In Sec 4.2 we described our computation of gradients with respect to the OOD prototype based on mixed-up representations. Finally, we estimate the high dimensional distribution of the gradients corresponding to training samples and formulate our detection score as a measure of the deviation to that reference distribution.

Given a test sample $x_{\text{new}}$, we compute the mixed-up interpolation $\hat{h}(x_{\text{new}})$ and then its gradient $\nabla H(\hat{h}(x_{\text{new}}))$ with respect to the OOD prototype. We then measure its distance to the nearest gradient among training ID samples to serve as the OOD score. Specifically, we define the OOD score $S(x_{\text{new}})$ as:

$$S(x_{\text{new}})=\min_{x\in\mathcal{D}_{\text{in}}}\|\nabla H(\hat{h}(x_{\text{new}}))-\nabla H(\hat{h}(x))\|_{2}.$$ (9)

Finally, based on this OOD score, the sample is classified as ID or OOD using a pre-defined threshold as explained earlier in Sec. 3.

By relying on the distance on the gradient space, our method provides a nuanced yet computationally feasible strategy for OOD detection. This approach leverages both gradient information and distance metrics, encapsulating insights from both gradient-based and distance-based methods. For a comprehensive overview, the complete GROOD algorithm is outlined in Algorithm 1 and Algorithm 2.

Algorithm 2 OOD score using GROOD

1:Training dataset $\mathcal{D}_{\text{in}}$, trained model $f$

2:Mixup parameter $\lambda$

3:$\{\nabla(x))\}_{x\in\mathcal{D}_{\text{in}}}$, $(\mathbf{p}^{\text{early}}_{y})_{y=1}^{C}$, $(\mathbf{p}^{\text{pen}}_{y})_{y=1}^{C}$, $\mathbf{p}^{\text{pen}}_{\text{ood}}$ from GROOD initialization

4:functions gpi and comp_grad (in Alg. 1)

5:Sample $x_{\text{new}}$, threshold $\tau$

6:$\hat{h}(x_{\text{new}})=$ gpi($x_{\text{new}},(\mathbf{p}^{\text{early}}_{y})_{y=1}^{C},\lambda$)

7:$\nabla(x_{\text{new}})=$ $\textsc{comp\_grad}(\hat{h}(x_{\text{new}}),(\mathbf{p}^{\text{pen}}_{y})_{y=1}^{C},\mathbf{p}^{\text{pen}}_{\text{ood}}$)

8:Compute OOD score using Nearest Neighbor search: $S(x_{\text{new}})=\min_{x\in\mathcal{D}_{\text{in}}}\|\nabla(x_{\text{new}})-\nabla(x)\|_{2}$

9:return ID if $S(x_{\text{new}})\leq\tau$ else OOD

5 Experiments

Table 1: Main results from OpenOOD v1.5 on standard OOD detection. GROOD showing superior results compared to existing baselines.

	CIFAR-10		CIFAR-100		ImageNet-200		ImageNet-1K
ID Acc.	95.06(±0.30)		77.25(±0.10)		86.37(±0.08)		76.18
Method	Near-OOD	Far-OOD	Near-OOD	Far-OOD	Near-OOD	Far-OOD	Near-OOD	Far-OOD
OpenMax [3] (CVPR’16)	87.62(±0.29)	89.62(±0.19)	76.41(±0.25)	79.48(±0.41)	80.27(±0.10)	90.20(±0.17)	74.77	89.26
MSP [20] (ICLR’17)	88.03(±0.25)	90.73(±0.43)	80.27(±0.11)	77.76(±0.44)	83.34(±0.06)	90.13(±0.09)	76.02	85.23
TempScale [15] (ICML’17)	88.09(±0.31)	90.97(±0.52)	80.90(±0.07)	78.74(±0.51)	83.69(±0.04)	90.82(±0.09)	77.14	87.56
ODIN [37] (ICLR’18)	82.87(±1.85)	87.96(±0.61)	79.90(±0.11)	79.28(±0.21)	80.27(±0.08)	91.71(±0.19)	74.75	89.47
MDS [36] (NeurIPS’18)	84.20(±2.40)	89.72(±1.36)	58.69(±0.09)	69.39(±1.39)	61.93(±0.51)	74.72(±0.26)	55.44	74.25
MDSEns [36] (NeurIPS’18)	60.43(±0.26)	73.90(±0.27)	46.31(±0.24)	66.00(±0.69)	54.32(±0.24)	69.27(±0.57)	49.67	67.52
Gram [49] (ICML’20)	58.66(±4.83)	71.73(±3.20)	51.66(±0.77)	73.36(±1.08)	67.67(±1.07)	71.19(±0.24)	61.70	79.71
EBO [40] (NeurIPS’20)	87.58(±0.46)	91.21(±0.92)	80.91(±0.08)	79.77(±0.61)	82.50(±0.05)	90.86(±0.21)	75.89	89.47
OpenGAN [28] (ICCV’21)	53.71(±7.68)	54.61(±15.51)	65.98(±1.26)	67.88(±7.16)	59.79(±3.39)	73.15(±5.07)	N/A	N/A
GradNorm [25] (NeurIPS’21)	54.90(±0.98)	57.55(±3.22)	70.13(±0.47)	69.14(±1.05)	72.75(±0.48)	84.26(±0.87)	72.96	90.25
ReAct [56] (NeurIPS’21)	87.11(±0.61)	90.42(±1.41)	80.77(±0.05)	80.39(±0.49)	81.87(±0.98)	92.31(±0.56)	77.38	93.67
MLS [21] (ICML’22)	87.52(±0.47)	91.10(±0.89)	81.05(±0.07)	79.67(±0.57)	82.90(±0.04)	91.11(±0.19)	76.46	89.57
KLM [21] (ICML’22)	79.19(±0.80)	82.68(±0.21)	76.56(±0.25)	76.24(±0.52)	80.76(±0.08)	88.53(±0.11)	76.64	87.60
VIM [63] (CVPR’22)	88.68(±0.28)	93.48(±0.24)	74.98(±0.13)	81.70(±0.62)	78.68(±0.24)	91.26(±0.19)	72.08	92.68
KNN [57] (ICML’22)	90.64(±0.20)	92.96(±0.14)	80.18(±0.15)	82.40(±0.17)	81.57(±0.17)	93.16(±0.22)	71.10	90.18
DICE [55] (ECCV’22)	78.34(±0.79)	84.23(±1.89)	79.38(±0.23)	80.01(±0.18)	81.78(±0.14)	90.80(±0.31)	73.07	90.95
RankFeat [53] (NeurIPS’22)	79.46(±2.52)	75.87(±5.06)	61.88(±1.28)	67.10(±1.42)	56.92(±1.59)	38.22(±3.85)	50.99	53.93
ASH [8] (ICLR’23)	75.27(±1.04)	78.49(±2.58)	78.20(±0.15)	80.58(±0.66)	82.38(±0.19)	93.90(±0.27)	78.17	95.74
SHE [68] (ICLR’23)	81.54(±0.51)	85.32(±1.43)	78.95(±0.18)	76.92(±1.16)	80.18(±0.25)	89.81(±0.61)	73.78	90.92
GEN [41] (CVPR’23)	88.2(±0.3)	91.35(±0.55)	81.31(±0.1)	79.68(±0.6)	83.68(±0.34)	91.36 (±0.45)	76.85	89.76
GROOD (OURS)	91.05(±0.001)	94.19(±0.3)	81.01(±0.01)	81.98(±0.7)	81.9(±0.07)	94.9(±0.012)	78.18	96.16

For a comprehensive evaluation of GROOD’s performance, we adhere to the OpenOOD v1.5 criteria [69, 65]. Results are aggregated in Tab. 1 including the performance of the nearest class prototype used instead of the classification head. For robustness, each evaluation metric except for ImageNet-1K is derived from three runs with unique initialization seeds. In the case of ImageNet-1K, we report results based on a single seed run provided by torchvision Author [2].

Benchmarking Strategy

Our evaluation involves four core ID datasets: CIFAR-10, CIFAR-100, ImageNet-200, and ImageNet-1K. Despite the existing literature predominantly addressing far-OOD scenarios [69, 65], we also scrutinize the relatively neglected realm of near-OOD.

Dataset Characteristics

• CIFAR-10 and CIFAR-100: Each encompasses 50k training and 10k test images. For near-OOD, we use CIFAR-100 and TinyImageNet; for far-OOD, MNIST, SVHN, Textures, and Places365.

• ImageNet-200: A subset of ImageNet featuring 200 classes and 64x64 resolution. SSB-hard and NINCO serve as near-OOD, while iNaturalist, Textures, and OpenImage-O constitute far-OOD.

• ImageNet-1K: Encompasses 1000 classes, and shares OOD datasets with ImageNet-200.

• Val-OOD: For ImageNet-1k and ImageNet-200, our approach utilizes 100 images from the OpenImage-O validation set. In the case of CIFAR-10 and CIFAR-100, we employ 100 images from the TinyImageNet validation set, adhering to the protocols established in OpenOOD [69, 65].

Experimental Configuration

• ResNet-18: Deployed for CIFAR-10/100 and ImageNet-200. Instead of training the models from scratch, we utilized pre-trained checkpoints provided by the OpenOOD framework [69, 65]. This approach ensures consistency with the established benchmark and allows us to focus on the evaluation of GROOD’s robustness. For a comprehensive analysis, we tested models with three distinct initialization seeds.

• ResNet-50, ViT-B-16, Swin-T: Applied to ImageNet-1K using pre-trained torchvision models. This setup extends beyond the scope of OpenOOD v1 [65], allowing us to explore the effectiveness of GROOD in a broader context.

Experimental results

In Fig. 5, we show GROOD’s consistent efficacy across diverse architectures pre-trained on ImageNet-1K including ViT-B-16 [9], ResNet-50 [17] and Swin-T [42]. Furthermore, Tab. 1 highlights our state-of-the-art performance on multiple datasets using the ResNet [17] architecture.

5.1 Choice of $\lambda$

The selection of the parameter $\lambda$ is pivotal to the performance of the interpolation mechanism described in Sec. 4.3. The value of $\lambda$ determines the extent to which the input image is perturbed towards the class prototype during guided interpolation. To empirically assess the impact of different $\lambda$ values, we conduct experiments on the CIFAR-10 and CIFAR-100 datasets. The results of these experiments are illustrated in Fig. 6, where we observe that the method is optimal when $\lambda\in[0.8,1]$, i.e. when the mixup perturbation is moderate. This is consistent since the lower $\lambda$ is, the less information we keep about $x$.

(a) Near-OOD Performance

(b) Far-OOD Performance

Figure 5: Performance comparison for different architectures on ImageNet-1K.

(a) Performance variation with different $\lambda$ values on CIFAR-10.

(b) Performance variation with different $\lambda$ values on CIFAR-100.

Figure 6: Evaluation of different $\lambda$ values on the CIFAR-10 and CIFAR-100 datasets demonstrating the impact on our method’s performance.

5.2 Ablations

Table 2: Ablation study using different losses and OOD scores to show the effectiveness of each proposed part. Evaluation done on CIFAR-10.

Model Variant	AUROC (%)
	Far-OOD	Near-OOD
(1) Without guided interpolation	93.64(±0.31)	90.1(±0.029)
(2) Distance to the Noise prototype	90.11(±4.43)	85.64(±1.03)
(3) Gradient L1-norm only	93.85(±0.44)	90.56(±0.013)
(4) Grads. wrt class prototypes	93.42(±0.33)	90.37(±0.006)
(5) OOD prototype with uniform noise only	92.83(±0.6)	89.5(±0.03)
GROOD	94.19(±0.3)	91.05(±0.001)

In Tab. 2, we empirically demonstrate the effectiveness of each component within GROOD on CIFAR10.

(1) First, we remove the guided interpolation mechanism, effectively fixing $\lambda=1.0$ instead of $\lambda=0.9$. This leads to a decrease in the AUROC for both near and far OOD, which shows that gradients obtained when applying this mix-up strategy allow for a better discrimination of ID and OOD samples, as shown also in Fig. 6.

(2) We investigate the implications of directly using the distance from $\hat{h}(x)$ to the OOD prototype as the OOD score. This approach shows the importance of using the gradients to construct the scores and the importance of including the OOD prototype.

(3) We also validate the importance of using the distance to the nearest neighbor in the gradient space by assessing the usage of the gradient $L1$-norm as in gradnorm [23] only as the OOD score. The former strategy indicates the value of comprehensive gradient information for OOD detection. The consistent enhancements across these varied experimental conditions affirm the robustness of the proposed GROOD framework.

(4) Subsequently, we explore the impact of employing gradients $\nabla_{\mathbf{p}_{\mathbf{p}^{\text{pen}}_{y}}}H(h)$ (wrt class prototypes) rather than $\nabla_{\mathbf{p}^{\text{pen}}_{\text{ood}}}H(h)$. The results reveal a distinct advantage in favor of the OOD prototype.

(5) Finally, we explore the importance of using OOD data from external dataset instead of pure random noise by running GROOD with an OOD prototype built with uniform noise only ($n_{ood}=0$ in Eq. (3)).

6 Conclusion

In this work, we presented the GRadient-aware Out-Of-Distribution detection in interpolated manifolds (GROOD), a novel approach to the challenge of detecting OOD samples in DNN image classifiers. By leveraging the strengths of gradient information and distance metrics, GROOD demonstrates a significant advancement in identifying OOD instances, crucial for deploying DNNs in real-world applications where reliability and safety are paramount.

Our methodology relies on the innovative use of class prototypes to encapsulate class-specific features, the design of an OOD prototype to encompass OOD information, and the employment of gradients of a newly defined cross-entropy loss relying on the distance to these prototypes for enhanced separation between in-distribution (ID) and OOD samples. The guided prototype interpolation and the subsequent nearest neighbor-based distance measurement offer a robust and reliable OOD scoring mechanism, as evidenced by our experimental results.

The experimental evaluation across various benchmarks illustrates the efficacy of GROOD in detecting both near and far OOD samples, independent of specific DNN architectures or training data distributions. The ability of GROOD to operate without extensive hyper-parameter tuning further underscores its practicality and adaptability.