Restyling Unsupervised Concept Based Interpretable Networks with Generative Models

Concept-based interpretable networks. We denote a training set for a supervised image classification task as $\mathcal{S}=\{(x_{i},y_{i})\}_{i=1}^{N}$. Each input image $x\in\mathcal{X}\subset\mathbb{R}^{n}$ is associated with a class label $y\in\mathcal{Y}$, a one-hot vector of size number of classes $C$. The by-design interpretable network based on learning concept representation is denoted by $g:\mathcal{X}\rightarrow\mathcal{Y}$.

Limitations with concept visualization in previous unsupervised CoINs . A central common trait among prior CoINs learning unsupervised concepts [7, 51, 59] is the deployment of a decoder to reconstruct input $x$ from $\Phi(x)$. Unlike supervised methods, they do not have access to any concept labels and thus need an additional visualization pipeline to understand the information encoded by each concept. However, their visualization pipeline does not utilize the decoder, but instead relies on proxy methods to probe the concept activation. Typically, it consists of finding an input that highly activates a concept, either by selecting from the training data [7, 59] or via input optimization [51]. In the former case, simply visualizing the set of most activating training samples lacks granularity to highlight the features encoded by the concept. Using input optimization, while relatively more insightful, is still difficult for a user to understand as the optimized images are often unnatural. Moreover, these issues exacerbate for large-scale images, as seen in Fig. 1. A natural strategy to overcome these limitations is to enable direct control of a concept’s activation and visualizing its effect on the input. Since the decoder defines the relationship between concept activations and input samples, a generative model is a perfect candidate for a decoder to unlock this ability, in contrast to standard decoders used previously.

Interpretable network design . The dictionary $\Phi$ consists of $K$ concept functions $\phi_{1},\dots,\phi_{K}$. Given an input $x$, each concept activation $\phi_{k}(x)$ is represented by a small convolutional feature map with non-negative activation. Thus $\Phi(x)=[\phi_{1}(x),...,\phi_{K}(x)]\in\mathbb{R}_{+}^{K\times b}$, where $b$ is the total number of elements in each feature map. We model computation of concept activations $\Phi(x)$ using the pretrained classification network $f$ and learn a relatively lightweight network $\Psi$ on top of its selected hidden layers denoted as $f_{\mathcal{I}}(x)$, i.e. $\Phi(x)=\Psi\circ f_{\mathcal{I}}(x)$. $\Theta$ is designed to simply pool the feature maps to obtain a single concept activation of size $K$ and make the final prediction by passing it through a linear layer followed by softmax, i.e. $g(x)=\Theta(\Phi(x))=\textrm{softmax}(\Theta_{W}^{T}\textrm{pool}(\Phi(x)))$, where $\Theta_{W}\in\mathbb{R}^{K\times C}$ are the weights in the linear layer. The simplified design of $\Theta$ makes estimating importance of each concept function $\phi_{k}$ for any prediction straightforward.
Viewability property . In order to improve visualization for unsupervised CoINs, and thus interpretation of learnt concepts, we propose to add the requirement for a viewability property. Given an input image $x$, this property requires to be able to reconstruct high-quality images from $\Phi(x)$, from a pretrained generative model $G$. More specifically, reconstructions should have high enough quality to “view” input samples through generated outputs and thus ground modifications to $\Phi(x)$ back to $x$. In our method, we propose to achieve this by learning an additional concept translator module $\Omega$ to map $\Phi(x)$ to the latent space of $G$, such that high-quality reconstructed images can be obtained from $\Phi(x)$ through $\Omega$ and $G$. The design of $\Omega$ depends entirely on the generative model used.

A short primer on style-based GAN . The work by Karras et al. [33] proposed a style-based generator architecture. The generator learns to map a noise vector ($z\in\mathbb{R}^{512}\sim\mathcal{N}(0,1)$) to a latent code ($w\in\mathcal{W}\subset\mathbb{R}^{512}$) that controls the scaling and biasing of feature maps at different resolutions in the synthesis network, controlling the style of the generated image. Subsequent updates on the original architecture [34, 32] have improved upon various aspects of generator architecture and regularization. One such update relevant for us is the StyleGAN2-ADA [31] which incorporates an adaptive discriminator augmentation in training. This enables training StyleGAN2 with limited amount of data, a regime where GANs are generally prone to collapsing. The StyleGAN architecture has been actively studied for certain aspects relevant to our use case. Primary among these are GAN inversion methods that aim to embed any given input to the latent space of a pretrained GAN so that the GAN can reconstruct the input using the latent code. The two approaches for this include either optimizing a latent code for any given input [1, 29], or training an encoder to predict the latent code [56, 6, 75]. While the optimization approaches can invert a given input better, they scale poorly compared to encoder based approaches when inverting many input samples. StyleGAN has also been studied closely in regard to its structure of latent spaces, including their suitability to embed inputs [1, 72, 73], discovering semantically meaningful directions or trajectories in latent space [25, 69, 65] and hierarchical style control with latent vectors for different resolutions [35].

Finally, the training loss and the optimization can be summarized as:

In the above equations, the loss hyperparameters are denoted by $\alpha,\beta,\gamma,\delta$. During training, the overall training loss is simultaneously optimized w.r.t parameters of only $\Psi,\Theta$ and $\Omega$, kee** $f$ and $G$ fixed.

(2) Concept visualization: Once the importance of a concept function is estimated, one can extract the most important concepts for a sample $x$ or class $c$ by thresholding $r_{k}(x)$ or $r_{k,c}$ respectively. For visualizing any concept $\phi_{k}$, following previous CoINs, one can start by selecting most activating training samples for $\phi_{k}(x)$ over the whole training set or separately for each class it is highly relevant for. However, the core of our concept visualization process, is to utilize generator $G$ to visualize the impact of $\phi_{k}$ on any input $x$ it is relevant for. We do so by (1) directly modifying activation of $\phi_{k}(x)$ by a factor $\lambda\times\phi_{k}(x),\lambda\geq 0$ while kee** all other activations in $\Phi(x)$ intact, and (2) Visualizing generated output for increasing value of $\lambda$. The case of $\lambda=1$ corresponds to $\tilde{x}$, the reconstructed version of $x$. This process is summarized in Fig. 3. Extremely high values of $\lambda$ can push the predicted latent vectors far from the average latent vector in which case the generated output is less reliable. For our experiments, qualitatively we found maximum $\lambda$ upto 3 or 4 reliable with consistent modifications.

Fidelity of reconstruction . Since reconstructed output plays a crucial role in our visualization pipeline, we evaluate how well does $G$ reconstruct the input. We do so by computing averaged per-sample mean squared error (MSE), perceptual distance (LPIPS) [82] and distance of overall distributions (FID) [28] of reconstructed images $\tilde{x}$ and original input images $x$.

Faithfulness of concept dictionary $\Phi$ . The aspect of faithfulness for a generic interpretation method asks the question “are the features identified in the interpretation truly relevant for the prediction process?” [7, 52]. This is generally computed via simulating “feature removal” from the input and observing the change in predictor’s output [27]. Simulating feature removal from input is relatively straightforward for saliency methods compared to concept-based methods, for example by setting the pixel value to 0. For CoIN systems, this is significantly more tricky, as concept activations $\phi_{k}(x)$ don’t represent the input $x$ exactly. However, through the decoder, we can evaluate if the concepts identified as relevant for an input encode information that is important for prediction. We adopt an approach similar to a previous proposal of faithfulness evaluation for audio interpretation systems [52]. Concretely, for a given sample $x$ with activation $\Phi(x)$, predicted class $\hat{c}$ and a threshold $\tau$, we first manipulate $\Phi(x)$ such that $\phi_{k}(x)$ is set to 0 if $r_{k}(x)>\tau,\forall k\in\{1,...,K\}$. That is, we “remove” all concepts with relevance greater than some threshold. This modified version of $\Phi(x)$ is referred to as $\Phi_{rem}(x)$. To compute faithfulness for a given $x$, denoted by $\text{FF}_{x}$, we compute the change in probability of the predicted class from original reconstructed sample $\tilde{x}=G(\Omega(\Phi(x),\Phi^{\prime}(x)))$ to new sample $x_{rem}=G(\Omega(\Phi_{rem}(x),\Phi^{\prime}(x)))$, that is, $\text{FF}_{x}=g(\tilde{x})_{\hat{c}}-g(x_{rem})_{\hat{c}}$. Ideally, we expect to see a drop in probability ($\text{FF}_{x}>0$) if the set of relevant concepts “truly” encode information relevant for classification. Following [52] we report the median of $\text{FF}_{x}$ over the test data for different thresholds $0<\tau<1$.
Consistency of visualization . We expect during visualization of a given concept $\phi_{k}$ that a user observes similar semantic modifications across different images. Thus, we hypothesize that if modifying any specific concept activation $\phi_{k}(x)$ leads to consistent changes for different samples $x$, then generated output for two versions of $\Phi(x)$, one with $\phi_{k}(x)$ set to a large value and one with $\phi_{k}(x)=0$, should be separable in the embedding space of $f$ (all other concept activations unchanged). In other words, embeddings for images with high $\phi_{k}(x)$ and low $\phi_{k}(x)$ should be well separated. To compute this metric, we first create a dataset of generated images with two different sets of activations. For each of training and test set, this is done by first selecting a set of $N_{cc}$ samples for which $\phi_{k}$ is highly activating and relevant for. Then we find its maximum activation $\phi_{k}^{max}$ among these samples, and create two generated outputs for each of $N_{cc}$ samples, one $x_{k}^{+}$ such that $\phi_{k}(x_{k}^{+})$ is set to $\lambda\phi_{k}^{max}$ with $\lambda\geq 1$, and the other $x_{k}^{-}$ such that $\phi_{k}(x_{k}^{-})=0$. The two sets of generated images are then gathered into a single dataset $\mathcal{S}_{k}=\{(x_{k}^{+},1),(x_{k}^{-},0)\}$ such that $|\mathcal{S}_{k}|=2N_{cc}$, and we learn a binary classifier $\varphi_{k}:\mathcal{X}\rightarrow\{0,1\}$, from pooled feature maps of intermediate embedding of $f$. We train for binary classification for sets created from training data $\mathcal{S}_{k}^{\text{train}}$ and test on sets created from test data $\mathcal{S}_{k}^{\text{test}}$. Our concept consistency metric $CC_{k}$ for a given concept $k$ is thus obtained as the accuracy of the binary classifier on $\mathcal{S}_{k}^{\text{test}}$:

This performance is tabulated for each concept $k$ for a fixed $\lambda$, and mean and standard deviation across all concepts is reported.

Baselines . The primary comparison methods for us are CoINs that learn unsupervised concepts efficiently, even for large-scale images, FLINT [51] and FLAEM [59]. SENN [7] suffers from computational issues for large-scale images as it requires to compute jacobian of concept dictionary w.r.t input pixels for its loss computation. Thus, for all the metrics we compare with FLINT and FLAEM as our primary baselines. Additionally, for accuracy evaluation, we track the performance of our pretrained classifier $f$. Note that $f$ (ResNet50) is not an interpretable model and trained entirely for accuracy. Lastly, for faithfulness we compare with a “random” baseline that randomly selects concepts for whom activation is set to 0. Since there is no notion of threshold in selection, in order to make it comparable for a given threshold, we select the same number of concepts randomly as we would for our method. Our proposed system for all evaluations is abbreviated as VisCoIN (Visualizable CoIN).

Quantitative results . Tab. 1(a) reports the test accuracy of all the evaluated systems. Our proposed system, VisCoIN performs competitively with the pretrained $f$ considered uninterpretable and purely trained for performance. It also performs better than the other recent CoIN systems for more complex classification tasks (CUB-200 and Stanford Cars) with large number of classes and diverse images. Metrics quantifying the fidelity of reconstruction on test data are in Tab. 2(a). The other baselines only optimize for pixel-wise reconstruction and FLINT achieves a lower MSE than VisCoIN. However, crucially, reconstruction from our method approximates the input data considerably better, in terms of perceptual similarity (LPIPS) and overall distribution (FID), which highly contributes to better viewability. Tab. 2(b) tabulates the median faithfulness $\text{FF}_{x}$ for the evaluated systems on 1000 random test samples for different thresholds. The performance of Random baseline being close to 0 even for small thresholds indicates that a random selection of concepts often contains little information relevant for classification of the predicted class. In contrast, concepts identified relevant as part of $g$ in VisCoIN tend to encode information about input that noticeably affects classification. In regard to other CoINs, while the faithfulness results are competitive on CelebA-HQ, for more complex datasets, concept dictionary in VisCoIN is significantly more faithful than FLINT or FLAEM, which do not demonstrate more faithfulness than the Random baseline. Finally, the mean and standard deviation for consistency of all concepts is reported in Tab. 1(b) with $\lambda=2$. Concepts learnt with VisCoIN demonstrate a higher mean consistency compared to baselines. The deviation across concepts is also lower for our method. We also evaluate consistency with higher values of $\lambda$ and observe increased separation with better classification performance (Appendix 0.C).

Qualitative results . Fig. 4 shows visualization for different class-concept pairs across the three datasets that are determined to have high global relevance $r_{k,c}$ through predictive structure of $g(x)$, as described in Sec. 3.5. For each class-concept pair, we show two maximum activating training samples for the concept from the corresponding class, the reconstructed input from $\Phi(x)$ ($\lambda=1$) and the generated output with modified concept activation $\phi_{k}(x)$ by a factor $\lambda=4$. In all the illustrations, increasing the activation of the concept, i.e. moving from $\lambda=1$ to $4$, strongly emphasizes some specific concept in the generated output that can be clearly grounded to the input. For instance, increasing the activation of concept for “Red-eye” in Fig. 4(a) increases the size of red eye of the bird, a key feature of samples from class 25 (“Bronzed-cowbird”). We can also qualitatively verify that the reconstruction has a high enough quality that allows us to “view” the input sample through the generated output and ground the modifications in generated output to the input. However, we also observed that learnt concept functions can be prone to modifying more than one high-level feature in the image. For, e.g., in Fig. 4(d), increasing the concept activation increases both “eye-squint” and “beard” in the generated output. A longer discussion about this and other limitations is available in Appendix 0.F.

Ablation study . We ablate multiple aspects of our system, with detailed results in Appendix 0.D. Most notably, we observed a tradeoff induced by strength of reconstruction-classification loss (weight $\gamma$). A high $\gamma$ positively impacts the faithfulness but negatively impacts the perceptual similarity of reconstruction.