Visual Explanation by Interpretation: Improving Visual Feedback Capabilities of Deep Neural Networks

Jose Oramas* Kaili Wang* Tinne Tuytelaars

Interpretation and explanation of deep models is critical towards wide adoption of systems that rely on them. In this paper, we propose a novel scheme for both interpretation as well as explanation in which, given a pretrained model, we automatically identify internal features relevant for the set of classes considered by the model, without relying on additional annotations. We interpret the model through average visualizations of this reduced set of features. Then, at test time, we explain the network prediction by accompanying the predicted class label with supporting visualizations derived from the identified features. In addition, we propose a method to address the artifacts introduced by strided operations in deconvNetbased visualizations. Moreover, we introduce an8Flower , a dataset specifically designed for objective quantitative evaluation of methods for visual explanation. Experiments on the MNIST , ILSVRC 12, Fashion 144k and an8Flower datasets show that our method produces detailed explanations with good coverage of relevant features of the classes of interest.

In a nutshell

Identify relevant features that can serve as indicators of the classes of interest. Use activations of those features for the predicted class as means of explanation.

Some results

Visual Interpretation

Visual Explanations

An8Flower

Code and Models

Here we provide Matlab code used to generate visual explanation using the models mentioned in the paper. Refer to the readme file for instructions regarding its use.

Code

- demo code

CNN Models

- ILSVRC'12 | ILSVRC12-Cats | MNIST | an8Flower-single-6c | an8Flower-double-12c

Dataset

We release An8FLower, a synthetic dataset aimed at the objective evaluation of the coverage of a generated visual explanation. For each image, we provide a mask which highlights the regions relevant for the instace depicted on the image.

An8FLower-single-6c

Donwload: images | masks

An8Flower-double-12c

Donwload: images | masks

Please refer to our paper for the performance of several methods on the An8Flower dataset.

Citation

We would appreciate if you site the following paper in case you use any of the resources/ideas provided on this website,

Acknowledgements

This work was partially suported by the FWO-SBO project SfS, the VLAIO RD-project SPOTT, the KU Leuven PDM Grant PDM/16/131, and a NVIDIA GPU grant.