Explaining image classifications using SHAP

Deep learning models used for image recognition and classification are often so complex that they are effectively black boxes. Investigate and develop methods for explaining these using concepts from game theory

In recent years, deep learning has shown promising results across numerous fields such as natural language processing, robotics, and computer vision. For computer vision, neural networks have almost single-handedly taken over tasks like image classification and segmentation through its innate ability to automatically discover complex features and generalize well to large datasets. Despite these excellent results, deep neural networks are not easy to interpret, especially among non-experts, and are generally considered a "black box". Research on how to explain these black boxes has become a trending topic in the field of AI, where many works try to visualize intermediate steps in the network to get a better understanding of what the network "sees" performing a specific task like image classification. Although these visualizations can be essential in debugging and maintaining a performant model, some argue that the actual explanations leave something to be desired. In this respect, SHAP is a recent proposal for better explanations of machine learning models using an approach from game-theory called Shapley values.

Supervised machine learning (ML) entails learning the relationship between a set of input variables (X1,X2, X3, ...) and a target variable Y, and the task of the ML model is to approximate the function f such that f(Xi)≈Y.
When approaching the question “How does each of the input variables affect the model outcome?”, the game-theoretic concept of Shapley values is useful.

In this project, the student should get familiar with Shapley values, and the computationally more efficient approximation SHAP (SHapley Additive eXplanations).
The project should investigate the use of SHAP to explain image classification models, compare the results to traditional methods and ideally also investigate the underlying assumptions of SHAP and their validity.

This project would suit an ambitious student with an interest in research.


Investigate and develop methods for explaining image recognition and classification models

Learning outcome

Some statistics knowledge and programming experience is required.


Programming and some statistics


  • Inga Strümke
  • Stephen Hicks

Contact person