In this paper, we describe our approach for the classification of medical images depicting the human gastrointestinal tract. Search-based classification is performed in three stages. In the first stage, we extract deep features for each image using pre-trained deep-learning models. In the second stage, we use LIRE  to index the generated features, so that we can then, in the final stage, search the index for similar images and make our predictions based on the results. With this approach, we achieved a MCC score of 0,54 and a accuracy of 0,94, which shows that deep features combined with search-based classification are a viable option for medical image analysis.