Improving Brain Tumor Classification Using Knowledge Distillation

Abstract

The increasing demand for deploying deep learning models on mobile and edge devices has brought the need for lightweight and efficient neural networks to the forefront of research. This is particularly crucial in medical imaging, where real-time and accurate diagnostic support is required. However, most high-performing models rely on large architectures, making them impractical in constrained environments. This thesis proposes an enhanced knowledge distillation (KD) framework that aims to build compact student models while preserving high diagnostic performance for brain tumor classification. The approach leverages a training-free student selection method based on the DisWOT score and introduces two key enhancements: the use of Layer-wise Relevance Propagation (LRP) instead of Grad-CAM for fine-grained semantic supervision, and cosine similarity instead of L2 loss for robust and scale-invariant feature alignment. Our experimental evaluations on a publicly available brain MRI dataset demonstrate that our improved KD pipeline significantly lessens the quantity of parameters and computation cost with minimal impact on accuracy of the teachers. Our best student model achieved 95.35% classification accuracy, while reducing over 85% of the parameters in the teacher model. Our results substantiate the use of high-order distillation signals for generalization and training stability. In conclusion, the proposed framework provides an effective and efficient solution for interpretable lightweight model development, and has great potential for use in real-world contexts such as in medical or other constrained resource scenarios.