Early Detection of Skin Cancer Using A Deep Learning Method and Optimization Methods

Abstract

The early and accurate diagnosis of skin cancer, particularly malignant melanoma, is critical for patient survival, yet it poses significant challenges due to the visual complexity of dermoscopic images and inherent dataset imbalances. This thesis investigates and develops a comprehensive pipeline for automated skin lesion classification using the ISIC 2017 dataset. The methodology encompasses a rigorous multi-stage preprocessing workflow, including hair removal and image enhancement, followed by an extensive data augmentation strategy to create a balanced training set (4116 images) from the originally imbalanced data. The study comparatively evaluates multiple segmentation techniques (U-Net, SAM) and classification strategies. The primary classification approach involves using a pre-trained EfficientNetB7 model as a deep feature extractor from whole images, with subsequent classification performed by various machine learning models, including Support Vector Machines (SVM) and Random Forests. Experimental results on a validation split of the balanced data showed promising performance, achieving up to 84.1% accuracy. However, a critical performance drop to approximately 66% accuracy was observed on the independent, imbalanced ISIC 2017 test set, with recall for the crucial melanoma class falling below 10%. This study concludes that while robust preprocessing and data balancing can yield high performance in a controlled setting, significant challenges in model generalization persist. The performance gap highlights that domain shift and a lack of robustness to real-world class imbalance are major obstacles, underscoring the need for advanced techniques to create clinically reliable automated diagnostic systems.