Face Image Analysis via Transformers : Algorithms and Applications

Abstract

Drowsiness is a major cause of traffic accidents worldwide, impairing perception, slowing reactions, and reducing decision-making. Accurate and reliable detection is therefore essential for enhancing road safety and mitigating fatigue-related incidents. Face image analysis has emerged as a promising approach for non-intrusive drowsiness detection, yet existing methods often struggle with robustness, temporal modeling, and reliance on large labeled datasets. This thesis investigates advanced deep learning frameworks to enhance driver drowsiness detection through robust learning of fatigue-related facial cues and data-efficient strategies. First, a Transformer-based framework is proposed, leveraging hierarchical self-attention mechanisms for extracting discriminative fatigue-related facial features. Driver faces are detected and aligned using Multi-Task Cascaded Convolutional Neural Networks (MTCNN), followed by a Swin Transformer to capture spatial facial patterns. The method demonstrates competitive performance on the NTHU Drowsy Driver Detection (NTHU-DDD) dataset, highlighting the effectiveness of Swin transformers in face-based fatigue analysis. Second, a semi-supervised learning framework is developed to reduce reliance on large labeled datasets while maintaining real-time applicability. YOLOv8 is employed for fast face detection, and a Swin Transformer learns drowsiness-related representations from sequential frames. Pseudo-labeling enables progressive incorporation of unlabeled data, improving model generalization across diverse datasets, including NTHU-DDD, YawDD, and UTA-RLDD. Finally, a computationally efficient driver-monitoring system is introduced, combining a custom CNN enhanced with channel attention and semi-supervised training. This framework leverages both labeled and unlabeled data to improve generalization while reducing annotation requirements. Evaluated on the UTA-RLDD dataset, the system achieves high accuracy and low computational complexity, ensuring suitability for practical deployment in real-world driving scenarios.