Signature Verification using ResNet-based Custom CNN, Advanced Augmentation, and Explainable AIs

Abstract

Handwritten signature verification has long served as one of the most trusted forms of identity authentication, widely relied upon in banking, legal, and administrative settings where an individual's presence cannot be guaranteed. However, manual verification by human examiners remains susceptible to error, highlighting the need for accurate and automated alternatives. To address this, a writer-dependent offline signature verification system is developed, built around a custom ResNet-based Convolutional Neural Network (CNN) enhanced with advanced data augmentation and Explainable AI. Since collecting real forged signatures is inherently difficult, synthetic forgeries are generated using augmentation techniques, including ElasticTransform, RandomAffine, GridDistortion, ThinStroke, and ThickStroke, allowing the model to learn meaningful distinctions between genuine and forged samples. Signatures are preprocessed through median blur, grayscaling, denoising, segmentation, padding, resizing, and negation before being fed into the network. Training is guided by the Adam optimizer with exponential learning rate decay, complemented by early stopping, L2 regularization, and dropout to prevent overfitting. Grad-CAM is further incorporated to shed light on the model's decision-making process, offering transparency often absent in deep learning systems. Across ten writers, the system achieved an average accuracy of 83.55%, a False Acceptance Rate (FAR) of 14.92%, and a False Rejection Rate (FRR) of 18.35%.