Character level attention siamese networks for Chinese handwriting comparison

Abstract

Handwriting recognition comparison is a critical yet challenging task in financial, commercial, and exam cheating prevention applications. However, the drawbacks of existing methods are that they compare the whole sentence or the entire signature with multiple characters susceptible to data noise. Their accuracy fluctuates widely. Unlike signature verification which treats the entire signature as a single biometric pattern, or general handwriting comparison which analyzes text at the word or paragraph level, our approach operates at the character level, enabling finer-grained analysis with reduced sensitivity to spacing and alignment variations. In this paper, we propose a character-level attention siamese network for handwriting comparison. The proposed model is a 2-channel CNNs model with multiple spatial-attention convolutional layers. Each channel receives an image of a single Chinese handwriting character. The model outputs the similarity between the two characters. Extensive experiments show that our model’s attention mechanisms and the additional information of characters can effectively reinforce valid information about signature comparisons. Specifically, the proposed CLASN+Fonts model achieves a False Acceptance Rate (FAR) of 11.93%, False Rejection Rate (FRR) of 9.36%, and Accuracy (ACC) of 90.78%, representing a 44.06% reduction in FAR, 42.6% reduction in FRR, and 16.12% improvement in ACC over the baseline SVM method. Since no suitable Chinese character signature dataset is currently available, we collected a large-scale Chinese signature dataset with approximately 48,000 handwritten character images of 800 users and 3,000 character images of 50 fonts. Further analysis of experimental results demonstrates that the proposed methods extract the comparison feature of Chinese character handwriting and select the most informative region automatically during the training.