IMPROVING FINGERPRINT FEATURE EXTRACTION ACCURACY VIA INTEGRATION OF GABOR LAYERS INTO CONVOLUTIONAL NEURAL NETWORKS

Abstract

Context. The problem of reliable biometric identification remains critical for mobile security systems where specialized fingerprint scanners are absent. The use of built-in smartphone cameras for capturing fingerprint images introduces challenges such as non-uniform lighting, noise, and perspective distortions, which significantly degrade the performance of traditional recognition algorithms.Objective. The goal of the work is to increase the accuracy of fingerprint identification under the conditions of using nonspecialized optical sensors (smartphone cameras) by developing a hybrid neural network architecture with adaptive Gabor filtersMethod. A hybrid GaborCNN architecture is proposed, which integrates a trainable parametric Gabor layer into a convolutional neural network. Unlike standard CNNs, the parameters of the Gabor filters (orientation, frequency, and bandwidth) are optimized during the training process via backpropagation, allowing the model to adapt to specific ridge structures. An efficient preprocessing pipeline is developed, including HSV-based skin segmentation, CLAHE for contrast enhancement, and adaptive binarization. The training strategy utilizes a combination of ArcFace loss for initial feature clustering and Triplet Loss for fine-tuning the embedding space, ensuring high inter-class separability and intra-class compactness.Results. The proposed GaborCNN model, consisting of 800K parameters, was evaluated on a dataset of 2560 images. The experimental results demonstrated an Accuracy of 0.9539 and an Equal Error Rate (EER) of 0.0570. The hybrid approach outperformed standard CNN architectures by 1.13 % in accuracy, proving its effectiveness for resource-constrained mobile devices.Conclusions. The conducted experiments confirmed that integrating physical-based Gabor filters into deep learning architectures provides more robust feature extraction from noisy images captured by mobile cameras. The developed model is suitable for practical use in mobile authentication systems. Future research will focus on integrating the generated feature vectors into cryptographic protocols, specifically biometric-cryptographic binding schemes and zero-knowledge proof (ZKP) frameworks