A New Hybrid Fusion Approach Based on Classical Methods (PCA, LBP) and Deep Learning (FaceNet) for Performance Improvement of Face Recognition Methods

Abstract

This article presents an empirical comparison of four automatic face recognition methods: Principal Component Analysis (PCA), Local Binary Patterns (LBP), the deep neural network FaceNet, and an original hybrid approach proposed by the authors, referred to as FLLF (Feature-Level Late Fusion). The experiments were conducted on a subset of the VGGFace2 database, comprising approximately 480 classes in the training set and 60 classes in the validation set. For closed-set identification, a new test set (20%) was extracted from the training set. Classification accuracy, training and inference times, and prediction confidence distributions were evaluated for each method. The results show a clear advantage of the deep learning approaches: FaceNet achieved an accuracy of approximately 98% with only five training images per person, whereas the classical methods—PCA and LBP—reached only approximately 7% and 22%, respectively. The proposed FLLF method, which fuses FaceNet embeddings with PCA-whitened LBP descriptors at the feature level and classifies them with a calibrated linear SVM, further improved accuracy to approximately 98.5% and produced the highest prediction confidence values of all tested methods. However, calibration quality was not directly assessed using standard metrics such as expected calibration error or reliability diagrams, so this observation should be interpreted as a confidence-distribution shift rather than a formal calibration improvement. The article also discusses the theoretical foundations of each algorithm, their respective advantages and limitations, and the architecture of the software system implemented for this study.