An Efficient SVD-Based Recommendation System with Data Preprocessing and Comprehensive Performance Evaluation

Abstract

The Recommendation system has now evolved to be an integral part of current e-commerce, media streaming, online learning, and social networking sites where the system recommends suitable products and content according to the past behavior of the user. Conventional collaborative filtering algorithms face issues related to sparsity of the data, scalability, and cold start problem which could adversely impact the quality of recommendations. This paper proposes an exhaustive implementation methodology using Singular Value Decomposition (SVD) in collaborative filtering, along with data pre-processing and evaluation of performance. The framework includes data cleansing, duplicates elimination, treatment of missing values, rating scaling, formation of user-item rating matrix and feature extraction to enhance the quality of the data before training the algorithm.The proposed framework is tested by using traditional recommendation system performance metrics such as RMSE (Root Mean Squared Error), MAE (Mean Absolute Error), Precision, Recall, and F1-Score. Experimental results have shown that systematic preprocessing coupled with SVD (singular value decomposition) based collaborative filtering method offers robust recommendation results while retaining efficiency. Proposed implementation framework can be used for many different recommendation domains including e-commerce, movies/media recommendations, review websites, and even learning systems among others.