History, Development, and Principles of Representation Learning—An Introductory Survey

Abstract

Representation learning has become a cornerstone of artificial intelligence, designed to automatically extract low‐dimensional, meaningful features from high‐dimensional, sparse raw data. By drastically reducing the reliance on manual feature engineering, representation learning enhances model performance across a wide range of tasks. The field has evolved significantly over the past decades, transitioning from early linear methods, such as Principal Component Analysis (PCA), to modern deep learning paradigms powered by neural networks, generative adversarial networks (GANs), and pre‐trained models. Although the rapid development of representation learning has significantly promoted the progress of natural language processing (NLP), computer vision, and recommender systems, the general practitioners still have a poor understanding of its historical background, core principles, and wide range of applications. To some extent, this limits the full development of its potential. To this end, this survey aims to provide a comprehensive and easily understandable overview for a wider audience. This survey conducts a systematic literature review to tease out the evolution of representation learning and analyse its core drivers. At the same time, this survey deeply explains the basic principles of representation learning, and introduces its practical application cases in various fields. This survey also points out the main limitations of current models and prospects the future research directions.