The Effect of Morphological Normalization on Retrieval Performance and Answer Accuracy in Turkish RAG Systems

Abstract

Retrieval-Augmented Generation (RAG) is a technique that enables large language models to leverage external knowledge sources to improve response quality. This study investigates the effect of morphological normalization on document retrieval and answer quality in Turkish RAG systems. Three indexes were constructed on the RAGTURK dataset: (i) RAW, using surface-form text, (ii) LEMMA, applying full lemmatization, and (iii) HYBRID, combining both representations. Retrieval performance was measured using Recall and MRR, while RAG answer quality was evaluated with ROUGE and BERTScore metrics. Results show that lemmatization does not improve dense retrieval performance for the multilingual embedding model used in this study, and leads to consistent degradation across all query categories. Similarly, RAG answer quality declined under normalization, with RAW indexing achieving the highest performance throughout. These findings suggest that modern multilingual models are capable of implicitly capturing Turkish morphological variation, and that additional normalization is not always necessary.