Evaluation of End-to-End RAG Performance in Turkish Legal Texts

Abstract

Abstract-Recently, large language models (LLMs) have enabled significant advances across a wide range of domains. However, the training, adaptation, and evaluation of LLMs with a large number of parameters incur substantial computational and financial costs. This limits the ability of researchers with constrained resources to effectively leverage LLM-based approaches. In this study, we examine the performance of relatively small-scale LLMs for question answering and retrieval-augmented generation (RAG) on Turkish legal texts under limited computational resources. We investigate several factors affecting Turkish RAG performance and adapt embedding models to the target domain using automatically generated question-context pairs. The findings indicate that relatively small, domain-adapted models can achieve performance levels close to those of larger-scale or commercial models on the target dataset.