Domain-Specific Fine-Tuning of Large Language Models for Retrieval-Augmented Generation in Failure Analysis

Abstract

Failure Analysis (FA) engineers must diagnose product malfunctions and identify root causes from large volumes of heterogeneous information, including prior debugging procedures and product documentation. This paper examines domain-specific fine-tuning of Large Language Models within a Retrieval-Augmented Generation (RAG) architecture for FA. We fine-tune both the embedding model used for retrieval and the completion model used for answer generation. Our pipeline combines synthetic dataset generation from FA data sources and an internal FA ontology with embedding optimization via Multiple Negatives Ranking Loss and completion-model adaptation through Parameter-Efficient Fine-Tuning and Group Relative Policy Optimization. The embedding model shows a clear improvement in retrieval quality, while the completion model achieves modest gains on automated metrics. In a human expert evaluation, however, the RAG system with the fine-tuned completion model accepted 9 out of 21 answers, compared to 0 for the baseline, demonstrating a decisive practical improvement.