Optimizing sample selection for large language model-based entity matching using AssistEM
TL;DR
AssistEM, a framework for efficient LLM adaptation to EM via principled data selection, demonstrates that selective fine-tuning not only accelerates adaptation but also improves training efficiency (requiring fewer GPU hours), enabling open-source LLMs to rival–and in some cases outperform–closed-source models.
Abstract
The meteoric rise of large language models (LLMs) has reshaped natural language processing, inspiring new approaches to data integration tasks such as entity matching (EM). While proprietary models like GPT-4 deliver strong performance, open-source alternatives (e.g., Mistral, DeciLM, Gemma-3) require supervised fine-tuning (SFT) to excel in specialized domains. However, naïvely training billion-parameter scale LLMs on uncurated corpora is computationally prohibitive and environmentally costly, often taking days while introducing inefficiencies that limit rapid application deployment. We introduce AssistEM, a framework for efficient LLM adaptation to EM via principled data selection. AssistEM integrates NER and TF-IDF signals into a composite score, filters entity pairs with domain-level thresholds, constructs denoised semantic embeddings, and applies graph-based clustering with class-aware sampling to obtain a compact yet representative set of ∼\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\sim $$\end{document}8000 high-quality training instances. This yields rapid specialization: AssistEM-trained LLMs converge in under 4.5 hours–versus 12+ hours for DeciLM-7B and 24+ hours for Yi-34B–while surpassing GPT-4-0613 by 4.3 F1 points and Ditto by 11 F1 points across diverse EM benchmarks. By aligning data quality with model specialization, AssistEM demonstrates that selective fine-tuning not only accelerates adaptation but also improves training efficiency (requiring fewer GPU hours), enabling open-source LLMs to rival–and in some cases outperform–closed-source models. These results highlight data selection as a critical lever for sustainable, domain-specific LLM deployment.