A framework for structuring Thai medical documents using systematized nomenclature of medicine clinical terms and large language models

Abstract

Systematized Nomenclature of Medicine – Clinical Terms (SNOMED CT) enables semantically interoperable health data, but implementation in Thailand is hindered by language and workflow challenges. We evaluated whether large language models (LLMs) can automate SNOMED CT concept extraction from Thai clinical documents. The objectives of this study are to develop a proof-of-concept framework for automated clinical named entity recognition and SNOMED CT concept linking from Thai medical documents, and to compare the performance and resource requirements of closed-source and open-source LLMs on a human-annotated dataset. The dataset was manually annotated from 220 Thai medical documents (6,390 entities) from three outpatient departments. Our retrieval-augmented framework was evaluated using four LLMs (Gemini 2.5 Pro, GPT-4.1, Qwen 3, and MedGemma) as experimental based models which were comparatively analyzed for entity recognition using precision, recall, F1 score and concept linking using character-level metric, Intersection over Union (IoU), as well as resource consumptions. For entity recognition, Gemini achieved the highest performance (loose F1 83.31%). For SNOMED CT concept extraction ranked most similar (Top-1) by encoder models for retrieval and re-ranking, IoU scores were 39.42% (Gemini), 34.44% (GPT), 29.86% (MedGemma), and 28.62% (Qwen). Average processing time for end-to-end extraction was 5–6 minutes per document for Gemini and GPT, with cost estimates in United states dollars (USD) of USD 0.035 and USD 0.019 per document, respectively; local open-source inference required up to 10 minutes per document. This study established a proof-of-concept framework for automated clinical entity recognition and SNOMED CT concept linking from Thai medical documents, demonstrating practical LLM performance with trade-offs between accuracy, cost, and efficiency across closed-source and open-source models.