A Comprehensive Approach to Misspelling Correction with Contextual Word Embeddings and Levenshtein Distance

Abstract

Spelling errors are common in written text and can seriously affect both readability and the performance of natural language processing systems. This problem is especially challenging in Persian due to complex spelling rules, similar letter forms, and context-dependent word usage. In this work, we propose a spelling correction framework that combines contextual information from BERT masked language models with edit-distance-based candidate generation. The method is designed to handle both non-real-word errors, where the misspelled word is invalid, and real-word errors, where a valid word is used incorrectly in context. To support this task, we construct a large Persian dataset with systematically generated spelling errors based on common typing and linguistic patterns. We also introduce heuristic correction strategies and a biasing mechanism that improves the model’s focus on erroneous words. Extensive experiments using multiple Persian BERT variants show that the proposed approach consistently outperforms a strong BERT-based baseline and widely used external systems. The results demonstrate notable improvements in F1-score, particularly for context-dependent real-word errors, while maintaining efficient runtime performance. These findings suggest that the proposed method is well-suited for practical Persian spelling correction applications.