Aug 2026· International Conference on Machine Vision and Deep Learning· Vol 14326, pp. 143263G - 143263G-9· 0 citations· 10 references
Engineering
TL;DR
This study validates the effectiveness of the BERT model in semantic similarity calculation, providing more accurate technical support for related application scenarios, and laying the foundation for subsequent model optimization and lightweighting research.
Abstract
Text semantic similarity calculation is a core technology in the field of natural language processing, and it is crucial in scenarios such as question answering systems and information retrieval. Traditional methods based on keyword matching or static word vectors are difficult to capture deep semantics of text and cannot effectively solve the problem of lexical ambiguity. This study aims to systematically evaluate the performance and advantages of the BERT model in text semantic similarity calculation. By constructing a multi-source dataset containing academic literature, online text, and standard evaluation data, and after preprocessing such as text cleaning and word segmentation standardization, a pre-training + finetuning training paradigm was adopted, and the model was trained by setting appropriate hyperparameters. The F1 score and correlation coefficient were used as core indicators to conduct comparative experiments with traditional models such as Word2Vec and GloVe. The results show that the BERT model achieves F1 scores of 0.84 and 0.91 on the CNSE and CNSS datasets, respectively, which are significantly better than traditional models, and it can effectively utilize contextual information to eliminate ambiguity and capture deep semantic relationships. This study validates the effectiveness of the BERT model in semantic similarity calculation, providing more accurate technical support for related application scenarios, and laying the foundation for subsequent model optimization and lightweighting research.
In this study, a hybrid approach to semantic text similarity combining distributed word embeddings with classical lexical similarity measures is developed. Analyzed are the limitations of modern deep learning models, namely computational overhead and weak interpretability in resource-constrained environments. Proposed is a hybrid architecture that integrates Word2Vec distributed representations with cosine and Jaccard lexical similarity metrics. Investigated is a weighted fusion mechanism that combines vector-based semantic distances with set-theoretic token overlap for robust scoring. Developed is a three-stage processing pipeline covering text preprocessing, sentence embedding generation, and similarity computation and fusion. Established is a tunable weighting parameter that experimentally balances semantic depth against lexical matching precision. Conducted are experimental evaluations on benchmark semantic textual similarity and paraphrase detection datasets using classification metrics. Determined is that the proposed hybrid model attains higher correlation with human judgment than standalone or traditional baselines. Demonstrated is a notable reduction of error rates for exact lexical matches frequently missed by vector-only models. Presented is an efficient and scalable solution that balances computational performance with semantic accuracy for practical tasks.
B. Muminov, N. Allaberganova, E. Ergashev et al.· International Conference on...· 0 citations
Semantic analysis has become a central challenge in natural language processing, driven by exponential growth in digitized textual data and the need for automated content processing across multiple applications including machine translation, text classification, sentiment analysis, and information retrieval. However, while semantic analysis methods are well-developed for resource-rich languages such as English, morphologically complex languages like Uzbek suffer from deficiencies in annotated corpora, lexical-semantic resources, and high-quality vector models – a gap amplified by governmental initiatives in digital economy development and national language technology advancement. This section grounds semantic analysis in the distributional semantics hypothesis principle that words exhibiting similar contexts possess similar meanings – thereby recasting the problem as a geometric challenge within continuous vector spaces. Two principal mathematical strategies are formalized: (1) prediction-based models (word2vec: CBOW/Skip-gram), which optimize context prediction objectives, and (2) count-based models (GloVe), which leverage global co-occurrence statistics through matrix factorization. Both project high-dimensional word co-occurrence relationships into low-dimensional dense vector spaces, enabling semantic analogy representation. For resource-scarce languages like Uzbek, cross-lingual embedding alignment (Procrustes optimization) enables semantic knowledge transfer from resource-rich languages, facilitating shared semantic spaces across the Turkic language family. The section concludes with formal problem specification: given vocabulary V and corpus C, semantic analysis is formalized as (1) a mapping problem preserving distributional properties, (2) an optimization problem minimizing loss through gradient-based methods, and (3) an evaluation problem assessing quality through semantic similarity, analogy, and downstream NLP task performance.
D. Akhmedjanova· Международный Журнал Теорети...· 0 citations
This paper presents a case study of developing a self-supervised keyword extraction system for Wanfang Data, a leading Chinese academic database platform facing the challenge of tagging millions of short-article titles without access to manually labeled data. The study proposed an attentive autoencoder framework that integrated a multihead self-attention mechanism for contextual representation, an importance scoring network for keyword ranking, and a dual-loss optimization function combining reconstruction loss and sparsity loss to enable accurate keyword identification without manual annotation. Using 7,278 real article titles from Wanfang's production environment as the test corpus, the study evaluated the system against five baseline methods. The results showed that the proposed approach achieved an F1-score of 0.3581 and a mean average precision of 0.4544 for top-three keyword extraction, outperforming all baselines and demonstrating 15–20% improvement in ranking accuracy on very short texts (tokens < 10).
Lei Peng· Journal of Cases on Informat...· 0 citations
Abstract The data science and artificial intelligence, optimizing information retrieval tasks has become crucial for extracting actionable insights from vast amounts of data. The problem is the need for precise query formulation to retrieve relevant data effectively, as LLMs can generate vast amounts of information that might include noise or irrelevant details. The objective of this study is to enhance the efficiency and accuracy of information retrieval tasks by leveraging large language models (LLMs) for data augmentation. Gathering a diverse dataset from various sources like online search engines, social media platforms, and online forums is crucial for meeting text and information needs effectively. The term frequency-inverse document frequency (TF-IDF) technique is applied to calculate the importance of each term in the dataset, allowing for the differentiation of significant words from common ones. This step is crucial in the data pre-processing phase to enhance the relevance and precision of information retrieval tasks. The goal is to optimize the normalized discounted cumulative gain (NDCG) metric, which measures the ranking quality of the retrieved documents. The framework integrates the fine-tuned LLM model with the chosen information retrieval system by incorporating the model’s outputs into ranking algorithms. The results show that the proposed method has the maximum accuracy, with an average accuracy of around 10 % when implemented using Python software. The future scope for optimizing information retrieval tasks with large language models (LLMs) for data enhancement is vast and promising.
C. Vaidya, Amudhavel Jayavel, Pradeep Kumar Mishra et al.· Journal of Nonlinear, Comple...· 0 citations
The massive volume of textual descriptions on hotel booking platforms makes it difficult for recommendation systems to accurately match user preferences. Traditional keyword-based retrieval methods, such as TF-IDF, often struggle to capture semantic relationships when relevant terms do not explicitly overlap. This study evaluates the performance of keyword-based (TF-IDF) and embedding-based (paraphrase-multilingual-MiniLM-L12-v2) retrieval approaches in a content-based hotel recommendation system using a small-scale dataset. The dataset consists of 30 unique Traveloka hotels in Yogyakarta collected from Kaggle, representing a resource-constrained experimental setting. Evaluation was conducted using a black-box approach with 10 dynamic synthetic queries and assessed through Precision@3 (P@3) and Mean Average Precision (MAP). The results indicate that MiniLM achieved higher retrieval effectiveness than TF-IDF, with a mean P@3 of 0.3667 and a mean MAP of 0.1378, compared with 0.3000 and 0.1333, respectively. These findings suggest that embedding-based retrieval is more effective in capturing semantic information, including synonym usage and implicit contextual relationships, within the evaluated dataset. Therefore, compact embedding models such as MiniLM may provide an alternative approach to traditional keyword-based retrieval methods for small-scale recommendation systems.
Ilham Yusuf Faturochman, Aprilisa Arum Sari, Nibras Faiq Muhammad· IC-ITECHS· 0 citations
Semantic text similarity is an especially hard task that can be performed in the Uzbek language because of its rich morphological and the absence of annotated data. The given paper introduces a very effective Bidirectional Encoder design using the monolingual model named “Bidirectional Encoder Representation from Transformers for Uzbek language” and optimized to provide scalable semantic search. A combination of knowledge distillation and metric learning are applied in a semi-supervised approach that is employed by the model. Triplet Loss with Hard Negative Mining is used to enhance the discriminative ability of the vector space. One such innovation is supporting the Matryoshka Representation Learning, which allows the model to produce dynamically truncated dimensions of embeddings. The architecture proposed wound up with a Spearman correlation of 0.835 within the Uzbek test set, topping the state-of-the-art results. Furthermore, Matryoshka Representation Learning achieves 6-fold compression of vectors with a negligible accuracy drop (ρ=0.816), having high-computation efficiency to implement Natural Language Processing systems in low-resource systems.
B. Muminov, N. Allaberganova, Olimjon Mamadiyorov et al.· International Conference on...· 0 citations