PLOT: Phrase-Based Language Model Optimization for Event Detection in Text Streams

Abstract

Event detection from text streams is a fundamental problem in natural language processing, with applications ranging from social monitoring to information retrieval. In Persian, the task is further complicated by the scarcity of labeled resources, the informal and highly dynamic nature of user-generated content, and the rich morphological and orthographic diversity of the language. Most existing approaches represent events as collections of isolated words, discarding the compositional semantic structure through which real-world events—particularly emerging ones—are most naturally expressed. We propose an unsupervised, phrase-based framework for event detection from Persian social media streams, in which semantically meaningful phrases serve as the fundamental units of event representation. Text segmentation is reformulated as an optimization problem, and the central contribution of this work is the definition of its objective function: rather than relying on traditional statistical resources, pre-trained language models are employed directly as the scoring backbone of the optimization process. The framework is instantiated and compared under three configurations—a statistical n-gram model built from Wikipedia, the multilingual model mBERT, and the Persian-specific model ParsBERT—enabling a controlled analysis of how the nature and capacity of the scoring function affect event detection quality. Experiments on a Persian Telegram dataset demonstrate that, under a unified evaluation protocol applied identically to all methods, the proposed framework outperforms word-based approaches and established baselines on both clustering coherence and topic recall, although no single parameter configuration of the proposed method maximizes both metrics simultaneously, and the reported gains are therefore metric-dependent. The language model–based configurations, in particular, show superior ability to identify novel and previously unseen phrases, owing to their capacity to evaluate semantic coherence independently of surface frequency. Beyond event detection, the principle of employing language model output distributions as formal objective functions within optimization algorithms represents a generalizable design paradigm with potential applicability to a broader class of natural language processing tasks.