Beyond The Surface: Characterizing Adversarial Boundaries in Synthetic Text Attribution Across Heterogeneous Domains

Abstract

The influx of large language models (LLMs), like GPT-4, Claude, and Llama, has made distinguishing between natural and artificial content more difficult. The existing detection algorithms have three inherent drawback: they have limited ability to detect on short sentence-length texts containing less than 60 words, they are domain-specific, and they are fragile to adversarial attacks that involve synonym-replacement and paraphrasing. To overcome all three limitations, this paper proposes a hybrid detection framework which combines semantically deep embeddings from the RoBERTa transformer with a set of carefully designed language statistics (vocabulary richness, burstiness, and information entropy) and linguistic statistics (part-of-speech distributions, Flesch Reading Ease scores). The feature vector of the resulting embedding a 778-dimensional vector is handed to an ensemble of gradient-boosting trees, specifically XGBoost. The experiments are carried out on a processed database of 27,333 essays from professional, technical and social media writing, that is written by students and generated by AI tools. The proposed model is able to classify 99.00%, with an F1-score of 0.9932 and an area under the ROC curve (AUC) of 1.0. The framework achieves a 99.67% accuracy rate for perfect precision and recall for AI-generated content in the presence of synonym-based adversarial paraphrasing, showing excellent resistance to the surface-level adversarial paraphrasing strategy. Near-perfect cross-domain generalization is demonstrated and inference latency is 0.351ms per sample on average, which makes them suitable for realtime applications. These findings pave the way for scalable, accurate, adversarial-resilient AI-generated text detection using hybrid feature fusion, a paradigm that is encouraged by the results presented herein.