The Invisible Signature: A Stylometric and Probabilistic Approach to Detecting AI-Generated Text in Academic Thesis Introductions

Abstract

Since ChatGPT became publicly available in November 2022, the use of Large Language Models (LLMs) in academic writing has spread well beyond institutions with abundant digital resources. The University of Madagascar is no exception: conversational AI assistants, accessible via a simple smartphone, have become commonplace tools among master's and doctoral students. Yet unlike traditional plagiarism, AI-generated text is, by definition, original, which renders textual comparison software powerless. This study explores an alternative approach grounded in stylometry and probabilistic analysis. Drawing on a corpus of 600 thesis introductions, 300 written by Malagasy students between 2015 and 2019 and 300 generated by GPT-4 on the same topics, we extract two types of markers: classical lexical indicators (TTR, POS tag distribution) and probabilistic metrics (perplexity, burstiness). Our results confirm that AI-generated texts exhibit significantly higher syntactic regularity (mean PPL = 21.6 vs. 87.3). We also uncover an under-documented signal: the frequent presence of typographic artefacts (asterisks, double dashes, Markdown icons) in 63% of generated texts, entirely absent from human writing. Finally, we discuss the risk of false positives for writers whose language of instruction is a second language.