Understanding Stylistic and Syntactic Backdoor Attacks in Text Classification

Abstract

Text classification models are widely deployed in real-world systems, making their security a key concern. Backdoor attacks are among well-known threats, and style transferbased (StyleBkd) and syntax template-based (SynBkd) attacks are two prominent types of backdoor attacks. Although StyleBkd and SynBkd intend to bind a target label to stylistic or syntactic patterns, poisoned samples also introduce distinctive lexical artifacts, e.g., a set of frequently repeated, attack-correlated words. This raises the question of whether backdoored models indeed learn the style or syntax, or do they instead memorize the attackcorrelated words. In this paper, we present a systematic study of this question. We introduce four complementary methods that progressively perturb lexical content, semantic similarity, and sentence structure. We evaluate how these methods affect attack success rates (ASR) across three model types and four datasets. Our results show that word-level substitutions, even when replacing the most attack-correlated words with semantically distant alternatives, typically yield limited reduction in ASR. In contrast, holistic paraphrasing and syntactic reordering of sentences can substantially reduce ASRs, particularly on datasets with shorter samples. Our findings indicate that StyleBkd and SynBkd are not primarily activated by memorization of specific words, but rather by stylistic and syntactic structures, contributing to a better understanding of backdoor effectiveness and dynamics.