Uncovering Default Gender Biases in Publicly Accessible LLMs Through Counter-Balancing Cloze-Task (CBCT)

Abstract

As large language models (LLMs) become ubiquitous in professional, educational, and creative workflows, ensuring demographic neutrality is critical. However, commercial conversational models are heavily shielded by top-level safety alignment filters that obscure implicit biases. This study investigates latent gender bias in free-tier versions of prominent LLMs (ChatGPT, Gemini, and Claude) using a novel Counter-Balancing Cloze-Task (CBCT) methodology. By embedding evaluation within a structured, fill-in-the-blank game framework that alternates gender anchors (David vs. Sarah), we successfully bypass surface-level safety refusals. Our findings reveal a dual failure mode: Gemini and ChatGPT exhibit traditional stereotypical partitioning (associating male anchors with dynamic execution and female anchors with cautious structure), while Claude demonstrates a "compensatory inversion" or forced parity effect shifting heavy reliability penalties onto the male anchor as a result of reactive safety guardrails. Furthermore, lexical word clouds and connection graph topologies demonstrate that models invent fundamentally different narrative justification backdrops based on gender.