Vulnerabilities in Personalization: Assessing Health Privacy Risks in ChatGPT Logs and Memory
TL;DR
A large-scale computational audit analyzing 179,057 conversations across India, Nigeria, Brazil, and Pakistan is presented to evaluate personal health disclosures and background memory synthesis in ChatGPT and concludes with sociotechnical design guidelines to restore user agency and consent-driven boundaries in stateful AI systems.
Abstract
As conversational LLMs become deeply embedded in daily life, users frequently disclose sensitive personal health information during routine interactions. We present a large-scale computational audit analyzing 179,057 conversations across India, Nigeria, Brazil, and Pakistan (N = 1,057) to evaluate personal health disclosures and background memory synthesis in ChatGPT. We find that 21.31% of audited conversations contain personal health data, with 3.62% posing high-to-extreme privacy risks involving stigmatized conditions, direct identifiers, and precise locations. When evaluating the memory entries of ChatGPT, we uncover a stark disconnect between corporate framing and system behavior: over 95% of profile entries are implicitly extracted without explicit user prompts or consent. Furthermore, background memory synthesis selectively condenses temporary, symptom-level disclosures into permanent diagnostic traits, stripping contextual integrity and amplifying re-identification risks. We conclude with sociotechnical design guidelines to restore user agency and consent-driven boundaries in stateful AI systems.
