Eugenia Kim

PyRIT: Democratizing AI Red Teaming Through Open-Source Tooling

Beyond"I Can't Help With That": How Child Safety Experts Evaluate AI Chatbot Safety

Youth increasingly turn to AI chatbots for social and emotional support, raising concerns about how these systems respond, especially in high-stakes situations. However, existing child safety evaluations of AI lack grounding in real-world harms that youth experience, rely on unvalidated assumptions about what counts as...

Probing Warmth-Mediated Harm in Speech-Enabled LLMs for Mental-Health Conversations

Audio LLM benchmarks measure understanding and dialogue quality, not whether speech-enabled models respond with relational warmth when a vulnerable user discloses a mental-health concern. We introduce a 7-turn scripted-disclosure probe grounded in WHO mental-health clinical guidelines, with each script run on the same...

Pluralis v0.1: Towards a Multicultural, Multimodal, Multilingual Benchmark for AI Risk and Reliability

This work introduces Pluralis v0.1, a novel multimodal, multi-regional, and multilingual dataset built from a culture-first perspective and calls upon the research community to utilize this foundation to advance the science of multilingual, multicultural evaluation to better support AI cultural alignment globally.

Psychological Influences of Conversational AI: Research and Design Directions for Reducing Harm and Promoting Well-Being

This work proposes a set of aspirational directions for guiding the behavior of general-purpose AI systems in ways that may reduce potential psychological harms and support user well-being, and identifies open questions and areas requiring deeper study.