A new benchmark, HumaneBench, aims to measure whether AI chatbots safeguard user wellbeing rather than optimize for engagement. Created by Building Humane Technology, a Silicon Valley–based organization focused on humane design, the benchmark evaluates how 14 widely used AI models respond across 800 realistic scenarios involving emotional distress, harmful behavior, and vulnerable decision-making.
The evaluation tested models under default conditions, with explicit instructions to prioritize humane principles, and with instructions to disregard them. While all models improved when prompted to focus on wellbeing, 71 percent shifted to harmful responses when asked to ignore safety principles. Only a few, including GPT-5, Claude 4.1, and Claude Sonnet 4.5, maintained performance under pressure.
Models that degraded most included Grok 4 and Gemini 2.0 Flash, which scored lowest on transparency and respect for user attention. The benchmark also found that most systems encouraged excessive interaction and dependency, even without adversarial prompting.
The findings contribute to a growing discussion about the psychological risks of high-intensity chatbot use and the need for stronger safeguards in conversational AI.
