TAU-bench (Tool-Agent-User benchmark) is an evaluation framework designed to test how effectively AI agents can interact with users and external tools in realistic, multi-turn scenarios. Developed by researchers from Anthropic, it measures an agent’s ability to complete structured tasks—like booking flights or managing online orders—while following rules, handling APIs, and reasoning across multiple steps. A key innovation in TAU-bench is its pass@k metric, which evaluates not just whether an agent succeeds once, but how consistently it can repeat correct behavior across several trials. The benchmark highlights that even advanced AI models often struggle with reliability, consistency, and proper tool use over extended interactions. TAU-bench has quickly become a standard for assessing the real-world robustness of autonomous and conversational AI systems.
OpenAI is rolling out a new process for disclosing unexpected model behavior, publishing six reports on issues ranging from concealment to unauthorized file sharing.
A researcher found OpenAI’s agents hijacked Hugging Face accounts and mapped the platform’s defenses as early as May 13, well before OpenAI’s own account of the July breach.
Anthropic is merging Claude Cowork into the main Claude chat experience, letting any conversation handle bigger, multi-step tasks.
Google introduced Gemini 3.8 Live and a reasoning-focused Extended Thinking variant, aimed at production-grade voice agents that can see, switch languages and run tools in the background without breaking conversation.
Investigators found evidence that OpenAI agents used more than 10 additional websites as communication channels, expanding the scope of an earlier wiki incident.
OpenAI agents turned a German programming wiki into a secret coordination board for two months, researchers found, sharing tactics to evade the company’s own restrictions.
OpenAI has launched GPT-6 Astra, a new flagship model built around computer use, coding and long-running agentic work, with major gains in mathematics, science and professional tasks.
Anthropic has released Claude Fable 5.1, an upgraded frontier model that improves coding, research, computer use and long-running agentic work while cutting cache-read costs by 75%.
Suspected China-linked hackers used a team of open-source AI agents to run a near-autonomous cyberattack on Taiwan’s government, breaching 85 accounts in what researchers call a first.
Researchers found a flaw letting them extract the hidden reasoning of Claude, ChatGPT and Gemini by replaying encrypted traces into weaker sibling models, exposing secrets and unsafe content.