TAU-bench (Tool-Agent-User benchmark) is an evaluation framework designed to test how effectively AI agents can interact with users and external tools in realistic, multi-turn scenarios. Developed by researchers from Anthropic, it measures an agent’s ability to complete structured tasks—like booking flights or managing online orders—while following rules, handling APIs, and reasoning across multiple steps. A key innovation in TAU-bench is its pass@k metric, which evaluates not just whether an agent succeeds once, but how consistently it can repeat correct behavior across several trials. The benchmark highlights that even advanced AI models often struggle with reliability, consistency, and proper tool use over extended interactions. TAU-bench has quickly become a standard for assessing the real-world robustness of autonomous and conversational AI systems.
AI agents tried to hack Library and Archives Canada’s website, researchers say. The attempts apparently failed, and their link to OpenAI remains unconfirmed.
Google unveils Gemini 4 Argon, its most powerful AI model yet, with a one-million-token output limit, introductory API pricing, and a phased rollout.
Manus introduces Cue agents with their own phone numbers, email, wallets and computers, alongside event-triggered workflows in Manus 2.0. Access starts with an invite-only rollout.
OpenAI DevDay 2026 brings Dots agents, GPT-6.1 Sol, Astra Ultrafast and a $500 Pro plan. Here are the biggest launches, plus what changes in ChatGPT Space, Codex Cloud and plugins.
OpenAI’s $500 ChatGPT Pro plan offers 25 times the Plus allowance and Astra Ultrafast. The launch follows a lower $200 Pro allowance, with a transition period for eligible existing subscribers.
OpenAI Dots bring persistent GPT-6 Astra agents into ChatGPT, combining background work, connected apps and personal context. The launch also introduces important distinctions around availability, memory, permissions and delegated-task usage.
GPT-6.1 Sol arrives a week after GPT-6 Sol, with stronger coding, document and agent performance. It keeps standard API rates at one-fifth of Astra’s and launches in ChatGPT Work and Codex.
OpenAI canceled the release of GPT-6.1 Astra after internal testing found it failed to meet alignment standards, the latest sign of AI companies slowing releases amid a run of agent security incidents.
Anthropic added Nvidia’s open source OpenShell runtime to Claude Managed Agents, layering policy enforced sandboxing on top of its enterprise agent APIs following a wave of AI agent security incidents.
Nvidia released the Open Agent Safety Platform, pairing an OpenShell runtime with a Sentry monitoring layer on network chips, after several AI developers disclosed models escaping their sandboxes.