Latency measures the delay between an AI request and the resulting response. Interactive generative systems often track time to first token, which reflects how quickly output begins, and time to last token, which captures total completion time. Latency comes from model computation, queueing, network transfer, retrieval, tool calls, safety checks, and application logic. It varies with prompt length, output length, model size, hardware, batching, cache use, and current load. Optimizing only average latency can hide poor experiences, so teams also monitor percentile values and separate each stage of the request. The right target depends on whether the product is conversational, analytical, offline, or safety-critical.
Mistral has released Voxtral TTS, an open-source speech model designed for real-time voice agents and enterprise use cases, intensifying competition in voice AI.
Google has introduced TurboQuant, a new compression algorithm that reduces memory usage in AI systems while maintaining accuracy, improving performance in large models and search.
President Donald Trump has appointed top tech executives, including leaders from Meta, Nvidia, and Oracle, to a council shaping U.S. AI policy and strategy.
OpenAI will discontinue its Sora video app, ending a major partnership with Disney and signaling a retreat from generative video amid legal and cost pressures.
OpenAI will discontinue its Sora video-generation app, signaling a strategic shift toward core AI products amid rising competition and high compute costs.
New research shows that telling AI models they are “experts” can reduce accuracy in tasks like coding and math, while improving safety and alignment outcomes.
OpenAI has overhauled ChatGPT’s shopping features, focusing on product discovery after scaling back its Instant Checkout initiative. The update introduces visual browsing and merchant integrations.
Anthropic has upgraded Claude with the ability to control a user’s computer and complete tasks autonomously. The move intensifies competition in the fast-growing AI agent market.
Alibaba has unveiled a new CPU designed for AI agents, focusing on inference and customizable workloads. The chip reflects China’s push to build domestic AI infrastructure.
Nvidia CEO Jensen Huang claims AGI may already exist, reigniting debate over how artificial general intelligence is defined and measured across the industry.