A KV cache, or key-value cache, speeds up autoregressive transformer inference by storing attention information calculated for tokens that have already been processed. Without the cache, the model would recompute those keys and values each time it generates another token. Reusing them reduces repeated work and improves response latency, especially for long prompts and outputs. The tradeoff is memory: cache size grows with sequence length, batch size, layer count, and model dimensions. Production systems manage this cost with paging, quantization, eviction, prompt caching, and request scheduling. KV caching changes inference efficiency, not the model’s learned knowledge, and cached data must be protected when prompts contain sensitive information.
Chinese startup Moonshot released Kimi K3, a 2.8-trillion-parameter open-weight model it says rivals Anthropic’s Claude, as the US-China AI gap narrows.
Anthropic is scheduling investor meetings ahead of a potential IPO as soon as October, a move that could take the Claude maker public before rival OpenAI.
OpenAI launched Codex Micro, a $230 mechanical keypad built with Work Louder that gives developers physical controls for its Codex coding agent, its first hardware to reach market.
NVIDIA unveiled Cosmos 3 Edge and said Japan’s robotics and manufacturing leaders, from FANUC to Sony, intend to join its Cosmos Coalition to build AI for the physical world.
Microsoft is coaching its sales force to argue its AI products beat OpenAI’s and Anthropic’s on cost, security and completeness, turning on the partners whose models it long relied on.
A Reuters analysis found xAI installed 59 unpermitted gas turbines to power its Colossus 2 data center, releasing pollution near predominantly Black Memphis-area communities.
OpenAI’s first hardware, reportedly designed with Jony Ive, is a portable, screenless smart speaker built as a proactive AI companion that learns its owner and moves on its own.
Twenty-six Meta workers sued the company, alleging its AI systems used metrics like token consumption to select layoff targets, disadvantaging people on medical or family leave.
Visa, Mastercard, Stripe, Google and 36 others formed a Linux Foundation body to steward x402, an open standard letting AI agents pay for services directly over HTTP.
OpenAI’s new GPT-5.6 prompting guide urges developers to write shorter, outcome-first prompts, saying leaner instructions raised scores 10-15% while cutting tokens and cost sharply.