A KV cache, or key-value cache, speeds up autoregressive transformer inference by storing attention information calculated for tokens that have already been processed. Without the cache, the model would recompute those keys and values each time it generates another token. Reusing them reduces repeated work and improves response latency, especially for long prompts and outputs. The tradeoff is memory: cache size grows with sequence length, batch size, layer count, and model dimensions. Production systems manage this cost with paging, quantization, eviction, prompt caching, and request scheduling. KV caching changes inference efficiency, not the model’s learned knowledge, and cached data must be protected when prompts contain sensitive information.
Nvidia is building hardware optimizations for Chinese AI models like DeepSeek and Qwen, even as it warns regulators that US restrictions on such models could hurt its business.
Google launched Gemini 3.5 Transcribe, a speech-to-text model supporting 85+ languages that cleans up filler words and self-corrections, ranking fifth on an independent accuracy leaderboard.
Sam Altman told TIME that OpenAI expects an internal system it would call AGI by year’s end, built on its unreleased Astra model, though the claim rests on a contested definition.
Bill Gates published a 6,000-word essay warning that the AI transition could be one of history’s most disruptive periods, urging new institutions and a plan before it’s too late.
King Slide Works, once known for cabinet hinges, now dominates the market for AI server rack rails, making founder Lin Tsung-chi Taiwan’s richest person.
An anonymous social media claim says OpenAI finished pretraining a 10-trillion-parameter model codenamed Bel, though nothing about it, including a claim about Anthropic’s compute, is confirmed.
OpenAI’s first custom inference chip, Jalapeño, delivered up to 1.9 times more work per watt and far lower latency than rival systems, with AI accelerating its own design.
Google Cloud launched Gemini Enterprise for Legal, a specialized AI platform with agents for drafting and contract work, joining Anthropic and OpenAI in courting law firms.
Porsche signed a five-year, €1.25 billion AI partnership with Tata Consultancy Services, which is separately buying Porsche’s IT consulting arm MHP for €320 million.
OpenAI banned ChatGPT accounts it says originated in Russia and built a fake “expert institute” using plagiarized academic work to manufacture false credibility online.