A KV cache, or key-value cache, speeds up autoregressive transformer inference by storing attention information calculated for tokens that have already been processed. Without the cache, the model would recompute those keys and values each time it generates another token. Reusing them reduces repeated work and improves response latency, especially for long prompts and outputs. The tradeoff is memory: cache size grows with sequence length, batch size, layer count, and model dimensions. Production systems manage this cost with paging, quantization, eviction, prompt caching, and request scheduling. KV caching changes inference efficiency, not the model’s learned knowledge, and cached data must be protected when prompts contain sensitive information.
King Slide Works, once known for cabinet hinges, now dominates the market for AI server rack rails, making founder Lin Tsung-chi Taiwan’s richest person.
An anonymous social media claim says OpenAI finished pretraining a 10-trillion-parameter model codenamed Bel, though nothing about it, including a claim about Anthropic’s compute, is confirmed.
OpenAI’s first custom inference chip, Jalapeño, delivered up to 1.9 times more work per watt and far lower latency than rival systems, with AI accelerating its own design.
Google Cloud launched Gemini Enterprise for Legal, a specialized AI platform with agents for drafting and contract work, joining Anthropic and OpenAI in courting law firms.
Porsche signed a five-year, €1.25 billion AI partnership with Tata Consultancy Services, which is separately buying Porsche’s IT consulting arm MHP for €320 million.
OpenAI banned ChatGPT accounts it says originated in Russia and built a fake “expert institute” using plagiarized academic work to manufacture false credibility online.
Apple launched new Mac Mini and Mac Studio models with its M6 and M5 Ultra chips, explicitly built for developers running AI models and agents locally instead of in the cloud.
Nvidia is in talks to invest in Perplexity at a valuation above $30 billion, a 50% jump from a year ago, as the AI search startup’s revenue triples on its new agent product.
Alibaba launched Wan3.0, an AI video model that turns business documents into 30-second clips, a day after raising $10.2 billion in Hong Kong’s largest-ever follow-on share sale.
Hugging Face is reportedly exploring a sale valuing it at $13 billion or more, nearly triple its 2023 valuation, though no deal has been reached and no buyer identified.