A KV cache, or key-value cache, speeds up autoregressive transformer inference by storing attention information calculated for tokens that have already been processed. Without the cache, the model would recompute those keys and values each time it generates another token. Reusing them reduces repeated work and improves response latency, especially for long prompts and outputs. The tradeoff is memory: cache size grows with sequence length, batch size, layer count, and model dimensions. Production systems manage this cost with paging, quantization, eviction, prompt caching, and request scheduling. KV caching changes inference efficiency, not the model’s learned knowledge, and cached data must be protected when prompts contain sensitive information.
Epic is shifting developers toward AI-generated code and art as its Disney shooter reportedly slips, while Tim Sweeney promises more ambitious creative output.
Z.ai’s GLM-Legal targets contract review, evidence analysis and cross-border compliance, with beta access by application and independently evaluated benchmarks.
Meta’s Muse Wonder models pair Active Memory with Active Focus for AI agents, coding and professional work, with API access and subscription-only Full.
Navitas and Nvidia have developed a 24kW board that converts 800V to 12V in one stage, targeting more efficient power delivery for AI data centers.
Microsoft has put Cisco Silicon One P200-powered systems into production, adding deep-buffer routing capacity for its growing cloud infrastructure.
Nvidia and Cisco outline USFP optics for denser, more efficient AI networks, alongside new Cisco systems for scaling and managing data centers.
Hackers disrupted French supermarket refrigeration, with Intermarché reporting temperatures up to 50°C for six hours as stores discarded food.
Zuckerberg says Meta has reached the AI frontier with Muse Wonder and rejects a slowdown, putting performance and safety claims under fresh scrutiny.
A user claimed to cancel Musk’s LEGO order after his Grok shopping demo. Musk denied it, turning the exchange into a debate over purchase privacy.
Microsoft plans to make Maia its primary AI inference system, with Maia 300 arriving in 2027 and a larger role for its homegrown chips.