A KV cache, or key-value cache, speeds up autoregressive transformer inference by storing attention information calculated for tokens that have already been processed. Without the cache, the model would recompute those keys and values each time it generates another token. Reusing them reduces repeated work and improves response latency, especially for long prompts and outputs. The tradeoff is memory: cache size grows with sequence length, batch size, layer count, and model dimensions. Production systems manage this cost with paging, quantization, eviction, prompt caching, and request scheduling. KV caching changes inference efficiency, not the model’s learned knowledge, and cached data must be protected when prompts contain sensitive information.
Google is testing a new Search feature that lets users move from AI Overviews into conversational AI Mode on the same results page. The rollout comes as OpenAI declares a “code red” and refocuses on ChatGPT.
AWS AI Factories bring dedicated AI infrastructure directly into customer data centers, enabling enterprises and governments to rapidly develop and deploy AI applications while meeting data sovereignty and regulatory requirements.
penAI has put advertising plans for ChatGPT on hold after declaring a “code red,” shifting its focus toward improving the AI assistant’s performance, stability, and security amid intensifying competition in the AI market.
DeepSeek has launched two new open-source AI models — V3.2 and V3.2-Speciale — that the company says match or exceed leading U.S. systems in advanced reasoning and tool-use tasks.
Amazon’s AI shopping assistant Rufus saw a sharp spike in Black Friday usage, fueling significantly higher purchase-conversion rates compared with non-AI sessions, according to new data from Sensor Tower and Adobe Analytics.
Baidu is scaling up its in-house AI-chip business via its Kunlunxin unit, unveiling a five-year roadmap with M100 and M300 chips and aiming to fill the gap left by restricted Nvidia GPU access in China.
For the first time, Washington is close to deciding how artificial intelligence should be regulated — but the fiercest battle isn’t over safety standards. It’s over whether states should retain the authority to pass their own AI laws.
Amazon employees demand ethical AI practices, warning that the company’s fast-paced AI rollout risks worker safety, democratic oversight, and environmental sustainability.
Microsoft has launched a major AI training initiative in Poland, aiming to provide one million people with AI competencies by the end of 2025. Free courses are available in Polish via the AI Skills Navigator hub.
OpenAI co-founder Ilya Sutskever says the era of scaling compute and data as the main driver of AI progress is ending, arguing that meaningful advances now depend on true scientific breakthroughs and solving AI’s poor generalization.