A KV cache, or key-value cache, speeds up autoregressive transformer inference by storing attention information calculated for tokens that have already been processed. Without the cache, the model would recompute those keys and values each time it generates another token. Reusing them reduces repeated work and improves response latency, especially for long prompts and outputs. The tradeoff is memory: cache size grows with sequence length, batch size, layer count, and model dimensions. Production systems manage this cost with paging, quantization, eviction, prompt caching, and request scheduling. KV caching changes inference efficiency, not the model’s learned knowledge, and cached data must be protected when prompts contain sensitive information.
Amazon is packaging the technology behind Alexa for Shopping into a new service for retailers, aiming to become the infrastructure layer for AI-powered commerce across the web.
A new study from researchers at Massachusetts Institute of Technology and University of Southern California finds generative AI is rapidly increasing the number of Americans filing lawsuits without lawyers in U.S. federal courts.
Robinhood is rolling out AI agent trading accounts and virtual payment cards, allowing AI systems to analyze portfolios, execute trades, and make purchases on users’ behalf.
BNP Paribas says it is strengthening cybersecurity defenses as increasingly powerful AI systems accelerate the discovery of software vulnerabilities across the financial sector.
Micron crossed a $1 trillion market valuation for the first time as booming demand for AI memory chips pushed shares to record highs amid the expanding AI infrastructure race.
Google DeepMind says its new AlphaProof Nexus framework solved nine open Erdős problems using AI-guided formal verification, though CEO Demis Hassabis cautioned the system is still far from AGI.
OpenAI CEO Sam Altman said AI has not caused the large-scale white-collar job losses he once expected, adding that human interaction still plays a critical role in many professions.
Anthropic co-founder Chris Olah told the Vatican that AI development cannot be left solely to technology companies, warning about commercial incentives, labor disruption, and the growing complexity of frontier AI systems.
Ashton Kutcher’s venture firm Sound Ventures invested $30 million in OpenAI years ago. That stake could now be worth more than $2 billion if OpenAI goes public at a higher valuation.
Anthropic says its unreleased Mythos AI model has identified more than 10,000 high- and critical-severity software vulnerabilities as part of Project Glasswing, a cybersecurity initiative focused on protecting critical infrastructure and open-source software.