A KV cache, or key-value cache, speeds up autoregressive transformer inference by storing attention information calculated for tokens that have already been processed. Without the cache, the model would recompute those keys and values each time it generates another token. Reusing them reduces repeated work and improves response latency, especially for long prompts and outputs. The tradeoff is memory: cache size grows with sequence length, batch size, layer count, and model dimensions. Production systems manage this cost with paging, quantization, eviction, prompt caching, and request scheduling. KV caching changes inference efficiency, not the model’s learned knowledge, and cached data must be protected when prompts contain sensitive information.
Anthropic has made Claude Cowork generally available across paid plans, adding enterprise controls and analytics to support company-wide AI deployment.
Anthropic has signed a multi-year deal with CoreWeave to power its Claude AI models, expanding compute capacity as demand for AI infrastructure surges.
OpenAI has introduced a $100 ChatGPT Pro tier offering 5x higher Codex usage limits, targeting developers amid rising competition in AI coding tools.
OpenAI to include retail investors in IPO as demand surges, highlighting shift toward broader ownership and funding for massive AI infrastructure plans.
OpenAI plans to restrict access to a powerful new cybersecurity-focused AI model, reflecting growing concern over misuse as capabilities approach real-world attack potential.
OpenAI has paused its Stargate AI infrastructure project in the UK, citing high energy costs and regulatory uncertainty. The move raises questions about the country’s AI ambitions.
Anthropic has launched Managed Agents, a new system designed to run long-horizon AI tasks by separating reasoning, execution, and memory layers. The approach aims to improve reliability and scalability.
Meta shares climbed 7% after unveiling Muse Spark, its new multimodal AI model, signaling investor confidence in the company’s revamped AI strategy.
A U.S. appeals court has denied Anthropic’s request to pause a Pentagon blacklist, allowing restrictions on its AI use in defense contracts to remain during litigation.
Meta has unveiled Muse Spark, a multimodal AI model with advanced reasoning and multi-agent capabilities, marking a step toward its vision of personal superintelligence.