A KV cache, or key-value cache, speeds up autoregressive transformer inference by storing attention information calculated for tokens that have already been processed. Without the cache, the model would recompute those keys and values each time it generates another token. Reusing them reduces repeated work and improves response latency, especially for long prompts and outputs. The tradeoff is memory: cache size grows with sequence length, batch size, layer count, and model dimensions. Production systems manage this cost with paging, quantization, eviction, prompt caching, and request scheduling. KV caching changes inference efficiency, not the model’s learned knowledge, and cached data must be protected when prompts contain sensitive information.
Novo Nordisk will use Anthropic’s Claude Science workbench to tackle drug discovery problems and speed up software development across the company.
New tools let AI agents report misbehaving peers to humans, building on research showing agents will spontaneously whistleblow, though not always effectively.
Nvidia’s Jensen Huang and Anthropic’s Dario Amodei offered opposing visions for AI safety at Salesforce’s Dreamforce conference, days after Amodei called for the industry to deliberately slow down.
Google introduced Gemini 3.8 Live and a reasoning-focused Extended Thinking variant, aimed at production-grade voice agents that can see, switch languages and run tools in the background without breaking conversation.
OpenAI’s policy chief says the company has spent weeks working with Anthropic and Google DeepMind on AI safety, as Washington debates how to respond to rising concern.
A former Google DeepMind researcher resigned and warned that AI has the potential to kill everyone, joining a growing wave of safety researchers speaking out.
Elon Musk expanded on his endorsement of Anthropic CEO Dario Amodei’s AI safety warning, saying danger is significant now and competitors should test each other’s models.
OpenAI pays hundreds of contractors to read real ChatGPT conversations and rate the chatbot’s replies, a practice most users likely don’t know exists.
Microsoft published draft rules for its AI models, days after Anthropic’s CEO called for slowing AI development and a researcher resigned over safety fears.
Sam Altman says safety work should constrain the pace of frontier AI development, following an interview in which he rejected a hypothetical 10% extinction risk as unacceptable.