A KV cache, or key-value cache, speeds up autoregressive transformer inference by storing attention information calculated for tokens that have already been processed. Without the cache, the model would recompute those keys and values each time it generates another token. Reusing them reduces repeated work and improves response latency, especially for long prompts and outputs. The tradeoff is memory: cache size grows with sequence length, batch size, layer count, and model dimensions. Production systems manage this cost with paging, quantization, eviction, prompt caching, and request scheduling. KV caching changes inference efficiency, not the model’s learned knowledge, and cached data must be protected when prompts contain sensitive information.
OpenAI pays hundreds of contractors to read real ChatGPT conversations and rate the chatbot’s replies, a practice most users likely don’t know exists.
Microsoft published draft rules for its AI models, days after Anthropic’s CEO called for slowing AI development and a researcher resigned over safety fears.
Sam Altman says safety work should constrain the pace of frontier AI development, following an interview in which he rejected a hypothetical 10% extinction risk as unacceptable.
Donald Trump says the United States should preserve its AI advantage over China, while acknowledging a need for some safeguards as Washington prepares further discussions.
Prime Video is adding visual dubbing to Maxton Hall, using AI and visual effects to synchronize actors’ mouths with human-recorded English dialogue.
Xi Jinping proposed joint language-model development and specialist training across BRICS, but announced no named models, access terms or rollout timetable.
Anthropic’s economic scenarios explore how AI could raise U.S. output by 2030 while shifting income toward capital and putting pressure on knowledge workers.
Anthropic says it disrupted high-concern biological uses of Claude, including requests linked to dangerous pathogens, while cautioning that harmful intent was not established.
Sam Altman has discussed slowing advanced AI development, Bloomberg reports, as OpenAI’s chief scientist argues for coordination and stronger safeguards.
OpenAI is packaging financial data, research and document creation into a specialized ChatGPT product aimed at investment bankers and equity researchers.