A KV cache, or key-value cache, speeds up autoregressive transformer inference by storing attention information calculated for tokens that have already been processed. Without the cache, the model would recompute those keys and values each time it generates another token. Reusing them reduces repeated work and improves response latency, especially for long prompts and outputs. The tradeoff is memory: cache size grows with sequence length, batch size, layer count, and model dimensions. Production systems manage this cost with paging, quantization, eviction, prompt caching, and request scheduling. KV caching changes inference efficiency, not the model’s learned knowledge, and cached data must be protected when prompts contain sensitive information.
ChatGPT is expanding group chats to Free, Go, Plus, and Pro users worldwide, allowing collaborative conversations with friends, family, or coworkers alongside AI assistance.
Voice AI startup Wispr has secured another $25M in funding as its Flow dictation app surges in enterprise adoption and user retention, bringing total capital raised to $81M.
Kraków, long a European hotspot for outsourcing, is facing large-scale job cuts as AI automates traditional back-office operations. The shift is emptying offices that once buzzed with multinational firms.
Nvidia reported strong third quarter results driven by soaring demand for its data center GPUs and Blackwell architecture. The company raised its outlook as AI infrastructure spending accelerates.
Target is deepening its partnership with OpenAI through a new ChatGPT shopping app and broader enterprise use of frontier models to streamline operations and improve guest experiences.
Google has released Gemini 3, its most powerful AI model yet, offering advanced multimodal reasoning, coding capabilities, and agentic planning tools across multiple platforms.
Stack Overflow unveiled new enterprise AI products at Microsoft Ignite, designed to convert internal expertise into AI-ready formats and streamline knowledge access for organizations.
Anthropic is committing $30 billion in Azure compute as part of a major new partnership with Microsoft and Nvidia, expanding Claude and deepening collaboration on AI infrastructure.
xAI has launched Grok 4.1, a major model upgrade designed to deliver more creative, emotionally aware, and coherent interactions, with new Thinking and Non-Thinking variants now live across all xAI platforms.
OpenAI is exploring how ads might fit into ChatGPT, but CEO of Applications Fidji Simo says any ad model must be radically different from traditional approaches.