A KV cache, or key-value cache, speeds up autoregressive transformer inference by storing attention information calculated for tokens that have already been processed. Without the cache, the model would recompute those keys and values each time it generates another token. Reusing them reduces repeated work and improves response latency, especially for long prompts and outputs. The tradeoff is memory: cache size grows with sequence length, batch size, layer count, and model dimensions. Production systems manage this cost with paging, quantization, eviction, prompt caching, and request scheduling. KV caching changes inference efficiency, not the model’s learned knowledge, and cached data must be protected when prompts contain sensitive information.
Former Oracle employees say they were asked to help train internal AI systems before being laid off, as the company shifts resources toward data centers and AI infrastructure.
A growing trend in China allows users to create AI replicas of former partners using personal data. The practice is raising concerns about privacy, emotional dependency, and relationships.
Anthropic is considering a major funding round amid strong investor demand and rapid revenue growth. The potential raise could value the company at up to $900 billion.
Microsoft says its revised OpenAI partnership strengthens flexibility while maintaining key advantages. The company reported AI revenue surpassing $37 billion amid growing multi-model demand.
SoftBank is preparing a new AI and robotics company to automate data center construction, with plans for a potential $100 billion IPO. The move targets growing demand for AI infrastructure.
OpenAI outlined how ChatGPT detects and responds to potential threats of violence, including escalation to human reviewers and law enforcement. The update follows growing scrutiny of AI safety practices.
Families of victims in a Canadian school shooting have sued OpenAI, alleging it failed to alert authorities about warning signs in ChatGPT conversations. The case raises questions about AI oversight and duty of care.
OpenAI is deepening ties with Amazon while restructuring its long-standing partnership with Microsoft. The shift reflects growing demand for flexible cloud access and AI infrastructure.
AI stocks fell sharply after reports that OpenAI missed growth targets and raised internal financial concerns. The news triggered a ripple effect across cloud and chip companies.
A high-profile trial between Elon Musk and OpenAI leaders has begun, centering on claims the company abandoned its nonprofit mission. The case could reshape governance in leading AI firms.