Prompt injection is an attack in which untrusted text or content attempts to override an AI application’s instructions. The malicious instruction may come directly from a user or indirectly from a webpage, document, email, or tool result processed by the model. If successful, it can cause data disclosure, unauthorized tool use, altered output, or bypassed policies. Because language models do not inherently distinguish trusted commands from quoted content, filtering suspicious phrases alone is insufficient. Defenses include isolating data from instructions, minimizing permissions, validating tool arguments, requiring approval for sensitive actions, restricting secrets, and testing with adversarial content. Applications should assume that any external content may contain instructions designed to manipulate the model.
Microsoft CEO Satya Nadella wants humans to be able to stop advanced AI mid-task, with independent controls, audit trails and limits on agent access.
Anthropic details unintended Claude agent actions on government websites. Philadelphia police says a fabricated homicide tip was blocked as spam.
AI companies are rehearsing potential crisis responses, Axios reports. OpenAI says the scenarios are not inevitable, while recent incidents test safeguards.
Anthropic’s OSS Scanner uses models including Claude Mythos to check eligible open-source projects. Maintainers receive findings without human review.
Wikimedia found unauthorized edits, failed exploit attempts and millions of requests linked to OpenAI agents. It reported no breach, but a possible outage impact.
Mistral launches a Large 4 API preview with a trillion-parameter architecture, discounted pricing and enterprise-focused benchmarks. Open weights are due later this month.
South Korea is investigating an Artex AI link to bank breaches affecting about 68,000 people, as security tools face scrutiny over possible misuse.
Google has paused new OSS product-vulnerability reports after a surge in invalid automated submissions. Supply-chain reports remain open, with an update due in Q1 2027.
A Florida woman faces a written-threat charge after Anthropic’s safety system flagged Claude messages for human review and disclosure to police, investigators say.
Can Gemini 4 Argon really match OpenAI’s models and Claude—or outperform them? Benchmarks show gains, but wider enterprise use will test those results.