Automation bias occurs when people give excessive weight to a computer-generated recommendation and reduce their own scrutiny. It can produce errors of commission, where a user follows incorrect advice, or omission, where a person fails to act because the system did not issue an alert. The risk grows when an AI tool is usually accurate, presented as authoritative, difficult to question, or used under time pressure. Simply placing a human in the loop does not solve the problem. Interfaces should communicate uncertainty, show relevant evidence, support easy disagreement, avoid manipulative defaults, and measure whether reviewers meaningfully detect errors rather than merely approve automated outputs.
Anthropic unveils a revised Responsible Scaling Policy with a Frontier Safety Roadmap, regular Risk Reports, and clearer separation between company commitments and industry recommendations.