Reviewing AI-generated code requires meaningful tests, actionable findings and a clear owner for the released change. Photo: Fotis Fotopoulos / Unsplash
How to Review AI-Generated Code: Tests, Evidence and Human Oversight
By Daniel Mercer• 10 mins read
Published:
, Updated:
Learn how to review AI-generated code using clear requirements, meaningful tests, fresh-context reviews, bounded fix cycles and a practical rollback plan.
AI-written code can look finished long before anyone has established that it works. A model may explain its changes fluently, produce a plausible test and confidently declare success. The useful question is whether the evidence supports the behavior you intended, including the cases the implementation might have overlooked.
Reviewing AI-generated code therefore needs more than a second request to “check everything.” It needs a clear specification, appropriate tests, an independent examination of the change and a person responsible for what ships. This guide follows an illustrative retry-button fix through that process, from the first acceptance criteria to a release that can be stopped or reversed.
Define the Change Before Reviewing the Code
A review becomes much easier when the expected behavior is precise. Suppose an application lets a user retry a failed upload. The requested fix is to prevent repeated clicks from starting multiple uploads while preserving a useful error message if the retry fails. That is a behavior to verify, rather than a vague request to improve the upload component.
Record what should remain unchanged. The retry should use the existing authorization path, preserve the selected file and avoid modifying unrelated upload settings. If the product needs a new retry policy or a different server contract, that is a separate decision to make deliberately.
Google’s review guidance emphasizes design, functionality, complexity and useful tests. It also asks reviewers to consider edge cases and whether a change serves its users. Those questions remain relevant when an agent, rather than a person, produced the first version.
For our retry-button example, write down the important states: ready to retry, request in progress, retry succeeded and retry failed. Then specify what the user should see in each state. This gives the implementer and reviewer a shared target without dictating every internal line of code.
Ask for Observable Evidence of Completion
Anthropic’s Claude Code guidance recommends giving the agent a check it can run, such as tests, a build or a visual comparison. The benefit is a feedback signal beyond whether the code appears complete. The check must still represent the requirement you care about.
Ask for the command that ran, its result and the scope it covered. “Tests passed” is incomplete if it refers only to a small helper while the changed screen was never exercised. A build can show that the application compiles without demonstrating that duplicate uploads are prevented.
For the retry fix, useful evidence might include a focused test showing that repeated clicks during one pending request create only one client request, plus a browser check of the disabled button and visible status. A failure case should demonstrate that the button becomes usable again when retry is appropriate.
If the agent could not run a check, preserve that limitation. A missing dependency, unavailable test service or unsupported platform should be reported as unverified coverage. Do not turn a description of what a test would do into a statement that it passed.
Choose Tests That Can Expose the Failure
A test should distinguish the broken behavior from the intended behavior. In our example, rendering the retry button is insufficient: the original bug concerns multiple actions while work is pending. A test that only checks the button’s label could pass before and after the fix without detecting the defect.
Where practical, establish that the regression case fails against the old implementation and passes against the corrected one. That does not prove the whole application is sound, but it shows the test is sensitive to the failure it claims to prevent. Keep the test focused enough that an unrelated error does not masquerade as that result.
For browser tests, Playwright’s testing guidance recommends checking user-visible behavior rather than relying on internal implementation details. It also favors isolated tests. Applied here, the test should exercise the rendered retry flow and controlled responses rather than merely inspect a variable named “uploading.”
Consider the boundary between client behavior and server behavior. Preventing two rapid clicks does not necessarily prevent a second request after a refresh, another browser tab or a network retry. If the requirement is that the underlying upload operation occurs once, the review must examine the server’s handling too. Do not claim a local button fix establishes a stronger guarantee.
Match the remaining checks to the change. A small copy edit rarely needs an elaborate new test suite, while an upload-state change warrants checks around state transitions and failure recovery. More tests are useful only when they increase meaningful coverage without creating unnecessary maintenance.
Give a Fresh Reviewer the Right Context
An independent review should start from the requested behavior and the actual diff, not just the implementer’s explanation. Anthropic’s best-practices documentation describes a writer-and-reviewer pattern using fresh context. The purpose is to examine the result without carrying forward every assumption from the implementation conversation.
Fresh context does not mean no context. Provide the requirements, relevant project instructions, the commit being reviewed and the evidence already collected. The reviewer also needs access to surrounding code when behavior depends on it. An isolated function can look correct while using an API in a way the rest of the application does not support.
A suitable request for our example would be: “Review the retry-button diff against the upload-state requirements. Look for duplicate requests, incorrect recovery after failure and changes to authorization behavior. Give the file location, triggering condition and user impact for each finding. Report uncertainty separately from confirmed defects.”
Do not instruct the reviewer to find a minimum number of bugs. A clean review should remain possible. Requiring several findings creates an incentive to present preferences or speculative problems as defects, which is particularly costly if another agent automatically changes code in response.
An agent reviewing an agent is still an automated check. Separate contexts can produce useful disagreements, but they can also share blind spots or miss a requirement that was never supplied. Human ownership and appropriately chosen deterministic checks remain part of the process.
Require Findings That a Developer Can Act On
A useful finding explains what is wrong, when it happens and why it matters. “The retry logic may be unsafe” leaves too much work for the reader. A concrete finding might show that a rejected request never clears the pending state, leaving the retry button permanently unavailable.
Ask the reviewer to distinguish introduced defects from existing problems. The retry fix should not become an unlimited mandate to redesign the upload subsystem. An important pre-existing issue can be recorded for separate work without confusing who introduced it or whether it blocks this change.
Verify each finding before applying a fix. Read the cited path, reproduce the condition when feasible and compare it with the intended behavior. Sometimes a reviewer has missed a guard elsewhere in the code; sometimes the implementation exposes a real gap in the specification.
The public Code Review documentation describes an automated service focused on identifying bugs in pull requests. Such a service is one possible review layer, rather than a replacement for defining your own acceptance criteria. Check its current setup and access requirements before adopting it.
Stop Review Loops From Becoming the Project
Anthropic’s engineering account describes one pull request that went through more than fifty review-and-fix rounds overnight. The team responded by limiting rounds, narrowing later reviews and avoiding repeated findings. The lesson is not that another pass is always valuable; review needs to converge on a decision.
For our example workflow, set a small initial limit, such as two fix-and-review cycles, then ask a person to decide whether further work is justified. This is a proposed starting policy, not a universal number. A sensitive change may deserve additional scrutiny, while a small fix should not accumulate unrelated cleanup indefinitely.
After a correction, inspect the new diff and rerun the checks affected by it. Reopening an already resolved naming preference should not restart the entire process. A genuinely new defect should be investigated even if the workflow is near its limit, but it may require changing the approach rather than requesting another identical review.
Track findings as fixed, disproved, accepted with an explanation or deferred to separate work. That record prevents the reviewer and implementer from repeatedly arguing about an issue that already has a decision. If the same defect appears in several places, address the shared cause or explicitly document the remaining scope.
Turn Repeated Feedback Into Enforceable Project Knowledge
Anthropic’s account also describes moving recurring review lessons into project instructions, review guidance or automated rules. The objective is to prevent the same mistake earlier, rather than paying for another explanation on every pull request. Keep those lessons specific to failures that matter in the repository.
For the upload application, a useful rule could say that every pending state needs an explicit recovery path for rejected requests. Include the relevant module and a representative test. A vague instruction to “always write perfect asynchronous code” cannot guide an implementation or a review.
Use automation for requirements that can be checked reliably. Formatting belongs in the formatter; a supported type rule belongs in the type checker. Reserve written guidance for context that requires judgment, such as which state transitions the product promises to its users.
Keep instructions short enough to maintain, and review them when the architecture changes. A rule copied from an old component can become misleading if the application adopts a different request model. Treat project guidance as maintained engineering material, not a growing archive of every comment an agent has produced.
Tie Approval to the Code That Will Ship
Approval should refer to an identifiable version. If a new commit changes the retry behavior after review, the earlier evidence may no longer cover the final result. GitHub’s branch protections can require reviews and status checks, with options for dismissing stale approvals or requiring approval of the most recent reviewable push.
These are repository controls, not instructions that the model can choose to remember. Configure them to match the team’s policy, including any bypass permissions. A green check is only meaningful if it comes from the expected workflow and evaluates the relevant commit.
Scale human review to the consequences of a mistake. A change to upload authorization or persistent data deserves different scrutiny from a button-label correction. Anthropic’s account keeps people responsible for architecture, risky changes, interpreting evidence and the outcome of shipped work. Its internal process is useful context, not automatic permission for another team to remove its own review requirements.
The author or owner should be able to explain what the checks do not establish. For the retry fix, a client-side test may not cover server concurrency, mobile layouts or an interrupted browser session. Those limits should inform the release decision rather than disappear into a general claim that everything passed.
Prepare Recovery Before Release
Git’s revert command records new commits that reverse changes introduced by earlier commits. That is useful for source-code recovery, but reverting code does not automatically undo an external message, restore deleted information or reverse a database migration.
For our retry-button example, identify the release containing the fix, the signal that would indicate a regression and the person who can stop the rollout. If a feature flag controls the new behavior, confirm that turning it off actually restores the intended fallback. Keep the change small enough that recovery does not also remove unrelated work.
After deployment, check the affected workflow and monitor relevant failures. A passing pre-release test cannot establish how every production condition will behave. Feed confirmed regressions into future test cases and review criteria, while preserving a clear distinction between evidence and speculation.
The wider agent evaluation process applies here too: measure useful results and the human effort needed to trust them. A good AI code review helps a team make a defensible shipping decision. It provides concrete findings, meaningful checks and a route back when something still goes wrong.
