Rethinking AI Code Review: Let Machines Verify, Keep Humans in Charge
Most teams have it backwards, asking AI to judge code quality and leaving humans to rubber-stamp the results. A better approach flips the workflow: automated agents verify against standards, while engineers make the actual decisions.
The conventional approach to AI-assisted code review has the priorities inverted. Teams typically feed a code diff to a language model, ask for a verdict on its quality, and then have a person glance at the output before approving. This setup misuses both the strengths of machines and the capabilities of humans.
An AI model can examine every line without fatigue and apply consistent standards across every pull request. What it cannot do is understand your team's specific engineering practices. It has no memory of previous architectural decisions, cannot evaluate whether a feature should exist in the first place, and bears no responsibility for its recommendations. As one observer put it, "Judgment needs intent, and accountability needs a person."
A better model inverts the roles: automated systems handle verification while humans retain decision-making authority. Aviator, a code review platform, has built this approach into a product called Aviator Verify. The system works through seven interconnected stages that separate verification from judgment.
The anatomy of an intent-driven review
Every code change should be evaluated against two distinct frameworks. Acceptance criteria define what a particular change is supposed to accomplish. Invariants represent the standing rules of an engineering organization or team—the principles that remain constant regardless of the specific modification.
Acceptance criteria without invariants force every engineer to rely on memory for team standards, or require reviewers to catch obvious mistakes that an agent makes. Invariants without acceptance criteria produce code that follows the rules but may solve the wrong problem, leading agents to over-engineer or miss the actual goal.
The seven-part framework builds both layers, validates changes against them, and ultimately places the decision in human hands:
- Invariants (defined before any change): standing rules of the team, extracted from its own code review history.
- Intent and acceptance criteria (captured during coding): the reason for the change and the expected behavior.
- Core decisions (captured during coding): the choices made throughout development, and who made them.
- A test plan (built at submission): scenarios derived from both criteria and applicable invariants.
- Evidence (gathered at submission): concrete proof from a running environment supporting each verdict.
- A review (decided by a person): assessment of intent, decisions, and evidence, with approval or feedback. Not code.
- Invariants refreshed (updated after merge): waivers and recurring patterns that refine the invariants.
1. Invariants
Most teams encode their standards in the minds of senior engineers, who express them as review comments. Use the Money type for currency values. Never write directly to the users table. Every error path must emit a counter. Each rule gets typed repeatedly, and AI agents keep repeating the same mistakes because the rule was never formalized anywhere a machine could check it.
An invariant is that rule, written once and applied to every matching change. Collectively, invariants form a registry of patterns the team repeatedly corrects—a catalog of what not to do.
Instruction files like CLAUDE.md serve as inputs to code generation, but the same non-deterministic model that wrote the code is asked to follow them, with no guarantee it did. Invariants belong on the verification side, checked by a deterministic script or a separate agent that returns a verdict for each rule.
A team's PR history already contains most of its invariants. Aviator Verify can read the comments posted in GitHub and draft invariants from the patterns it discovers. It can also extract them from CONTRIBUTING.md files, agent instruction files, or from real-time input when someone tags @aviator invariant in a PR comment or Slack message. Teams can generate their invariants using the Aviator Verify free account and retain them regardless of whether they continue with the full product.
Three factors determine whether an invariant proves useful or becomes noise:
- Ownership: Every draft goes to the appropriate team with the reasoning behind it, and becomes active only after that team approves. AI-drafted rules do not become standing checks without explicit human approval.
- Wording: State the desired outcome while leaving implementation details open, so the rule survives code refactoring and file reorganization.
- Scope: A rule should only run in contexts where it can actually fail. A frontend invariant should not evaluate database migrations.
2. Intent and acceptance criteria
The straightforward approach to capturing intent involves writing a specification beforehand or creating a detailed ticket. Specifications demand time, are written before implementation reveals what actually matters, and diverge from reality as soon as coding begins.
The most accurate intent already exists in the conversation between engineer and agent. Every time the agent asks a question and the engineer makes a decision, acceptance criteria are being defined in real time. There is no reason to discard all that information when the session ends.
Intent explains why the change exists. Acceptance criteria specify how it should function.
Verify extracts both from the session the engineer already conducted. An agent skill and a hook run before the agent opens a pull request. The agent pulls the intent and criteria from everything it observed during the session, and the engineer reviews and adjusts them before submission.
3. Core decisions
A decision is any point where the implementation could have taken a different path. Where to store state, how to handle a dependency failure, whether to extend an existing helper or write a new one. AI-generated pull requests contain numerous such decisions, and almost none appear in the diff itself.
An AI-written PR is full of calls like these, and almost none of them show up in the diff.
Verify records these decisions alongside the intent, labeling whether humans or agents made each call. This allows reviewers to understand who decided what and to challenge specific choices.
4. A test plan
Upon submission, Verify examines the intent and criteria and uses the diff to determine which invariants apply. The two layers combine into a single list of scenarios: steps an agent takes to validate behavior, derived from requirements rather than code. The engineer stays focused on the change itself, while the registry ensures standards are met.
5. Evidence
When the same model both writes and reviews code, they share identical blind spots. Verification must be structurally independent from generation, and it must deliver more than an opinion.
When the same model writes and reviews the code, they share the same blind spots.
Verify launches a preview environment and deploys verifier agents to execute the scenarios. They gather screenshots, database snapshots, server logs, and console output. Every verdict—for a criterion or an invariant—points to supporting evidence. For user interface changes, the verifier navigates the interface and confirms the results.
A preview environment is not necessary from day one. Code scans can handle structural checks directly from the diff without additional infrastructure, and many invariants can be converted to deterministic scripts. A preview becomes worthwhile once criteria begin asking about behavior that the diff alone cannot demonstrate.
6. A review
This is where human judgment returns, and the review process looks fundamentally different. Instead of reading code line by line, a reviewer examines evidence. They read the intent, focus on the critical areas where changes occur, evaluate the decisions and the verdict for each criterion and invariant, and open the live preview if they want to see the change in action.
The pull request workflow remains largely unchanged. Verify adds a link to the review, the diff stays in its usual location, and results appear as a status check similar to CI output.
The reviewer answers a single question: does the evidence align with what the team committed to building? They can approve, challenge a decision, request another scenario, or waive a failed invariant with justification. This is judgment, and someone is accountable for it.
7. Invariants refreshed
Most reviews conclude at merge, and the insights they generate disappear. The next pull request repeats the same mistake, and the same comment appears again.
Invariants preserve what each review teaches. When an invariant fails on a PR, the author can waive it with a reason: perhaps it does not apply, the invariant is flawed, or the issue gets resolved separately. The team owning the invariants sees every waiver.
Insights reveal which invariants are most frequently applied, how often each fails, and the reasons people waive them. An invariant that gets waived repeatedly is broken and needs rewriting or narrowing. A comment reviewers repeatedly type becomes the next draft. Over time, invariants evolve into a continuously maintained set of rules that protect every code change.
Where to start
Do not activate everything simultaneously. Begin with invariants, since they are the least resource-intensive and your PR history already contains them. The first month will feel like extra work, with the team still reviewing code while the registry accumulates. In month two, invariants catch enough issues that reviews become shorter. By month three, reviewers focus on intent, decisions, and evidence, opening the diff only when something seems off.
The code reviews your team already conducts are packed with rules waiting to be formalized. Aviator Verify converts them into invariants, validates every change against them, and leaves the decisions to people. Teams can generate their invariants using the Aviator Verify free account and keep them even if they choose not to continue with the full service.