Code review

How automated code review works in GitHub

Capy team

Automated code review is software, often built on coding agents, that reads a pull request on GitHub and posts what it finds as comments on the changed lines. It runs when the pull request opens or gets new commits, alongside your CI, so every diff has been read once before a person opens it. It doesn't approve the change. The people with write access still decide what merges.

Where automated review sits in a pull request

A pull request on GitHub already collects feedback in two places. Status checks report whether builds, tests and scans passed, and a branch protection rule can require them before a merge. Reviews carry comments on specific lines, plus a verdict: comment, approve or request changes.

An automated reviewer posts reviews. CI answers "does it build, and do the tests we wrote pass?" The reviewer answers a different question: "is there a bug or a risk the tests don't cover?"

What a review run does

Most automated reviewers follow the same steps on each run:

  1. Start on an event. The pull request opens, a commit arrives, or someone asks for a review in a comment.
  2. Check out the code at the pull request's head. The diff alone is rarely enough. A changed function has callers, and a changed query has a schema it runs against.
  3. Read the change in context. The reviewer traces what the changed lines touch and compares them with the conventions the repository writes down.
  4. Write findings. Each one names a file and line, what goes wrong, and how serious it is.
  5. Post what clears a threshold. Serious findings become inline comments. Minor ones can stay off the pull request, so the comments that do post are worth reading.
  6. Review again on the next push. A good re-review reads only what changed since the last round and closes the findings the new commits fixed.

What it catches and what it leaves to people

An automated reviewer finds problems that show in the code: a missed edge case, a condition that reads the wrong field, a query built from user input, a convention the team wrote down and someone broke. It gives the 2,000-line pull request the same attention as the 20-line one, which people rarely do.

It can't tell you whether the change is the right thing to build, and it wasn't in the meeting where your team agreed on it. It doesn't replace your tests or CI either. Treat it as one more reviewer whose comments someone reads, not as the approval.

A worked example: one pull request, two rounds

The image above shows an example pull request in Capy. The change disables the merge button while CI is red.

  1. The pull request opens and the first round posts its findings. The high finding says the merge gate reads only one kind of check result, so a failing check of another kind wouldn't block the merge. Medium and low findings and a note come with it.
  2. The Capy thread that opened the pull request reads the findings. It fixes the high one, marks one it can show is a false positive as irrelevant, and pushes.
  3. The push starts a second round. It reviews only the new commits, with the whole pull request as context, and its summary says the fix holds and one earlier risk remains.
  4. A person reads the open findings, accepts the remaining risk or asks for a fix, and merges.

The reviewer approved nothing. It put the risks in front of the person who merges.

Where Capy fits

Capy is a platform that orchestrates agent swarms, and its review agent is the reviewer in the example:

And where it doesn't:

Follow-up questions

Does an automated review count as an approval?

No. Your branch protection decides what a merge needs, and a person makes the merge. The review hands that person a list of risks to decide on.

How do I stop the same false positive from coming back?

In Capy, mark the finding irrelevant, and re-reviews never raise it again. For a pattern that's always fine in your codebase, write it in AGENTS.md, and the next review follows it.

Should it review every push or only once?

Every push keeps the findings current on a pull request that changes a lot. Once costs less, because each round bills credits. You set the mode per repository.

Can it review pull requests that people opened?

Yes. Automatic review covers ready pull requests in every repository you turn it on for, whoever opened them. Drafts wait until they're marked ready. Any member of your Capy organization with a connected GitHub account can also ask for a round with @capy review.