Skip to main content

A Code Review Can Be Correct and Still Be ....

Updated 10 min readMarkdown

One thing we've been thinking about while building Codzee:

A code review can be technically correct and still be almost useless.

We keep coming back to this because it's easy to assume that if a review identifies a problem, the review has done its job.

I'm not sure that's true.

Consider a reviewer looking at a function and writing:

“Consider refactoring this function for readability.”

Maybe they're right.

Maybe the function really is difficult to read.

But as the person who wrote the code, I'd probably have a few questions:

  • Which part is difficult to read?

  • Why is it a problem?

  • What kind of refactoring are you suggesting?

  • Is this actually a problem, or just your preferred style?

  • What would “better” look like?

Now compare it with something like:

“This returns null when the user has no active subscription, but the caller immediately dereferences .plan. That turns a valid ‘no subscription’ state into a runtime exception. Handle the empty case before accessing plan, or make the function contract explicit if null isn't expected here.”

That's a very different review comment.

It doesn't just point at the code.

It explains the reasoning.


This is the problem we're interested in

The distinction we're exploring is basically:

Flagging vs. explaining.

A flag says:

“Something might be wrong here.”

An explanation says:

“Here's what is happening, here's why it matters, and here's what I think you should investigate.”

Those are not the same thing.

And this isn't really an AI problem.

Humans write vague review comments all the time.

We've all seen:

“This could be cleaner.”

“Please refactor.”

“Potential issue.”

“Why are we doing this?”

“This isn't scalable.”

Sometimes these comments are perfectly reasonable starting points.

The problem is when the author has to spend more time figuring out what the reviewer meant than fixing the actual issue.


What we initially thought

When we started thinking about Codzee, one obvious direction was:

Find more things in code.

That sounds reasonable for a developer tool.

If a reviewer can catch a bug, performance problem, security problem, or maintainability issue that someone else missed, that's valuable.

But the more we think about code review, the less satisfied we are with “find more problems” as the entire objective.

Because there is a second question:

What happens after the problem is found?

If the output is:

“Potential performance issue.”

the developer still has work to do.

They need to figure out:

  • What causes the performance issue?

  • How serious is it?

  • Under what conditions?

  • Is it actually relevant to this code?

  • What should change?

So we've started thinking less about the number of issues a review can identify and more about the quality of the information attached to each issue.


What we've changed our mind about

Initially, I think we were too focused on the idea that a good automated reviewer should behave like an extremely thorough human reviewer.

Find everything.

Point out everything.

Leave comments everywhere something looks suspicious.

I'm less convinced now.

There is a cost to every comment.

Someone has to read it.

Someone has to understand it.

Someone has to decide whether it matters.

Someone may have to respond to it.

And if enough comments are noisy, people start mentally filtering the review.

That's not a theoretical problem.

Reviewers and authors have finite attention.

So we're increasingly interested in a different question:

Can a review produce fewer, higher-confidence observations with enough context to make them useful?

I don't know yet whether that's the right approach.

But it's a more interesting problem to us.


What developers actually seem to value

We don't want to claim we've solved this.

We haven't.

But one thing that keeps standing out in how we think about review feedback is that developers need context.

For example, compare:

Generic

“This query is inefficient.”

Contextual

“getCustomer() runs a database query inside this loop. If this returns 500 orders, the request can trigger hundreds of additional queries. Consider loading the required customers before the loop and reusing them here.”

The second one gives the engineer something to reason about.

It doesn't necessarily prescribe the final implementation.

It explains the mechanism.

That's important.

Because sometimes the reviewer is right about the problem but wrong about the solution.

A good review should make that disagreement possible.


We're thinking about reviews in three parts

One simple framework we've been using internally is:

WHAT?

What exactly is wrong?

WHY?

Why does it matter?

NEXT?

What should the engineer investigate or change?

For example:

WHAT: This can return null.

WHY: The caller immediately dereferences the result, turning a valid empty state into an exception.

NEXT: Handle the empty case before accessing the property.

This isn't meant to become some rigid format where every review comment becomes three paragraphs.

For a typo:

“Typo: recieve → receive.”

is enough.

The framework is mostly useful when the issue isn't obvious.


We also don't want to confuse preference with defects

This one is surprisingly important.

Consider:

“I'd use a map here.”

Okay.

Why?

Maybe the current implementation is O(n) for every lookup.

Maybe there's a performance problem.

Or maybe the reviewer just likes maps.

Those are very different things.

Compare:

“I'd use a map here.”

with:

“This lookup scans the collection every time, and this path can execute once per item in the request. Since we're repeatedly looking up by ID, an indexed structure would avoid the repeated scans.”

Now there's an engineering argument.

The author can agree or disagree with it.

That's what we want.

We don't want tooling to turn every reviewer preference into an authoritative-sounding “issue.”


What we're testing

We're exploring whether review feedback becomes more useful when the system is pushed toward:

  • specific code references

  • concrete failure modes

  • consequences rather than generic warnings

  • relevant repository context

  • distinction between defects and suggestions

  • confidence around claims

  • explanations that are proportional to the evidence

The interesting part is that some of these are easy locally.

For example:

“This value can be null and is dereferenced here.”

That's mostly a code-level observation.

Others are much harder.

For example:

“This retry policy is incorrect.”

That might require knowing:

  • what the upstream service guarantees,

  • whether requests are idempotent,

  • what the expected timeout is,

  • whether retries happen elsewhere,

  • and what the production failure mode looks like.

We're still figuring out how much context is actually necessary before an automated system should make claims like that.


What could be wrong with our thesis?

Quite a lot.

The first possibility:

Maybe developers don't actually want more explanation.

Maybe concise flags are better because experienced engineers already know what to investigate.

That's probably true for some comments.

A senior engineer doesn't need a paragraph explaining what a null pointer is.

So we're trying not to confuse “more context” with “more words.”

The goal should be useful information, not verbosity.


Another possibility:

Maybe the reviewer shouldn't explain the solution at all.

There are cases where identifying a problem is enough.

And sometimes prescribing a solution is harmful because the author understands the surrounding system better than the reviewer.

That's why we're interested in explanations of the problem rather than automatically generating a complete fix for every comment.


Another:

Maybe AI can't reliably understand intent.

This one seems especially important.

A piece of code can look redundant because the reason for it exists somewhere else:

  • an external API is unreliable,

  • an old client has a strange contract,

  • a database migration is in progress,

  • a security requirement isn't obvious from the code,

  • a previous production incident changed the implementation.

If the system doesn't know that context, it can produce a technically sophisticated explanation that's still wrong.

That's probably one of the biggest risks in automated review.

A confident explanation isn't useful if the premise is wrong.


There's another problem: review fatigue

One thing we've become more cautious about is the assumption that finding more issues automatically improves code quality.

It doesn't necessarily.

Imagine opening a pull request and seeing:

Potential issue.

Consider refactoring.

This could be optimized.

Possible security concern.

This might be cleaner.

Consider changing this.

Even if several of those comments are technically defensible, the reviewer now has a much bigger cognitive task.

Which ones matter?

Which ones are real?

Which ones are preferences?

Which ones are worth fixing before merge?

At some point, the review system itself becomes noise.

So we're interested in a slightly uncomfortable possibility:

A good automated reviewer might sometimes be the one that says nothing.

If the evidence isn't strong enough, perhaps the correct action is not to generate a comment.

We're still working out where that threshold should be.


One thing we definitely don't believe

We don't think the answer is:

“AI should replace human code review.”

Code review contains judgment.

It contains system context.

It contains architectural decisions.

It contains questions about intent.

Those aren't things we want to pretend can always be inferred from a diff.

The more interesting role for automation, at least from where we're sitting, is helping surface things worth discussing while giving the engineer enough context to decide whether the observation is actually correct.

The human still gets to say:

“No, that's intentional.”

And ideally, the system should be able to understand that response rather than simply repeating the same warning.


So where are we now?

We're still building.

And we're still uncertain about several things.

We're trying to understand whether engineers prefer:

more findings

or

fewer findings with stronger reasoning.

We're trying to understand how much repository context is necessary before an automated reviewer can make a reliable claim.

We're trying to understand when a comment should be:

“This is wrong.”

versus:

“I may be missing context, but if X is possible, doesn't this create Y?”

We're trying to understand how to distinguish:

“This is objectively dangerous”

from:

“This is not how I would write it.”

And we're trying to understand whether explanation quality can actually be evaluated in a useful way.

Those questions are probably more interesting than simply asking how many comments a code-review system can generate.


The question we're left with

The thesis we're exploring with Codzee is simple:

A label isn't a code review.

bug, security, performance, style, and design can all be useful labels.

But they're only the beginning.

The useful part is the reasoning underneath them.

What is happening?

Why does it matter?

What should the engineer think about next?

We're not sure we've got the formula right yet.

That's part of why we're building.

I'd genuinely like to hear from other founders and developers working on developer tools or dealing with large codebases:

  • How do you evaluate code-review quality?

  • What review comments annoy you most?

  • What makes you trust an automated review?

  • Where should AI stop and let humans decide?

  • Do you prefer fewer high-confidence comments or broader coverage?

Interested in hearing how other teams think about this.

About the author

  • Codzee is building the future of AI-powered code reviews for modern engineering teams. We help developers review pull requests faster, improve code quality, and ship software with confidence. Codzee analyzes your code, highlights potential issues, suggests meaningful improvements, and delivers actionable feedback that fits your team's coding standards and workflow. Our mission is simple: make code reviews faster, smarter, and more collaborative without disrupting how developers work. Whether you're a startup shipping your first product or a growing engineering organization, Codzee helps reduce review bottlenecks, maintain consistent code quality, and improve developer productivity. At Codzee, we believe AI should support developers, not replace them. That's why we're building AI Code Review That Respects Your Team.

No comments yet. Be the first to comment.

More from this blog