How do I review AI-generated code that looks correct?
Published
By the Unrust team · 5 min read · Editorial method
Write the contract, find a counterexample and check the fix. Practise on a JavaScript validator that passes ordinary examples but accepts an empty batch.
Review a claim you can test
Translate the requirement into observable behavior before judging how readable the implementation looks. State which inputs are valid, what the result means, and which side effects are allowed. Then choose an example that could make the implementation disagree with that statement. An assistant's explanation is a claim to verify alongside the code.
Start at the changed behavior. For a validator, inspect missing and boundary values. For a retry, inspect the number of attempts and repeated side effects. For a data transformation, inspect empty input, duplicates and mutation. The point is to choose cases from the contract, rather than asking the same tool for an unrestricted second opinion.
Try it: should this batch be accepted?
The contract is narrow: input is a dense array of integer readings; a valid batch contains at least one reading, and every reading must be between 0 and 100 inclusive. This is an original illustrative review exercise, not a claim that a particular model generated the code. Predict all three results and identify the contract violation.
function validBatch(readings) {
return readings.every(n => n >= 0 && n <= 100);
}
console.log(validBatch([0, 100]));
console.log(validBatch([101]));
console.log(validBatch([]));Reveal the answer and explanation
The results are true, false and true. The empty batch violates the requirement that at least one reading exists. every() returns true on an empty array because there is no element that fails the predicate.
Require a nonempty array before checking its elements. This repair matches the stated dense-integer-array contract. If callers can supply arbitrary JSON or sparse arrays, the boundary needs additional validation; this helper alone does not establish that broader guarantee.
function validBatch(readings) {
return readings.length > 0 &&
readings.every(n => n >= 0 && n <= 100);
}Make the review comment actionable
A useful review comment contains the requirement, the counterexample and the observed result. Here: ‘A batch must contain a reading. validBatch([]) returns true. Add a nonempty check and keep the empty-array case as a regression test.’ Another developer can reproduce that comment without accepting your authority or the assistant's confidence.
Check the repaired version against more than the failing case: [0] and [100] should pass; [-1] and [101] should fail; [] should fail. A repair that rejects everything would fix the reported symptom while breaking the contract. Keep valid cases in view when reviewing a proposed correction.
Follow the claim across files when necessary
A small function can be correct in isolation while its caller supplies the wrong units or ignores its result. Follow one representative value from the boundary through the changed function to the output or side effect. Write down where validation happens and which assumptions still depend on another file.
For database work, inspect the actual rows and null cases a query returns. For an incident, distinguish the first visible symptom from the line that caused it. Review is strongest when you can point to an input, a path through the program and an observation that supports your conclusion.
Practise the decision before the next pull request
Unrust offers drills in finding flaws, tracing state and checking plausible code against a requirement. Its practice library also includes cross-file reasoning, log investigations and SQL query exercises. These let you isolate a review decision before trying it in a larger repository.
The free diagnostic includes AI-slop detection alongside five other dimensions. A result does not certify a production review, and passing a drill does not cover your application's security or operational requirements. Use the practice to build explanations you can verify, then apply the real project's tests and review process.
Questions from the community
These public discussions informed the question this article answers. They reflect personal experiences, not evidence of how common a problem is or what caused it. Sources checked October 28, 2026.
- Medium: debugging plausible AI-generated code
Ayoub Nainia's publicly visible introduction raises the problem of readable code with incorrect behavior. We inspected that introduction; the full article is member-only.
- X: reviewing agent changes and their assumptions
Karpathy's account describes conceptual mistakes and assumptions he had to catch while using coding agents.
References and next steps
Try another review decision
Open Unrust's free public code-review challenges for more original examples with counterexamples and explanations. These examples do not produce a diagnostic score.