• Core Finding: LLMs are systematically worse at catching bugs in their own code than in code authored by competing models.
  • In Greptile's evaluation of 1,000 PRs, GPT-5.5 caught more high-severity bugs in Claude-authored PRs (60.0% vs. 53.7%), while Claude Opus 4.7 beat GPT on Codex-authored PRs (62.0% vs. 50.5%).
• Blindspot Correlation: The specific bug types a model introduces most frequently mirror the exact categories it misses during self-review.
  • Claude PRs frequently suffer from missing behavior/data errors (where GPT excels as a reviewer).
  • Codex PRs tend to produce semantic intent and error-handling flaws (where Claude exhibits stronger recall).
• Contrasting Review Methodologies:
  • Claude Opus 4.7 (Breadth-First): Dedicates ~59% of trace context to broadly scoping diffs; posts 7–8 comments per PR, hedging on author intent, pointing out future risks, and giving stylistic feedback (higher false positives).
  • GPT-5.5 (Depth-First): Dedicates ~83% of trace context to deep investigative tool calls; posts only 1–2 comments per PR due to aggressive post-training noise-suppression and developer-scoping alignment.
• The "Tunneling" Omission Failure: Traces revealed GPT frequently identified severe bugs (e.g., database deadlocks) during internal reasoning, but omitted them from final output because alignment tuning prompted it to minimize developer friction.
• Model Inversion Routing: Based on these findings, Greptile introduced an automated routing feature that inspects commit trailers, branch prefixes, or PR titles to ensure the reviewing LLM is never the same model that authored the PR.

• Context & Scale: With Claude writing ~90% of Claude Code's codebase and changes across Anthropic's monorepo surging from 2,500/day to >30,000/day, the team transitioned from manual line-by-line review to an automated, multi-layered pipeline, more than doubling merged PRs while reducing reverts by over 40%.
• Core Philosophy: Treats human attention as the scarce bottleneck, reframing code review from an end-of-process human gate to continuous, automated validation paired with fast rollback mechanisms; success is measured by human review hours without increases in reverts or incidents.
• The Seven Review Layers:
  • 1. Pre-PR Verification: Claude must execute verifiable tests, builds, or browser runs before opening a PR; a fresh-context subagent reviews the diff against project guidelines (CLAUDE.md, REVIEW.md).
  • 2. Automated AI Review: Multi-agent reviews analyze the entire repo and intent; bug-hunting agents propose findings that a voting panel of verifier agents filters (dropping nits/style) and ranks by severity (Important, Nit, Pre-existing).
  • 3. Dedicated Security Review: An automated security pass runs independently; unresolved findings from either AI bug review or security review strictly block merges until fixed or dismissed with explicit rationale.
  • 4. Standard CI Gates: Cross-platform builds, type checks, linting, and automated test suites must pass completely.
  • 5. Risk-Based Human Approval: A classifier categorizes risk tiers to determine human oversight:
    • Low Risk (docs, copy, internal-only, flag-guarded changes): Zero human approvals needed once automated checks and AI reviews pass.
    • Medium Risk (standard features, caching, system prompts, auto-updater): Exactly one human approval.
    • High Risk (critical paths, CI workflows, designated sensitive files): Requires two human approvals.
  • 6. Merge Queue & Gradual Rollout: Validates changes against latest main via merge queues to avoid integration drift, followed by employee-dogfooding behind feature flags and staggered channel releases (latest vs. stable).
  • 7. Rapid Detection & Fast Rollback: Emphasizes quick flag deactivation and immediate reverts over lengthy fixes, conducting post-incident reviews to identify which upstream layer failed.
• Ownership Model ("You Land It, You Own It"): The engineer directing the agent is fully responsible for the changes and counts as an author/reviewer; AI approvals can never substitute for required human slots.

• The AI Code Review Bottleneck:
  • Rapid AI code generation shifts the primary software engineering bottleneck from writing code to reviewing and maintaining it.
  • Unvetted AI-generated code leads to growing cognitive debt, potential production outages, and burnout among senior engineers.
• Automated Guardrails & Self-Correction:
  • To scale or eliminate manual code reviews, feedback loops should be engineered directly into the local agent environment.
  • Deterministic tools like Semgrep, linters, and architectural unit tests enforce coding standards and prevent bad patterns automatically.
  • Harness stop hooks and execution loops (e.g., Ralph loops or goal modes) automatically feeds error output back to agents for self-correction.
• Importance of the Agent Harness:
  • The harness (providing tools, memory, and execution capabilities) often impacts performance and success rates more than the specific LLM.
  • Tool capabilities and harness behaviors change rapidly, requiring teams to continuously experiment rather than imposing rigid vendor policies.
• Shift to Upfront Architecture & Specification:
  • AI shifts the engineer's core role toward high-level system design, defining modular boundaries, and writing behavioral specifications/tests up front.
  • Developers can mine AI session logs to identify recurring manual corrections and turn them into permanent automated guardrails.

1. The Scaling Crisis (Why Reviews are "Dead")

• Throughput vs. Cognition: The fundamental problem is a mismatch in speed. AI agents (like Cursor, Claude Code, or GitHub Copilot) can generate code at a rate 10x–100x faster than a human. If a developer uses an agent to clear 20 tickets in a morning, the human reviewer becomes a permanent, overwhelmed bottleneck.
• The "Rubber Stamp" Failure: In high-volume AI environments, human reviewers stop actually reading the code and start "rubber-stamping" (approving without checking). This creates a false sense of security while letting "slop" (low-quality, redundant, or slightly buggy AI code) leak into the codebase.

2. The Shift to "Spec-Driven Development"

The author posits that the "checkpoint" for quality is moving "upstream." * Reviewing Intent, Not Implementation: Engineers will spend their time reviewing the Specification or the Prompt rather than the resulting lines of code. If the spec is correct and the constraints are tight, the implementation is treated as a disposable artifact. * Natural Language as the New Source Code: The high-level description of a feature becomes the primary source of truth. If a bug is found, the human doesn't "fix the code"; they "fix the spec" and have the agent regenerate the solution.

3. Verification over Review

The traditional manual "looking at code" is being replaced by automated verification loops: * Agentic Testing: Agents are now responsible for writing their own unit tests and integration tests. The human's job is to review the test plan and verify that the tests actually cover the business requirements. * Formal Methods & Type Safety: The article emphasizes using "bounded interfaces" (like Infrastructure-from-Code or strict APIs). When the architecture is strictly typed, the AI is physically unable to make certain classes of errors, reducing the need for human oversight of the "plumbing."

4. The Emergence of "Cognitive Debt"

A primary warning in the text is the rise of "Cognitive Debt"—a state where a codebase is so dominated by AI-generated logic that no single human understands how the entire system works. * Tech Debt vs. Cognitive Debt: While tech debt is "bad code that's hard to change," cognitive debt is "working code that is impossible to reason about." * The New Senior Role: Senior engineers must transition into "System Curators" who manage this debt by enforcing architectural simplicity and ensuring that AI-generated modules remain modular and replaceable.

5. Future Prediction

• 2025: The year human-written code became the minority.
• 2026: The year the "Pull Request" as a human-to-human review ritual officially becomes obsolete in high-performing teams, replaced by automated agent-to-agent validation and human-to-spec oversight.

(comment with good examples of code review comments)

codereviews

In doing a code review, you should make sure that:

• The code is well-designed.
• The functionality is good for the users of the code.
• Any UI changes are sensible and look good.
• Any parallel programming is done safely.
• The code isn’t more complex than it needs to be.
• The developer isn’t implementing things they might need in the future but don’t know they need now.
• Code has appropriate unit tests.
• Tests are well-designed.
• The developer used clear names for everything.
• Comments are clear and useful, and mostly explain why instead of what.
• Code is appropriately documented (generally in g3doc).
• The code conforms to our style guides.