Panout review packOverviewFinal memoMVPLive outputsJudge roundsAppendicesDecisions

Source: docs/research/2026-09-03-defensibility/appendices/B-public-evidence.md

Panout — public evidence file

Compiled 2026-09-03. Every claim carries a URL. Items I could not confirm from a primary or reputable secondary source are tagged UNVERIFIED. Facts only — no recommendations.

Method note for the GitHub measurements: I used the authenticated gh CLI (gh api search/issues, gh api repos/{o}/{r}/pulls/{n}, .../pulls/{n}/reviews). Search results were sorted by updated, so samples skew toward recently-active repos and are not a uniform random sample of all merged agent PRs. Treat the percentages as indicative of a large, real population, not as unbiased estimates.


Q1. Do developers merge agent-authored code without reading it, at scale?

1a. Population size (measured 2026-09-03 via gh api search/issues)

Merged PRs on public GitHub, by agent identity:

QueryMerged PR count
is:pr is:merged author:app/copilot-swe-agent1,496,410
is:pr is:merged head:codex/ (Codex creates codex/* branches)5,830,786
is:pr is:merged head:cursor/694,347
is:pr is:merged author:app/google-labs-jules200,925
is:pr is:merged head:devin/180,667
is:pr is:merged author:app/devin-ai-integration144,640
is:pr is:merged "Co-Authored-By: Claude"346,405
is:pr is:merged author:app/claude33,193

Reproduce: gh api -X GET search/issues -f q='is:pr is:merged author:app/copilot-swe-agent' -f per_page=1 --jq .total_count Search UI equivalent: https://github.com/search?q=is%3Apr+is%3Amerged+author%3Aapp%2Fcopilot-swe-agent&type=pullrequests

Notes: author:app/codex, author:app/chatgpt-codex-connector and author:app/openai-codex all return 0 — Codex pushes branches under the human's identity, so head:codex/ is the usable proxy. Branch-prefix queries therefore mix agent-authored and human-authored commits on the same branch; counts are an upper bound. Counts across rows overlap.

1b. Review depth on merged agent PRs (measured samples)

I sampled merged PRs and pulled, for each, the formal review count (/pulls/{n}/reviews length), review comments, issue comments, diff size, and created→merged latency.

Sample A — author:app/copilot-swe-agent, all repos, n=150

Sample B — author:app/copilot-swe-agent stars:>500, n=98

Sample C — head:codex/ stars:>500, n=100

Sample D — author:app/devin-ai-integration, all repos, n=150

Sample E — CONTROL: merged PRs NOT from app/copilot-swe-agent, stars:>500, created:>2026-08-01, bot authors (*[bot]) stripped, n=85

This control substantially qualifies the headline. In popular repos, merging with no formal review is the norm for all PRs, not a distinctive property of agent PRs: the Copilot >500-star sample (50% zero-review) and the codex-branch >500-star sample (40%) are statistically indistinguishable from the 49% baseline. Solo maintainers self-merging, trunk-based teams that review out-of-band, and PRs used as a deploy mechanism all produce zero-review merges.

Two things survive the control:

  1. Merge latency in the unfiltered Copilot sample is an order of magnitude faster — median 1.3 minutes and 70% merged inside 10 minutes, versus 25.1 minutes and 34% for the control. A 397-line mean diff merged at a 1.3-minute median is not consistent with reading.
  2. The control is itself contaminated in a way that cuts toward the thesis, not against it. Codex, Claude Code and Cursor push under the human's GitHub identity, so an unknown and growing share of the "control" is itself agent-authored (see the Claude Code ~4%-of-public-commits estimate in Q6). There is no clean human-authored baseline available from the GitHub API in 2026.

Reproduce the control: gh api -X GET search/issues -f q='is:pr is:merged -author:app/copilot-swe-agent stars:>500 created:>2026-08-01' -f per_page=50 -f sort=updated

Caveat: "zero formal reviews" is a lower bound on unread merges in one direction (a reviewer can read without leaving a review) and an upper bound in another (an approving review can be rubber-stamped). Merge latency under 10 minutes on a 1,000+ line diff is the stronger signal.

Example merged PRs with zero reviews, zero review comments, zero issue comments, self-merged by the repo owner:

1c. Stated behaviour: developers say they don't read the diffs


Q2. Do unread agent merges cause incidents?

2a. Named incidents

2b. Revert volume on GitHub (measured 2026-09-03)

QueryMerged PR count
is:pr is:merged in:title "Revert" in:title "codex"1,793
is:pr is:merged "Revert" head:revert- in:title "copilot"385
is:pr is:merged in:title "Revert" in:title "Devin"37
is:pr in:title "revert" "AI-generated" is:merged29
is:issue "AI-generated" "broke production" in:body29

These are title-string matches, so they undercount (most reverts don't name the agent) and overcount slightly (title text can coincide). No star-filtered examples returned, so I have no high-profile named revert to point at.

2c. Aggregate defect / churn / instability numbers


Q3. Is anyone asked for provenance of AI-written code?

Short answer from the evidence: provenance obligations today attach to AI-generated content and training data, not to the authorship of source code. I found no framework, auditor requirement, or standard questionnaire that asks "what fraction of your code is AI-generated."


Q4. Comparables — network-effect trust layer, priced per transaction


Q5. What incumbents already ship at this boundary (Sept 2026)

For each, the three probe questions are: (a) learns a per-team acceptance bar from history? (b) enforces autonomy levels per task class? (c) records rejected/discarded agent work that never became a commit?

GitHub Copilot code review

https://docs.github.com/en/copilot/concepts/agents/code-review

GitHub rulesets + merge queue + auto-merge

Linear — Coding Sessions and Guided Reviews

Cursor — Bugbot, Agent Review, Security Review

Anthropic Claude Code — hooks, OTel, autonomy modes

OpenAI Codex — code review

https://developers.openai.com/codex/integrations/github

"Entire CLI" (experts, judge)

UNVERIFIED — I could not find any product named "Entire" with experts or judge features in public search. Either the name differs, or it is unindexed/private. The nearest public analogues are Cursor's selectable effort levels and public benchmark harnesses (SWE-bench Verified, Terminal-Bench 2.0) used as judges.

Summary of the three probes across all incumbents

Product(a) learns per-team bar from history(b) autonomy per task class(c) records discarded agent work
GitHub Copilot code reviewNoNo (effort levels only)No
GitHub rulesets / merge queueNoNoNo
Linear Coding Sessions + Guided ReviewsNoNoPartial (within-session only)
Cursor Bugbot / Agent ReviewNo (hand-written BUGBOT.md rules)No (effort dial)No
Claude Code hooks + OTelNoPartial (modes/hooks, hand-written)Partial (OTel tool calls)
OpenAI Codex reviewNoPartial (fixed P0/P1 filter)No

No incumbent found in this pass ships (a). None ships (b) as a per-task-class policy engine. None ships (c) as a first-class record.


Q6. Size of the wave

Share of code written by AI

Users and revenue

GitHub-wide volume

Measured directly today rather than cited: >5.8M merged PRs on codex/* branches and >1.49M merged PRs from app/copilot-swe-agent alone (see Q1a). A total annual GitHub PR/commit figure for 2026 (Octoverse-equivalent) was not located in this pass — UNVERIFIED.


Confidence summary

Strongest, directly measured: Q1a population counts and Q1b review-depth samples — these are my own measurements against the GitHub API and are reproducible today. But read Sample E before using Q1b: the zero-review rate on agent PRs in popular repos is not meaningfully different from the all-PR baseline. The defensible measured claim is about merge latency, not about review counts.

Strong, well-sourced secondary: Q2c aggregates (DORA, GitClear, Veracode, METR), Q5 product capabilities (mostly vendor docs and changelogs), Q4 comparables.

Weakest: Q2a named incidents beyond Amazon (mostly blog aggregation), Q3 (the honest finding is negative — nobody is asking for code-authorship provenance yet), and the Q6 revenue/user figures, which are almost entirely third-party estimates.