Appendix J: Contracts and fault injection on repos that are not the founder's (2026-09-04)
Read-only against local clones under ~/.explore/repos/github.com/. Same scripts as appendix I with STRICT_AGENT=1 (trailer-only attribution; a project named "codex" mentions itself in every commit). Fault injection via the shipped panout inject on a scratch worktree with a throwaway data directory; no hooks installed in any external repo.
Why this exists
The round-5 judge's list of what 90 requires included "two contracts with fault-injection catch rates that separate on a repo that is not the founder's" and a check that the retrospective override rate is not a property of the founder's habits. Six public repositories with agent-attributed commits were cloned (shallow since 2026-02-01): garrytan/gstack, cursor/plugins, openai/codex, obra/superpowers, badlogic/pi-mono, steipete/summarize. gstack is the judge's own methodology repo and is 91% agent-attributed in the window, which makes it the most relevant single test.
Retrospective contract backtest, six public repos
| repo | commits in window | measured | agent (trailer) | override rate |
|---|---|---|---|---|
| garrytan/gstack | 347 | 346 | 316 | 56% |
| openai/codex | 4,590 | 600 | 0 | 60% |
| badlogic/pi-mono | 2,153 | 600 | 2 | 33% |
| steipete/summarize | 696 | 600 | 12 | 29% |
| obra/superpowers | 358 | 358 | 45 | 15% |
| cursor/plugins | 114 | 114 | 27 | 12% |
| pooled (non-bot) | 2,565 | 402 | 38% |
Pooled: exposure 100%; override rate 38% (agent 52%, human 35%). Per contract: size-guard fails on 20% of commits (agent 46%, human 15%), tests-touch on 29% of evaluated commits (agent 12%, human 31%), no-secrets on 9 commits.
openai/codex has zero agent trailers in 4,590 commits. The repository is built with the product it ships, and its squash-merge workflow strips every co-author line. This is the attribution-loss claim from appendix D observed at scale on the vendor's own repo: nothing in git says which of those commits an agent wrote, and no one can add it afterwards.
Predictive test, external
| contract | population | rework if fail | rework if pass | delta (pp) | 95% CI |
|---|---|---|---|---|---|
| size-guard | all | 92% (515) | 62% (1,994) | +30 | +27 to +33 |
| size-guard | agent | 95% (183) | 80% (210) | +15 | +9 to +21 |
| tests-touch | all | 62% (500) | 77% (1,267) | −14 | −20 to −10 |
| no-secrets | all | 78% (9) | 68% (2,500) | +9 | −23 to +32 |
| any contract | all | 77% (964) | 63% (1,545) | +14 | +10 to +17 |
Size-stratified any-fail versus all-pass: +5, −2, −8 points in the three comparable bins. Size-adjusted tests-touch: −0 weighted (−9 agent-only). Size-adjusted no-secrets: +8 on six failures. The founder-repo result replicates exactly: unadjusted, contract failure predicts 14 to 30 points more rework; adjusted for size, the defaults carry no signal except size itself.
gstack in detail
| value | |
|---|---|
| commits measured | 346 (316 agent, 30 human) |
| override rate | 56% (agent 58%, human 30%) |
| size-guard fail rate, agent commits | 55% |
| tests-touch fail rate (244 evaluated) | 6% |
| agent rework within 30 days | 91% |
| human rework within 30 days | 53% |
| size-guard fail → rework, agent | 97% vs 85% if pass (+11, CI +5 to +18) |
| tests-touch fail → rework | 71% vs 93% if pass (−22, CI −45 to +1) |
More than half of gstack's agent commits exceed 400 added lines, and nine in ten agent-written lines are changed within 30 days. That is a repo that would receive an override record on every second commit under the defaults, and whose author would learn on day one that size-guard is the only default carrying signal there.
Fault injection, external
panout inject --commits 20 replays accepted agent-attributed commits on a scratch worktree, applies four faults, and credits a catch only when the contract passed the unfaulted diff.
| repo | contract | fault | presented | caught |
|---|---|---|---|---|
| gstack | tests-touch | drop-test | 15 | 15 (100%) |
| gstack | no-secrets | secret-string | 15 | 15 (100%) |
| gstack | size-guard | oversize | 1 | 1 |
| gstack | any contract | assertion-delete | 19 | 0 |
| openai/codex | tests-touch | drop-test | 14 | 14 (100%) |
| openai/codex | no-secrets | secret-string | 16 | 16 (100%) |
| openai/codex | size-guard | oversize | 5 | 5 (100%) |
| openai/codex | any contract | assertion-delete | 18 | 0 |
| cursor/plugins | all | all | 0 | not measurable in this checkout |
Excluded because the contract already failed at baseline: 57 cases on gstack, 48 on codex. On gstack size-guard was presented only once because it was already failing on 55% of the replayed commits; a contract that fails at baseline cannot be credited with catching anything, which is the rule that appendix I's size confound makes necessary.
Two contracts (tests-touch, no-secrets) separate cleanly on two repos that are not the founder's: each catches 100% of the fault it was written for and 0% of the other three. The assertion-delete fault is caught by nothing, on every repo, including the founder's. That is the first contract the product should ship next, and the injection harness already measures whether it works.
What this adds to the memo
- The retrospective override rate is not a founder artefact. It ranges 12% to 60% across six public repos, 38% pooled, higher than the founder's 26%.
- The size confound replicates. No default contract other than size predicts rework once size is controlled, on repos the founder never touched.
- Two contracts separate under fault injection on external repos, which was on the judge's list. Whether that separation is worth paying for is still an empirical question about contracts a team writes itself, not about the defaults.
- The vendor's own repository has no attribution. The record Panout writes at commit time is the only place it could exist.
Limits
- Six repos chosen for having agent trailers; not a sample of anything. gstack contributes 316 of 402 agent commits.
- Shallow clones since 2026-02-01: rework endpoints before mid-March may be truncated; the window starts 2026-03-08 to reduce this.
cursor/pluginsfault injection presented zero cases (the cherry-picks did not apply in that checkout); not investigated further.- Same proxy caveats as appendix I.