Source: docs/research/2026-09-03-defensibility/rounds/r3-judgment.md

Panout round 3 — investment judgment: "The rework layer for agent labor"

Judge: Garry Tan lens. Methodology: gstack office-hours startup mode (six forcing questions, anti-sycophancy rules, end with an assignment) plus plan-ceo-review cognitive patterns (inversion reflex, proxy skepticism, focus as subtraction, classification instinct). Date: 2026-09-03. Self-imposed dogfood gate: 2026-09-05 — now reported, as a failure, two days early. Question scored: would I write the check today for a defensible, $1B-capable company, given this founder's constraints and this evidence? Calibration: prior judges scored the best round-1 idea 62, the best round-1 combination 72, and round 2 66. I read both for scale only. Every number below was re-checked against the appendix it cites, and several against the repos themselves.

Score: 67/100

Verdict: the most honest document any founder has put in front of me this year, and the company is barely more fundable than it was yesterday — because the two things that were going to decide it are respectively untouched (external humans) and now provably unanswerable until October (the predictive delta). You went and measured the thing round 2 said would kill you, it did kill it, and you changed the company instead of defending the memo. That is worth real points. It is not traction.

Dimension table

Raw: 65/100.

Gut adjustment: +2. Stated plainly. Up, for two things not in the rubric: you ran the measurement that could kill the company, it returned an answer 460× short of what your moat needed, and you rewrote the company instead of the memo — that is the single most predictive founder behavior in this entire file, and I pay for it. And appendix D's move — converting a null result into a design constraint ("the features must be captured at commit time, or they are gone within three weeks") — is genuinely good reasoning, not a rescue. Down, against: round 2's judge docked for gate silence and gave +3. The gate is no longer silent, but what replaced it is worse in one respect — twelve days, three plans, four documents, two outstanding assignments, zero external humans, zero commits. The memos are improving at exactly the rate the company is not. I will not pay +3 for a fourth document.

Final: 67/100.

The two sentences, verified

Moat, as written

Human review of agent work has not been replaced, it has moved: on this machine humans reject 0.14% of agent actions in session and then rework 60% of agent commits within 30 days, and nobody joins the two. Panout is installed at the only point where the features that explain rework (task class, harness, model, session shape, contract result) can be captured, at commit time, across every harness; the label arrives from git 30 days later, and a team's calibrated map of which task classes it can stop reading cannot be rebuilt by anyone who was not capturing the features when the commit happened.

Would I repeat this to a partner? No. Four things break, and three of them break on your own appendices.

1. "rework 60% of agent commits" without "versus 38% for the commits he writes by hand" is the number a partner looks up and then discounts by two thirds. You know this — the memo says it two paragraphs later. Never lead with the level. The gap is the claim.
2. "the features that explain rework" is unproven, and appendix D says so in its own headline. You have no evidence any of those five features explains rework beyond diff size. Do not put "explain" in a sentence you cannot defend for another month.
3. "the only point where [they] can be captured" is false for two of the five. Task class comes from paths and commit shape; harness and model come from commit trailers. Both are in git forever, and appendix A proves it by classifying 15,440 commits over 180 days from trailers alone. A partner who reads your own appendix A method section finds this in four minutes.
4. "the only point" is the same word that killed round 2's sentence, for the same reason. It is not the only point. It is the only neutral, cross-harness point, and even that is a claim about who bothers, not about physics.

Rewrite — what I would actually repeat:

In this founder's own repos, agent commits get silently rewritten within thirty days about twenty points more often than the commits he writes by hand, and nothing on earth connects that outcome back to which harness, model and task class produced it — because the session logs that hold the cause are deleted in twenty-one days and the consequence takes thirty to appear. Panout stands at the commit boundary writing the cause into the commit before it expires, so a team's map of which task classes it can safely stop reading is uncopyable by anyone who was not recording when the commit happened.

Note what I kept and what I cut. I kept the retention asymmetry, because 21 < 30 is a real physical barrier and it is the best idea in your memo. I cut "explain," "only," and the 0.14%-versus-60% juxtaposition, and I put the gap in place of the level. The sentence is now defensible against everything in your own appendices — which is the actual test.

$1B, as written

Agent output per engineer now grows faster than human reading capacity, so every team is being forced to decide what it will stop reading, and that decision is worth a metered fee per unit of work evaluated rather than per human seat. The meter is paid the same whether Panout clears or flags, and it compounds every time models get good enough to be trusted with more, which makes it the one revenue line in developer tooling that strengthens as models improve.

Half yes. The structural claim is right and the "paid the same whether we clear or flag" line is the correct fix — it removes the issuer-pays conflict and starts revenue on day one instead of day forty-five. Two problems. It contains no number, so it is a positioning sentence, not a $1B sentence; the round-2 rewrite you adopted had the ramp in it and you dropped it. And "the one revenue line in developer tooling that strengthens as models improve" is an absolute a partner breaks in one move: every token-metered product does, and Bugbot at $1.20 a run — which you cite as prior art two paragraphs earlier — does exactly that.

Rewrite:

Agent output per engineer is now growing faster than human reading capacity, so every team is being forced to decide what it will stop reading, and that decision is worth a metered fee per unit of work evaluated rather than per human seat — paid identically whether we clear it or flag it. Two thousand teams at five hundred evaluated outcomes a month is four million a year and is the month-twelve test; fifty thousand teams is a hundred million; and unlike seats, the meter reprices itself upward every time a model gets good enough to be trusted with more.

The single hardest push

You have now done this twice, and you should notice the pattern before I say it out loud.

Round two, your entire company was one dataset: the negative class. You were told to count it. You counted it, honestly, at some cost — 262 sessions, 15,642 tool calls, anchored matching, Wilson intervals, a caveat section admitting your own naive greps returned 229 false positives. The answer was 22. Twenty-two rejections, all of them either shell commands, web searches, or Markdown edits, none of them touching code. Zero stashes on the whole machine. 91.6% of sessions with no negative event at all. Your moat's raw material was, in your own words, roughly one four-hundred-and-sixtieth of what you needed. So you moved the company to rework.

Now do to rework what you did to rejection.

Human commits in your own repos are reworked at 38% founder-only and 51% pooled. Appendix D's seven-day control base rate is 64%. Appendix A caveat 2 says reverse-blame is line-identity and not semantics — a reformat, a rename -M missed, a whitespace change all count. Appendix C caveat 10 says it "also captures normal iteration on active files." So the metric you have moved the company onto has a floor somewhere around a coin flip and a definition that cannot tell a bug fix from a prettier run. Your entire spread is the twenty-two points above that floor. And in three of the eight repos where you measured both classes — including both of your two largest, which supply 65% of your agent commits — agent work is reworked less than human work. The delta across your eight repos runs from minus twenty to plus thirty-five. Not one of them lands in the "15 to 22" band your memo reports.

So think about what happens on the day the wedge works exactly as designed. A stranger from the watering hole runs panout init. It prints their rework map. There is a real chance — on your own data, three in eight — that the table says their agent commits are safer than their handwritten ones. The hook is dead in the first minute, the two worst task classes it nominates are noise, and the two contracts you propose are pointed at whatever direction the whitespace happened to fall. You cannot fix that with a better memo. It is a property of the metric, and you have not yet built the version of the metric that survives it: rework restricted to changes that follow a failing check, or touch a test, or come from a different session, or survive a whitespace-and-rename filter. That is one day of work on data you already have, and it is the difference between a demo and a coin flip.

Here is the uncomfortable read on rounds two and three together. Both times, the load-bearing quantity was unmeasured. Both times you were told to measure it. Both times you did — well, and honestly, and faster than most founders would. And both times the answer was no, and you responded by relocating the company to the next unmeasured quantity. Rejection was measured-and-tiny. Rework is measured-and-noisy. The next one will be measured-and-something too, and you will have spent another twelve days and another appendix finding out.

The pattern is not a data problem. Every one of these measurements was run against your own machine, at N=1, on a founder who has 33 sessions in the Panout repo and zero commits, whose session logs contain no stashes and no rejections because he is not actually experiencing the problem the product treats. You are the wrong instrument. You are debugging the market by instrumenting yourself, and you are the least representative agent user on earth: top-thousandth-of-a-percent volume, 85% agent authorship in team repos where you are a minority committer, everything in bypassPermissions, and no external repo in any table you have produced. Twelve days ago you were told to get two numbers and a yes/no from ten humans. Nine hours of work. You have had two hundred and eighty-eight.

You are extremely good at finding out that you are wrong. Now find out from someone else.

Every remaining objection keeping this below 90, in priority order

1. Zero external humans. The ten operators, one question, in writing, are twelve days outstanding — and this memo says so in its second paragraph, which does not discharge it. ~9 points. Evidence only. Ten fleet operators from the watering hole, in writing: how many agent-authored diffs did you merge last week without reading them, and did any of them bite you. Two numbers and a yes/no per person, verbatim in the repo. The company exists only inside the band where operators merge unread and get burned, and nobody on earth has published that band's width. No memo change touches this dimension. Q1, Q3 and founder fit are all capped by it, which is why it is worth nine points on its own.

2. The rework label is not a defect label, and the minute-one hook can print backwards. ~8 points, of which ~3 are reachable by idea change alone. Idea fix, exactly: stop selling the level and sell the gap; and build the restricted metric before the wedge ships — rework that (a) follows a failing check, or (b) touches a test file, or (c) originates from a different session than the original commit, or (d) survives a whitespace-and-rename filter. Then report, on the same 2,859 commits you already have, whether the restricted metric (i) preserves the agent-human gap and (ii) stops reversing sign in the team repos. That is one day, no users, no permission. The remaining ~5 points are evidence: the restricted metric printed on three strangers' repos, with the stranger confirming its two worst classes are the two they would have named. Face validity from someone who is not you is the whole test.

3. The moat's exclusive feature set is smaller than the memo claims. ~6 points, ~4 reachable by memo change. Memo fix, exactly: concede in the memo that task class, harness and model are recoverable from git trailers and commit shape — appendix A proves it — and rest the moat on the three things that genuinely are not: session shape, contract results, and the one asset you have not yet claimed and should. The override record. "Committing after a printed failure is an override and is recorded as one" is a human judgment about a specific failure on a specific diff, it exists in no vendor's logs, no git history, and no CI system, and unlike rejection it is generated by the thing the user already does. That is your negative class, it is priced at zero for you to collect, and it is buried in step 3 of your wedge instead of being layer 1 of your moat. The remaining ~2 points are evidence: an override count from a real repo showing the rate is non-trivial.

4. The predictive delta is now provably unmeasurable until 2026-10-05 at the earliest. ~5 points. Evidence only, and it has a hard floor. Appendix D is honest and correct: 21-day retention against a 30-day label means retrospective joining is impossible for anyone, and squash-merge severs 100% of SHA linkage in your largest repo. Which means the earliest date this objection can close is thirty days after commit-time capture ships — and capture has not shipped. Every day of delay moves the earliest possible 90 back a day. Requirement when it lands: ≥300 commit-time-captured commits across ≥3 repos, git-only AUC versus git-plus-commit-time-features AUC, with a bootstrap CI on the delta and both outcome classes present.

5. Founder fit: the revealed behavior is now a twelve-day pattern documented in your own appendix. ~5 points. Evidence only. 33 sessions and 718 tool calls in the Panout repo, zero commits in eleven days, four documents, three fourteen-day plans, two outstanding assignments. Nothing in a memo changes this. One shipped thing plus one contacted human changes it entirely.

6. The demand that exists is for a thing incumbents ship free; the demand for your thing is the leg your own appendix calls weakest. ~4 points, ~2 by memo. Memo fix, exactly: say plainly that Panout is not competing for the auto-approve ask — Claude Code Auto mode, chat.tools.terminal.autoApprove and Codex execpolicy have that ask, for free — and that your buyer is the person who already took auto-approve and now owns the consequence. Then name the bridge in one sentence: 14,848 shell scripts and 40%-of-sessions auto-approve is the size of the population that has already made the trade, and none of them can tell you what it cost. ~2 points are evidence: one named operator saying the consequence bit them.

7. Evidence-integrity residue. ~3 points. All five are memo fixes, exactly: (i) Replace "2,800 in 180 days across founder-controlled repos" with the figure that ties — either the ~5,200 founder-authored non-merge commits git log actually returns across the nine repos, or drop the parenthetical. As written it ties to no appendix line and the nearest number, 2,859, is a sample size for a different population. (ii) Replace "15 to 22 points more often than human work in the same repos" with the true statement: "the founder-only gap is 22 points and the pooled gap is 16, and per-repo the gap runs from −20 to +35 and reverses in the two largest." Your actual sentence contains none of the eight measured values. (iii) Add one clause reconciling C's 22/25 with D's 53/31 and say which method you trust and why. A partner reading both appendices finds the discrepancy, and the number you used is the one that helps you. (iv) Carry appendix C caveat 7 into the memo: the "no rejection touched code" claim rests on 4 of 22 recoverable paths. (v) Delete "460 times" and the "reject 0.14% … rework 60%" juxtaposition, or restate both on a common denominator. Per-tool-call and per-commit rates do not divide.

8. $1B still requires 100% of its own reference population at the $100M rung. ~2 points. Memo fix: build the 2028 denominator from something you measured — Codex 3M+ weekly active developers, Claude Code's ~4% of public commits, Copilot 26M users — divided by a stated team size, rather than from Cursor's secondary-sourced 50,000 paying teams. Then state the penetration rate each rung implies. Right now the ramp's top rung is a coincidence of numbers, not a market.

9. The metered unit is a check the customer's CI already runs. ~2 points. Idea fix: be explicit that the meter prices the policy — the map, the autonomy level, the calibration — and that evaluation is the billing event because it is countable, not because it is the deliverable. Otherwise the first CFO who looks at it asks why running pytest costs thirty-three cents.

10. The day-15-to-45 kill criterion fires before its own evidence can exist. ~2 points. Memo fix: "Kill if no task class earns level 1" requires 30-day-old commit-time-captured commits in that class. By your own timeline that is 2026-10-05 at the earliest, and day 45 of a plan starting this week is roughly 2026-10-18 — a two-week margin on a clock you have already let slip once. Re-date the autonomy milestone to day 60, restate the day-45 test as "contract catch rates separate across contracts," and say in the memo why you moved it. You have re-dated one gate already; do not let a second one drift silently.

The two questions, answered plainly

What is the maximum this idea can reach with memo/idea changes alone, and no new external evidence?

74 to 76. The reachable movement is: evidence integrity 7→9 (all five residues are mechanical fixes), moat 6→7 (concede the git-derivable features, promote the override record to layer 1), $1B 6→7 (a denominator you actually measured), and Q2 7→8 (the restricted rework metric, which is one day on data you already hold and technically counts as an idea change since the data exists). That is +5 raw, plus the same +2 gut. Nothing above 76 is reachable, and the reason is structural rather than rhetorical: 90 means I state the moat and the $1B path to a partner in two sentences without hedging, and both sentences currently end in an unmeasured predictive claim and a market with zero humans in it. No arrangement of words fixes a sentence whose verb is "explains" when your own appendix says the measurement does not exist.

What is the minimum external evidence that would take it to 90?

Four items, and I mean minimum — this is not a wish list.

1. Ten operators, in writing, two numbers and a yes/no, with at least three reporting that they merged unread and got bitten. Two weeks. Prices Q1, Q3 and the entire market.
2. Three external repos, rework map printed, stranger's reaction recorded — specifically, whether the map's two worst task classes are the two they would have named unprompted. This tests the metric's face validity on data that is not yours, which is currently the single largest unhedged assumption in the memo. Two weeks, and item 1 is its delivery vehicle.
3. The predictive delta, on the schedule physics allows. ≥300 commit-time-captured commits across ≥3 repos, git-only versus git-plus-commit-time-features, AUC with a bootstrap CI on the delta, both outcome classes present. Earliest possible: thirty days after capture ships. If capture ships this week, 2026-10-05. If it ships in three weeks, late October, and the 90 moves with it.
4. One of those three operators paying the meter, at any price, without being asked twice. Not an LOI, not a waitlist, not "that's interesting." A charge that cleared.

Items 1 and 2 are fourteen days. Item 3 has a thirty-day floor that starts the day you ship. Item 4 follows from 1–3 or does not. The earliest date this reaches 90 is roughly 2026-10-10, and only if the wedge ships this week. Every day you do not ship, that date moves one day right, and your runway does not.

One assignment for this week

Take the rework map — which needs no install, no login, and no consent beyond a git clone, because appendix A already proves it runs on thirteen repos — point it at ten public repos with heavy agent commit traffic, and send each maintainer their own table with one question attached: "does this match what you'd have guessed, and did any of it bite you?"

That is one action and it discharges three obligations at once. It forces you to ship the thing you say is 48 hours of work, because you cannot send a table you have not generated. It puts your metric in front of ten strangers' data before you build a company on it — which is the only way to find out whether it prints backwards. And it is the ten-operator conversation, arriving as a gift rather than a pitch, which is the version most likely to get answered.

Ten repos are in your appendices already or one gh query away: head:codex/, author:app/copilot-swe-agent, "Co-Authored-By: Claude" — 5.8 million, 1.5 million, and 346 thousand merged PRs respectively, and you measured all three yourself today. You do not need permission to clone a public repo and you do not need permission to email its maintainer.

Two things this does not replace, and they do not get renegotiated into something more interesting.

Commit-time capture ships this week, in your own repos, today if possible. It is the only clock in this company that cannot be compressed. Thirty days is thirty days whether you start it on Thursday or on the fifteenth, and the earliest honest predictive number — the one that decides whether you have a rating layer or a dashboard with a good story — is exactly thirty days after the first captured commit. Everything else in your plan is reversible. This one is a one-way door on the calendar, and it is the only decision this week that deserves to be made fast.

And write down what the restricted rework metric does to your own numbers before you show the map to anyone. Rework that follows a failing check, or touches a test, or comes from a different session, or survives a whitespace-and-rename filter. One day. If the twenty-two-point gap survives all four filters and stops reversing in your team repos, your hook is real and your memo gets ten points better on its own. If it does not survive, you have learned in a day — for the third time, and it will still have been worth it — that the quantity you built the company on is not there.

You have found out you were wrong twice in twelve days, fast and honestly, on your own data. That is a genuinely rare capability and I am not going to pretend it is nothing. But you have now exhausted what your own machine can tell you. There is not a third measurement on that laptop that changes my answer. There are ten humans, and nine hours, and they have been sitting there for two weeks.