Skip to content

Commit 88ab86c

Browse files
authored
Merge pull request #3 from BackendStack21/fix/website-ratio-consistency
fix: reconcile cost/ratio figures across site, README, and protocol
2 parents 39aacbe + 9ce7a45 commit 88ab86c

3 files changed

Lines changed: 8 additions & 8 deletions

File tree

README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,7 @@
1010

1111
## The Problem
1212

13-
AI generates code 16,000× cheaper than we can verify it. Generation costs dropped 100–150× since 2023. Human review is still capped at ~200 LOC/hour. The **Cv/Ci ratio** — cost-to-verify divided by cost-to-implement — has exploded from ~33:1 to **~3,300:1**, and it's degrading exponentially.
13+
AI generates code 3,300× cheaper than we can verify it. Per-PR generation cost dropped ~20× since 2023 (raw token prices fell ~200×). Human review is still capped at ~200 LOC/hour. The **Cv/Ci ratio** — cost-to-verify divided by cost-to-implement — has exploded from ~167:1 to **~3,300:1**, and it's degrading exponentially.
1414

1515
## The Answer
1616

ai-verification-protocol.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -64,7 +64,7 @@ The protocol is versioned. **v3.0** added Active Repair Mode. **v3.1** hardened
6464

6565
## 0. Operating Context
6666

67-
Generation costs have collapsed 100-150x. Verification costs have not budged. The industry is shipping code 10,000x faster than humans can review it, and the **verification gap** — the fraction of generated code receiving no meaningful verification before production — compounds daily.
67+
Per-PR generation cost has collapsed ~20x (raw token prices fell ~200x). Verification costs have not budged. The industry is shipping code 10,000x faster than humans can review it, and the **verification gap** — the fraction of generated code receiving no meaningful verification before production — compounds daily.
6868

6969
This protocol operates as a **multi-agent pipeline**, not a single reviewer. The whitepaper's central correlated-failure warning forbids relying on any one model to verify another model's output unaided. Roles must be played by distinct model instances, ideally from distinct provider families.
7070

index.html

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -399,7 +399,7 @@ <h1>The AI <span>Verification</span> Protocol</h1>
399399
<div class="cert-metric"><div class="m-val">890:1</div><div class="m-label">Cv / Ci</div></div>
400400
<div class="cert-metric"><div class="m-val">4.1h</div><div class="m-label">Δ Debt</div></div>
401401
</div>
402-
<div class="cert-reason"><strong>Why:</strong> axis 2.3 (security surface) 🔴 — new crypto code with 3 SAST-high findings. PR size cap triggered (647 LOC in security path). Certificate identifies exact files and lines for human auditor to inspect.</div>
402+
<div class="cert-reason"><strong>Why:</strong> axis 2.3 (security surface) 🔴 — new crypto code with 3 SAST-high findings. η 0.71 sits below the 0.80 auto-threshold, and a 🔴 security axis forces Human Review Required. Certificate identifies exact files and lines for human auditor to inspect.</div>
403403
</div>
404404
</div>
405405

@@ -436,9 +436,9 @@ <h1>The AI <span>Verification</span> Protocol</h1>
436436
<section>
437437
<div class="section-inner">
438438
<div class="section-label">The Problem</div>
439-
<h2 class="section-title">AI generates code 16,000&times; cheaper than we can verify it</h2>
439+
<h2 class="section-title">AI generates code 3,300&times; cheaper than we can verify it</h2>
440440
<p class="section-desc">
441-
In 2023, generating a 200-line PR cost $0.30 and verifying it cost ~$50. Today, the same PR costs $0.015 to generate — but verification still costs ~$50. The ratio has exploded from 33:1 to over 3,300:1, and it's degrading exponentially.
441+
In 2023, generating a 200-line PR cost $0.30 and verifying it cost ~$50. Today, the same PR costs $0.015 to generate — but verification still costs ~$50. The ratio has exploded from 167:1 to over 3,300:1, and it's degrading exponentially.
442442
</p>
443443
<div class="formula-block">
444444
<code>Cv / Ci &rarr; &infin;</code>
@@ -487,9 +487,9 @@ <h2 class="section-title">Why is AI verification debt the defining problem of th
487487

488488
<div class="why-item">
489489
<div class="why-num">01</div>
490-
<h3>Generation costs collapsed 100–150&times;. Verification costs haven't budged.</h3>
490+
<h3>Generation costs collapsed ~20&times;. Verification costs haven't budged.</h3>
491491
<p>
492-
In March 2023, GPT-4 cost $30/$60 per 1M tokens. By May 2026, DeepSeek V4-Flash costs $0.14/$0.28 — two orders of magnitude cheaper. But human review is still capped at ~200 lines of code per hour, the same sustainable rate documented by SmartBear and Cisco since 2006. The bottleneck shifted from generation to verification overnight.
492+
In March 2023, GPT-4 cost $30/$60 per 1M tokens. By May 2026, DeepSeek V4-Flash costs $0.14/$0.28 — token prices fell roughly two orders of magnitude, yet per-PR generation cost dropped ~20&times; as agents grew more token-hungry. But human review is still capped at ~200 lines of code per hour, the same sustainable rate documented by SmartBear and Cisco since 2006. The bottleneck shifted from generation to verification overnight.
493493
</p>
494494
<div class="cite">
495495
<strong>The AI Verification Debt</strong> &mdash; "The industry is celebrating the collapse of generation cost. That is half of the story. [...] Google's engineering practices recommend changes under 200–400 LOC for thorough review. The SmartBear/Cisco study found that reviewing 200–400 LOC/hour is the sustainable human rate for detecting meaningful defects."
@@ -533,7 +533,7 @@ <h3>The components exist — they're just not wired together.</h3>
533533
<div class="why-num">05</div>
534534
<h3>The Cv/Ci ratio is already ~3,300:1 and degrading exponentially.</h3>
535535
<p>
536-
Projecting forward: if the price war between US and Chinese labs continues and inference costs drop another 10&times;, the ratio hits ~16,700:1. The team that invests in verification infrastructure can safely ship 10&times; more code than the team that invests in better generation alone. The team that does not — will drown in verification debt within this decade.
536+
Projecting forward: if the price war between US and Chinese labs continues and inference costs drop another 10&times;, the ratio hits ~33,000:1. The team that invests in verification infrastructure can safely ship 10&times; more code than the team that invests in better generation alone. The team that does not — will drown in verification debt within this decade.
537537
</p>
538538
<div class="cite">
539539
<strong>The AI Verification Debt</strong> &mdash; "The trap is not that the code is bad. The trap is that <em>nobody knows how bad it is</em> until the verification debt compounds. [...] The team that invests in verification infrastructure can safely ship 10&times; more code than the team that invests in better generation alone."

0 commit comments

Comments
 (0)