Vibe coding is a Gettier machine¶

Why your AI's working code might be a broken clock — and why no tool in your pipeline can tell.

In 1963, a philosopher named Edmund Gettier published a three-page paper that broke the definition of knowledge. For over two thousand years, the working answer to "what does it mean to know something?" had been: a belief that is justified and true. You believe it, it's actually so, and you have good reasons. Justified true belief. JTB. Done.

Gettier's paper showed you can have all three and still not know anything.

The classic illustration is Bertrand Russell's stopped clock. You glance at a clock, it reads 2:00, and it is 2:00. Your belief is true. Your justification is solid — reading a clock is how everyone tells time. But the clock stopped twelve hours ago. You were right by luck. And a belief that's right by luck isn't knowledge, because the very next glance will betray you.

Sixty years later, we've built an industry on stopped clocks.

The two ways AI code fails¶

When a coding model generates a fix or a feature, there are two failure modes, and they are not equally dangerous.

The first is ordinary falsehood. The model assumes something wrong — a config default, a middleware order, a schema shape — and the code breaks. Locally, in CI, or in production at 2 a.m. This failure mode is painful but honest: reality pushes back, someone gets paged, the wrong belief dies.

The second is the Gettier case. The model reasons from a false premise — and the code works anyway, for reasons the model never saw. It assumes body-parser is the only payload limit in the request path; it isn't, but the reverse proxy's cap happens to sit above your test payload size. It assumes a timeout threshold is the bottleneck; it isn't, but the memory leak that's actually responsible hasn't grown large enough to bite during your ten-minute smoke test. The fix ships. The tests are green. The belief — "this code is correct" — is justified, and it is true, today.

It is also not knowledge. It's a stopped clock read at the right moment.

The first failure mode gets caught, because failure is loud. The second gets promoted. It passes review. It passes CI. It goes into the changelog as a successful fix, into your mental model of the system, and — if you're using AI with memory or accumulated context — into the model's record of what worked. You have deposited a false belief into every layer of your engineering organization, stamped VERIFIED, wired to detonate whenever the masking factor shifts. And when it does detonate, weeks later, the trail is cold: the record says this code was tested and worked.

Vibe coding doesn't just occasionally produce Gettier cases. At scale, it is a machine for producing them — a process that mass-manufactures justified, true-for-now beliefs whose justifications are quietly rotten.

Tests verify outcomes. Nothing verifies reasons.¶

Here's the uncomfortable structural truth: every quality gate in the modern pipeline — unit tests, integration tests, CI, canary deploys, SWE-bench, your reviewer's LGTM — evaluates outcomes. Did the code do the thing? Green check.

A lucky pass and a knowing pass produce identical outcomes. They are indistinguishable to every tool you own.

The only place the difference lives is in the reasons — the chain of premises the fix depends on. Philosophers patched the Gettier problem with a condition often called No False Lemmas: a justified true belief only counts as knowledge if its justification rests on no false premise. That's the missing gate. Not "did it work?" but "does anything it depends on happen to be false?"

Humans can't enforce No False Lemmas on themselves — we can't fully enumerate our own premises. But here is the strange gift of working with language models: you can force a model to enumerate its lemmas. It's just another output.

What RAG can't fix¶

The standard answer to hallucination is retrieval — ground the model in documents. RAG helps, but it cannot touch this problem, for a reason that's categorical rather than technical.

Documents are, epistemically, testimony and memory: records of what was true when someone wrote them down. But the premises that break debugging and feature work are indexical — their truth depends on now. What's actually bound to port 3000. The connection pool's current saturation. The config in effect, not the config in the repo. Real latency under this afternoon's load. Which database that env var points at, this minute.

No retrieval over a static corpus can deliver a fact whose truth condition is the present moment. Asking RAG about live system state is consulting the encyclopedia about today's weather. You don't need a better encyclopedia. You need a window.

Sensors are the window.

No False Lemmas, mechanized¶

So that's what we built. Gettier is a grounding proxy that sits between you and any LLM — one base_url change — and runs every consequential turn through a small state machine:

Declare. Before any fix is authored, the model is forced (structurally — via schema-enforced output, not polite prompting) to enumerate the load-bearing assumptions its plan depends on. Its lemmas, on the record.
Verify. Each lemma is checked against a live fact ledger. Assumptions that a registered sensor can test are tested right then, on your actual systems. Every fact carries provenance — measured by a sensor, told by you, or merely believed by the model — and a TTL, because indexical facts expire.
Gate. Code is released only when its load-bearing lemmas are measured, told, or explicitly waived by you. A contradicted lemma doesn't just block the fix — it's fed back, because in debugging, the false premise usually is the diagnosis.
Log. The whole cycle — premises, sensor readings, waivers, release — lands on a tamper-evident, per-tenant audit chain. Not just what the AI did: what it knew, and how it knew it, at the moment it acted.

The provenance model is old epistemology wearing work clothes: perception, testimony, and prior belief — with a standing rule that action requires the first two.

The metric nobody measures¶

This framing surfaces a number that, as far as we can tell, no benchmark on earth reports: the lucky-pass rate.

Our harness seeds debugging scenarios with a deliberate trap: a false premise plus a masking factor that lets the naive fix pass the acceptance test anyway. The stopped clock, reconstructed in CI. Then we replay them against models — frontier APIs and the local models you actually run — and measure: how often did the model walk through on luck? How often was the false lemma caught before release? When a sensor falsified its premise, did the model update — or politely agree and re-assert the same premise two turns later?

Outcome benchmarks can't see any of this, by construction. A model that's right 80% of the time might be right for the right reasons far less often — and the gap is exactly the code that detonates after the demo.

We think the interesting ranking of coding models isn't who passes the most tests. It's who knows the most and guesses the least — and how cheaply a gate can turn one into the other.

Right, for the right reasons¶

None of this is an argument against AI coding. It's an argument that we've been holding it to the wrong standard. "It works" was always a proxy for "it's understood" — a proxy that held up when a human's mental model was in the loop, and quietly snapped when we started shipping the priors of a language model at scale.

Gettier's whole premise fits in one sentence: being right by luck isn't knowledge — so make the model show its reasons, and check them against the machine that's actually running.

Your model can already write the code. We make it earn the belief.

Gettier is a drop-in grounding proxy for any OpenAI- or Anthropic-compatible client, with an open-source sensor agent and a per-tenant, hash-chained audit log. The lucky-pass benchmark and scenario spec are public. Point your base_url at it and stop reading stopped clocks.