Two posts ago, R and D were two roles, two rulebooks, one person switching hats. Last time was D’s story: the loop that made shipping safe. By now the two don’t split cleanly — every decision and every build happen in the same conversation. This round adds a third: QA, an external auditor with no stake in the code already being good, brought in specifically to check R and D’s work from the outside.

Getting something to alpha proves it works. That’s not the same question as beta. Beta is four separate things you have to prove, honestly, about the same piece of software:

  • Does it hold up to scrutiny you didn’t supply yourself?
  • Can it get more capable without spending down the trust its first design earned?
  • Does it survive contact with real use — not just the tests you thought to write — and is it actually pleasant to use?
  • Can people rely on explicit promises about what will and won’t change under them?

That’s the checklist. Here’s how it played out over the following two weeks, building Omnist, a schema library I work on in my own time. The point isn’t Omnist — it’s the moves. Omnist just supplies the proof they’re necessary.

  1. Invite an outside review, and argue back where it’s wrong.
  2. Add capability without spending down what makes you trustworthy.
  3. Re-prove your core guarantees survived the change.
  4. Stress-test it against the real world, not just your own tests.
  5. Make it easier to use without touching what makes it correct.
  6. Make explicit, durable promises about what won’t change.
  7. Keep the habit running after the deadline is gone.

Step 1: Invite an outside review, and argue back where it’s wrong

Self-review has a blind spot, and it’s a specific, well-known one: the selective-attention effect where you miss what you weren’t looking for, even when it’s right in front of you. The AI that wrote the code has the same blind spot as any author — its own assumptions are invisible to it precisely because they’re its assumptions. It can’t flag its own hallucinations as hallucinations, and it can’t see its own style as a choice rather than just “how the code looks.”

So bring in scrutiny you don’t control — a reviewer with no stake in the code already being good, and no shared blind spots with the one who wrote it. That’s QA’s whole job: a different AI, not R or D, brought in to audit Omnist specifically to catch what the builder couldn’t see about its own work — confidently invented claims, code that worked but read badly, patterns the original author was too close to question. Expect real findings mixed with noise, and expect some of the real findings to be inconvenient — QA’s first pass came back with a verdict of “production ready.” It wasn’t, not yet, and I said so. Same rule this whole series runs on, going back to the first post: disagreement owes a reason, not a shrug.

Then triage before you fix. Whatever’s loudest first is how you end up chasing noise instead of what matters. A tracked queue, not a reflex. A few real fixes came out of Omnist’s: safer behavior on bad input instead of a silent fallback, clearer error messages instead of a loose string, dependencies pinned instead of left floating.

Step 2: Add capability without spending down what makes you trustworthy

At some point your first design won’t cover a case real users actually have. The instinct is to loosen the rule everywhere it might be needed. Resist that. Loosen it in exactly one place, explicitly marked, never automatic. The value of a strict design is what it lets you promise — give that up broadly and you’ve quietly downgraded the whole thing to best-effort.

Omnist’s schema model started fully closed: every field exactly one type, no exceptions. That’s what let it answer real questions — is this change safe, are these two schemas the same — with an actual answer instead of a guess. Real data didn’t always fit. The fix wasn’t loosening everywhere; it was one narrow, explicit escape hatch (what the project calls any), usable only where marked, with its cost written down before any code changed.

Step 3: Re-prove your core guarantees survived the change

Any time you loosen a strict system, you owe yourself proof the strictness that’s left still works. Not a hope — an actual re-check, done by someone who didn’t write the change. That’s the same never-self-verify rule from the loop that made shipping safe, just applied to a riskier kind of change. Ship the loosening in pieces small enough that each one can be verified before the next goes out.

Done properly, that check isn’t a formality. On Omnist it caught a real infrastructure bug early, and later something subtler: a test that had quietly stopped testing what it claimed to, after an unrelated change shifted what it actually exercised. It still passed. Only someone checking the real behavior — not trusting the test’s own claim about itself — caught it.

Step 4: Stress-test it against the real world, not just your own tests

Your own test suite only covers what you thought to check. The harder, more honest test is real use you didn’t design for: real files from outside your own project, not synthetic cases built to be easy.

Running Omnist against real external file formats surfaced things self-testing never would have. One format had no real specification behind it at all, which made everything checked against it weaker than the field count alone would suggest. An unrelated parsing rule broke real files outright, for a reason that had nothing to do with the logic being tested. And the single most rigorously specified format found a limit none of the messier ones could: checking that a value has the right type isn’t the same as checking it’s in the right range. Four honest limits, worth writing up as guidance afterward instead of leaving as one-off surprises.

Step 5: Make it easier to use without touching what makes it correct

Correctness earns trust. Ease of use earns adoption. You need both, and they shouldn’t be the same piece of work — a usability improvement should ship without anyone having to re-verify the model underneath it.

Omnist got what I’ve been calling array sugar: shorthand for a pattern that used to take several lines to write out, readable at a glance instead of parsed line by line. Pure convenience — it expands and collapses automatically, and changes nothing about what the data means underneath. Even something this small went through the same discipline as everything else: spec first, a failing test before the fix, independent review after. That review still found something, not in the feature itself but in the seam between two people’s pieces of it — exactly the kind of gap the discipline exists to catch.

Step 6: Make explicit, durable promises about what won’t change

Eventually you have to tell people, in writing, what they can rely on and what they can’t. That’s a different kind of document than a bug fix. It’s a public commitment, and getting it wrong is worse than not having one — now people are relying on something imprecise.

Before Omnist could honestly call itself beta, one real bug had to go: malformed input crashing instead of failing with a clear message, on exactly one side of an otherwise-guarded path. And a written policy had to exist — what changes only with warning, what carries no promise at all. Both went through the same review-before-merge discipline as everything else; the policy in particular got a paragraph rewritten after review found it blurring two things that needed to stay separate. The version number moved only after every claim it would make had already been checked.

Step 7: Keep the habit running after the deadline is gone

The label changing doesn’t end the work, it just removes the excuse. Real discipline shows in what happens once nobody’s checking whether you kept it up. Small releases should keep moving through the same loop that got you to the milestone, with nothing shortcut just because the finish line is behind you now.

Right after Omnist reached beta, QA came back for a second pass, and got the exact same treatment as the first: keep what’s real, argue back on what’s not, queue the rest instead of reacting to it. One of the resulting calls is itself worth naming as good practice — a proposed test-quality check got shelved rather than shipped, because an early run produced numbers too unreliable to trust. Publishing an unreliable number would have been worse than publishing nothing.

What actually changed

Each of the four questions from the top got answered honestly, not assumed. QA found real problems and got argued with where it was wrong. New capability shipped without loosening the guarantee that made the thing worth trusting. Real-world use found actual limits, not hypothetical ones. And a written promise now exists about what changes and what doesn’t.

That’s the real difference between a demo and production software — not a milestone, but a habit that keeps holding even after nobody’s checking whether it still does.