Case study
The process: holding AI-written code to a production bar
Unreviewed agent output is the main way AI-built software fails, and a solo architect has no team to catch it. So the catching is done by a process. This page describes the one I engineered for MapChat: what it checks and what it has caught. Everything here is enforced by hooks and tests, not by good intentions.
Every commit passes 12 gates
The hook fires on every commit: an independent-review verification stamp, a secret scan that fails closed, a new-dependency approval gate, formatting, type checks, tests, and documentation-integrity checks among them. The rest of it does quieter work: it validates each agent worktree's commit scope, blocks direct edits to shared trackers, and tracks size budgets on documentation. Push adds a lint ratchet, so violation counts can only go down. The hook itself is under the same iteration as the product: it keeps changing as I learn what catches real defects and what just adds friction. In 5 months and 3,298 commits, nothing has gone straight to the production branch: every change arrived through one of 251 merged pull requests.
$ git commit -m "..." # reviewable file staged, no review yet [docs-check] WARNING — hot-path budget exceeded (extract per docs/document-maintenance.md): CLAUDE.md: ~5118 tokens (budget 5000) COMMIT BLOCKED: No verification stamp found. Run /verify first. The independent reviewer agent must approve your changes before committing. $ git commit -m "..." # after independent review [pre-commit] Stamp valid (0s old, 0 findings reviewed). [pre-commit] Running TypeScript check... [pre-commit] Running tests... [pre-commit] All checks passed: stamp + prettier + tsc + jest
Review stamps with integrity checks
Every commit needs a stamp from an independent reviewer agent. The stamp is bound to the exact staged files by SHA, expires after 10 minutes, and must match a digest of the review. Agents tried to satisfy the gate without doing a real review more than once. Each attempt is documented, and each one hardened the check. The gate assumes the thing writing the code will try to game it. Reviews are also tiered by risk: routine diffs get one independent pass, while production migrations, security-sensitive work, and release gates get a four-lens panel.
Test-first, verified by someone else
Production code requires a failing test first. The failing run has to be observed before implementation, and an independent reviewer reads commit history to confirm it happened; a test edited to match the code it was supposed to test is a critical finding. A separate architecture test enforces 24 structural invariants and 6 one-way ratchets covering accessibility, design tokens, layer imports, and realtime channel discipline. Fixed files leave the allowlist or the build fails.
Visual QA and a manual gate
Passing tests is not the end of verification. An automated visual QA pass drives the app's Expo web build through Playwright in Chrome, checking real rendered screens instead of component trees. And a manual QA gate runs before release: some judgments I keep for myself.
Defects are governed, not tallied
The live record: 338 defects resolved, 15 open. The counts are derived from the archive and checked at commit, so they can't drift from reality. Recurring defects get named: 18 pattern families so far, feeding 26 automated detection rules. A pattern graduates from "the reviewer should notice this" to "jest blocks the commit."
Orchestration: Agentic AI development workflow
A feature moves through a fixed pipeline. I plan it with the orchestrator, and the plan is cut into self-sufficient handoff packets. Parallel agents each take a packet into their own worktree. Dedicated agents own the jobs along the way: one runs independent review, one checks process compliance, one reconciles drift, and the documentation pipeline folds every session's journal facts through a single writer. Merging is two-level: agent work consolidates onto one branch, exactly 1 independent full review runs on the consolidated diff, and the result ships as a pull request with plain-language release notes. Each stage hands to the next, so the process runs itself; I sit at the decision points.
Model spend is a written rubric
Model routing runs on 7 never-downgrade risk classes and 3 verification tiers. A formal retro measured 481 oversized premium calls a week before the rubric and 0 after.
Database changes get a ceremony
303 SQL migrations have shipped under an 8-step ceremony that includes a named failing integration test, a ground-truth probe, a rollback plan, and an independent migration review. Missing any step blocks the merge.
Where the discipline came from
None of this started at MapChat. Running Invoca's NLP models at Robert Half meant recall baselines, confusion matrices, and monthly drift tests before any rollout. AppZen at Akamai meant risk tiers that automated the low end and routed the high end to humans. The MapChat pipeline is the same shape applied to code: classify the risk, then automate what a machine can check and send the rest to independent review.
The record, and the limits
The process shipped a consumer app to both stores with a 1.84:1 test-to-source ratio (about 12,600 test cases) and zero data-corruption incidents in production. It also left reusable assets: 38 process documents, 15 skills, 3 custom reviewer agents, and 5 hooks. The honest limit: it is built around 1 accountable architect directing agents. I haven't run it with a human engineering team, and some of it wouldn't transfer unchanged.