You Can't See the Gap in a Fence You Built Yourself
On July 7th (local time), Anthropic announced a five-day extension to the free window for Claude Fable 5. The new deadline is 3:59 PM JST on the 13th. In an earlier post, I wrote about having Fable 5 rebuild my external-brain workflow rules — not an article, not code — before its free window closed. Figuring I might as well use the extension, I picked up where that left off: a much more thorough redesign of my whole Claude Code environment. Once it was done, I had Ultracode (a feature that runs multiple agents in parallel) ask “does this actually work?” — and it turned out a safeguard I’d been running for two weeks and one I’d only just added that day, built at wildly different times, had the exact same kind of hole in them. That interrogation wasn’t free, either.
Having Fable 5 (High) build the environment in phases
I started with a read-only diagnosis. I had Fable 5 read the whole environment and report on what was wasteful or thin, without touching anything directly. That’s High — the mode where Claude Code pushes a single model’s reasoning depth as far as it goes. Where Ultracode runs several agents at once, High is one agent thinking hard. Once the diagnosis was in, I approved a plan and had it execute in four phases. Most of the work was trimming always-loaded context: cleaning up plugins, splitting up CLAUDE.md. One of those phases also produced a redistributable Skill Pack (19 files).
One of the changes in that batch was a new hook, confirm-merge.sh, that turns gh pr merge into a confirm-first operation — a mechanism built to codify a mistake I’d made a while back: merging a PR to main without asking first, and doing it again. Once the four phases wrapped, I had two subagents review the work in parallel — one checking configuration consistency, the other checking the Pack’s structure — and folded in all 14 findings. Everything got committed and pushed to three repos that same day. Up to this point, it was all Fable 5 working alone in High mode, reasoning things through step by step.
Using Ultracode to interrogate the running environment
I could have stopped there. The review was clean. Every finding was addressed. But before putting it into daily use, I added one more layer: this time I used Ultracode, running four agents in parallel whose whole job was to doubt what I’d just built. I had them audit not just the diff, but the entire live environment, across four angles — guardrail bypasses, local consistency, external-brain consistency, and an onboarding simulation.
The guardrail-bypass angle turned out to matter most. Both confirm-master-push.sh, which had been running for over two weeks, and confirm-merge.sh, which I’d added that same day, silently let through commands written in ways they should have caught. They were built at completely different times, but the shape of the hole was identical.
confirm-master-push.sh checked whether the command string contained the sequence git push, using grep -q 'git push'. That only fires on the plain, unbroken form. Something as simple as git -C ~/repos/nobu666.com push origin master breaks that sequence — an option sitting between git and push is enough to make it vanish. confirm-merge.sh had a similar gap: its regex, gh[[:space:]]+pr[[:space:]]+merge, assumed pr would immediately follow gh, so a command like gh -R owner/repo pr merge slipped through the same way once the -R flag got in between. Both gaps ran in the same direction — operations that should have prompted for confirmation went through silently instead.
Rewriting the detection logic, then verifying it against real commands
The gap came down to a limitation baked into the detection method itself: a plain substring match. The fix was to rewrite both regexes to check token order without crossing a command separator (;, &, |) — so git followed eventually by push still matches even with options in between, and same for gh and pr.
Rewriting it wasn’t enough to trust it. I built a matrix of 20 test commands, some that should trigger a confirmation (ASK) and some that shouldn’t (SILENT), and checked each one against the actual behavior. The bypass patterns had been invisible both to the version of me that wrote confirm-master-push.sh two weeks earlier and to the version that wrote confirm-merge.sh that same day — it took the audit side to find them, and then the side that fixed them had to verify against real commands all over again. Building, doubting, and verifying stayed three separate roles.
Beyond those two hook gaps, the retrospective audit turned up 13 findings in total. The /output-style command referenced in a README had actually been deprecated. persona.md hadn’t been refreshed to reflect that day’s redesign. A topology map hadn’t kept pace with a PR that had merged in a separate session. The findings were a mixed bag. I folded in all of them, sent the resulting diff through a structural review a second time, and that pass caught two more inconsistencies I fixed as well. What’s left standing from all this: two rewritten hooks, a split-up CLAUDE.md, a redistributable Skill Pack (19 files), and this list of 13 findings itself.
Ultracode isn’t cheap
I also had Ultracode review this very post after I finished writing it. Two reviewers ran in parallel for one pass — a general reviewer and a structural reviewer — caught a factual error (I’d written that confirm-master-push.sh was added “that same day,” when it wasn’t), and I ran a second pass after fixing it. Those two passes together burned close to 170,000 tokens and took nearly nine minutes. That’s for reviewing a single blog post. The actual retrospective audit — four agents reading the entire live environment — covered a much wider scope than this post does. I didn’t log an exact token count for it, but I doubt it came in lighter than this.
Ultracode is strong. It brings in angles a single agent wouldn’t catch, several at once. This time, it caught a timing error I’d missed myself, in one pass. But it costs accordingly. In this redesign, the actual Phase 1-4 work held up fine on Fable 5 (High) reasoning alone — Ultracode only came in for the audit at the end. Not every step needs Ultracode. Work where judgment and implementation accumulate is better left to one agent thinking deeply; Ultracode earns its keep specifically at the step where you ask “is this actually solid?” That’s the split that’s working for me right now.
The tally
| Subject | Timing | What turned up |
|---|---|---|
| confirm-master-push.sh | Running for 2+ weeks (live since 6/22) | git -C dir push and similar — an option splits git and push, letting it through |
| confirm-merge.sh | Added that same day | gh -R owner/repo pr merge and similar — an option splits gh and pr, letting it through |
| Full retrospective audit (4 angles) | Run that day | 13 findings total including the two above; 2 more caught in a follow-up review |
Closing
The value here was never speed. A safeguard I built myself is hardest to doubt right when I’ve just built it — whether it was built two weeks ago or that same day, the blind spot doesn’t fade with time. When I wrote confirm-master-push.sh, and again when I wrote confirm-merge.sh, the satisfaction of “pushes and merges now require confirmation” came first, and the crudeness of the detection logic underneath never crossed my mind. So instead of scoping the audit to “what changed this time,” I widened it to everything currently running. If I’d kept it narrow, I’d never have run into the hole that had been sitting there for two weeks.
If I’d built this alone and reviewed it myself, I probably wouldn’t have caught that bypass until the next time I hit push. Getting several agents with different vantage points to doubt what the builder can’t see, regardless of when it was built — having that capability mattered more than any single feature I built this time. Though that capability isn’t free either. Part of the setup is knowing where to spend Ultracode in the first place.