How I Spent 3 Days of Frontier-Model Access Auditing My AI Stack

Quick conclusion

Claude Fable 5 landed in my subscription with a catch: I only get to use it at full capacity until July 7. Three days. My first instinct was to build something new with it. Instead, I pointed it at something less exciting and more useful: a full audit of the 18 automation “skills” that run my five blogs. It found a security-relevant bug, a model-dependency I didn’t know I had, and a failure mode in its own subagents. Here’s what that looked like, and the framework I used to decide it was the right call.

The setup: a 3-day window and an audit debt

Fable 5 has had a strange few weeks. It launched on June 9, then got pulled within days after a temporary US export-control order restricted it, and came back globally on July 1 with a usage cap that runs out on July 7. If you use Claude, you’ve probably watched that whole cycle too, and if you’re anything like me, you spent the last few days of “it’s finally back” figuring out what to actually do with it before the clock runs out again.

I run five blogs solo, mostly through a set of reusable AI skills, roughly 18 of them, about 10,600 lines total, that handle everything from drafting to fact-checking to publishing. I built this system gradually, over months, patching things as I noticed they were broken.

That means it also carries an audit debt: small inconsistencies, half-fixed edge cases, and assumptions that quietly stopped being true as the system grew. Nobody was ever going to sit down and read all 10,600 lines end to end. It’s the kind of task that’s always technically important and never urgent enough to schedule.

Then Fable 5 showed up with a hard deadline attached. That deadline is what made the decision for me.

What I asked it to do

Instead of asking Fable 5 to write anything, I asked it to run an audit: find bugs, wasteful token usage, and anything that would break if I switched models later. It split the work across six Sonnet 5 subagents, each auditing a different slice of the system, then verified the highest-severity findings itself. Across the full audit, that added up to more than 100 individual checks confirming that referenced files actually existed.

The overall verdict: the system was healthier than I expected. Almost no broken references, the existing safety checks were firing correctly, and nothing was quietly on fire. That itself was worth three days, since I’d been operating on faith, not evidence.

What showed up

Four findings are worth walking through, because each one taught me something different about running an AI-operated system.

A hardcoded model name. One of my unattended daily routines had a specific model version written directly into its instructions. Fable 5 flagged this and rewrote it so the model name lives in one config field instead. Next time a new model ships, I change one line instead of hunting through the file. This is the fix I care about most, and I’ll come back to why.

A bug that could quietly unpublish a page. A setup script I use for new WordPress sites had a default behavior where running it in “update” mode, without an explicit flag, would silently downgrade an already-published page back to draft. I’d never triggered it. I could have, without noticing, on a page that mattered. It’s fixed now, with unit tests to prove it.

Subagents that got stuck waiting. Of the six Sonnet 5 subagents Fable 5 spun up, three stalled out waiting for “another agent” to do the work, an empty-response loop I’d seen before but hadn’t fully closed off. The fix was one line added to every delegation prompt: do this yourself, right now, don’t wait on anyone else. All six recovered instantly once that line was in place. A smarter model doesn’t mean fewer coordination bugs.

A changelog diet. Several of the instruction files my routines read every morning had grown heavy with historical changelog entries, in one case roughly a third of the file’s total length. Trimming those down to the last few entries each cut about 22KB off what gets read every day, unattended. Not glamorous, but it’s the kind of cost that compounds.

What I did not do

Some lower-priority findings, mostly cosmetic version-number mismatches and documentation drift, got logged and explicitly skipped, with the reason written down next to them. Not everything a smart model finds is worth fixing on a deadline. Writing “skipped, here’s why” felt more honest than quietly ignoring it.

The trap I was watching for

The hardcoded model name is the fix that matters most, because it’s the opposite of what I was worried I’d do. It would have been easy to let Fable 5 restructure things in ways only Fable 5 could maintain well, then find myself stuck once the subscription window closed and I was back to my regular model. Auditing with a smarter model is only useful if the fixes still hold up after it leaves.

A framework for your own time-limited upgrade

If you get a taste of a frontier model on a countdown, three questions helped me more than any feature list:

  1. What’s the audit debt I’ve been putting off because it’s never urgent?
  2. Can I use this model as a critic of something I already built, instead of the author of something new?
  3. Will this fix still work once I’m back on my regular model?

FAQ

Why did I only have 3 days with Fable 5?
Fable 5 launched on June 9, then was pulled within days under a temporary US export-control order. It came back globally on July 1, included in Pro, Max, and Team plans at a capped share of weekly usage only through July 7. After that, further use runs on separate usage credits.

Did Fable 5 write any of my blog content during this window?
No. I used it entirely for auditing my own automation system, not for drafting articles.

How long did the audit take?
One session, split across six subagents auditing in parallel, with the highest-severity findings verified afterward.

Is my system now dependent on Fable 5?
That was the risk I was actively guarding against. The main fix I kept (removing a hardcoded model reference) makes the system less dependent on any single model, not more.

Next steps

With the remaining time in this subscription window, I’m turning the same audit approach on each individual blog: reviewing the editorial policy for each one and pulling together a list of improvements per site, rather than treating the vault-wide system as the only thing worth checking.

Leave a Comment

CAPTCHA