Making AI Agents Model-Proof — a Handoff System That Survives Model Upgrades
使用ツール: Claude Fable / Claude Opus / SKILL.md
Most AI workflows quietly die the day you switch models. This is how I kept mine alive — by moving the operating knowledge out of a chat window and into a document any model can pick up and continue.
The problem
Every AI operator eventually hits the same wall. Your carefully tuned workflow lives inside one chat session, with one model, held together by context you built up conversation by conversation. The moment anything changes — you switch to a newer model, open a fresh session, or hand the work to an autonomous coding agent — that accumulated context evaporates and quality collapses. You end up re-explaining your business, your constraints, and your standards from scratch. I call this the “context tax,” and it gets more expensive every time the model landscape shifts, which lately is every few weeks.
For a solo operator running several projects at once, this tax is not a minor annoyance — it is existential. If your operation only works while one specific model is loaded and one specific chat thread is alive, you do not really have an operation. You have a fragile live performance that has to be re-staged from memory every single time. And because the knowledge lives only in your head and in a disposable session, there is no way to delegate it, no way to audit it, and no way to trust that today’s output was held to the same standard as last week’s.
The diagnosis
The root cause was not the model. It was that the operating knowledge lived in the wrong place: inside a volatile session instead of in a durable, portable artifact. Everything a model “knew” about how to do the work was tribal knowledge — re-derived from scratch each session and lost the moment that session ended.
Diagnosing it this way turned a vague frustration into a concrete list of requirements. The operating knowledge had to be (1) written down in one authoritative place, (2) readable by any model or agent before it touches the work, (3) explicit about what must never happen and about what a human must always own, and (4) enforced by something other than the model’s own self-assessment — because, tested repeatedly, a model grading its own homework reliably inflates the score. Any solution that missed even one of these four would leave the context tax partly in place.
The approach
I codified the entire operating procedure into a portable skill file (SKILL.md) that any model — Claude Fable, Opus, or an autonomous coding agent — reads before starting work. The file is ordered by priority:
- Hard Rules — non-negotiable constraints: naming conventions, secrets handling, and specific things that broke production before. One example: a CDN’s email-obfuscation feature once silently mangled the ”@” character in version strings and broke page titles. That painful lesson is now a written rule, not something a future session has to rediscover the hard way.
- A Definition of Done per task, with explicit phase gates and ON-FAIL branches, so that “finished” means exactly the same thing to every model and every session.
- A human-only scope — authentication, payments, code merges, and final submissions always stay with a person and are never delegated to an agent, no matter how capable it appears.
- A scoring rubric (Process 30 + Quality 40 + Data 30; anything below 70 is an automatic redo), applied by a separate AI session acting as an adversarial reviewer. Separating the reviewer from the author is the entire point: it is the mechanism that keeps the scores honest.
Crucially, none of this is model-specific. The file describes how the work must be done and checked, not which model does it. That is what makes it survive an upgrade.
The result
- New sessions and different models resume work with zero re-briefing. Onboarding a new model costs one file read instead of an hour of re-explaining context.
- Self-reported “done” claims are no longer taken at face value. The independent review step routinely catches the gap between “the metric passed” and “the goal was actually achieved” — the exact place where autonomous agents tend to declare false victory.
- The same four-phase loop (plan → autonomous execution → adversarial review → correction) now runs across a self-run news site, a video pipeline, and an iOS app, using one shared method rather than three bespoke, fragile workflows.
- Each model upgrade has shifted from being a disruption to being a straight upgrade: the same operating file simply runs on a stronger engine.
What this means for your business
The value here isn’t a clever prompt. It’s the written system around the agent: the rules, the quality gates, and a clear line for who is responsible when something goes wrong. That system is a document your team owns — and it is exactly the kind of work I take on, and finish.