Route by Completion Cost
The central mistake in model routing is treating the strongest model as the default executor. In long autonomous loops, the right unit is not benchmark score or prompt cost. It is completion cost: tokens, wall time, retries, human inspection, failed branches, and the opportunity cost of consuming a scarce model window. A model that is 20 percent weaker but 5 times cheaper and 3 times faster may be the correct default if verification catches its errors cheaply.
Start by labeling tasks by failure cost. Architecture, migration strategy, interface contracts, security-sensitive reasoning, and final review are high-failure-cost steps because a bad answer poisons downstream work. Bulk implementation, mechanical refactors, log triage, classification, doc updates, and retry generation are usually low-failure-cost if the harness can test and revert them. Visual work is a separate class: some frontier models are unusually good at layout, interaction, and aesthetic coherence, while high-scoring coding models can still fail on lighting, event timing, responsive composition, or Three.js behavior.
That gives the first routing rule: expensive models should reduce entropy before the loop branches. Use them to choose the plan, constraints, invariants, test strategy, and decomposition. Then hand bounded implementation tickets to workhorse models. Do not spend frontier tokens discovering what the project is. Spend cheaper tokens on research and codebase mapping, compact that into a precise goal prompt, then let the scarce model act where one correct decision can save hundreds of downstream attempts.
Make the Harness the Router
Once routing is based on completion cost, the router cannot be a static `if task_type then model` table. It has to sit inside the harness and observe state. The harness should own the queue, task metadata, model policy, spend ledger, artifacts, verification results, and escalation rules. Agents should receive scoped work packets, not open-ended authority over the whole project.
A practical schema is simple: `task_id`, `phase`, `risk`, `blast_radius`, `inputs`, `expected_artifacts`, `verification_command`, `max_attempts`, `model_policy`, and `escalation_policy`. Planning tasks might route to a frontier model with a requirement to emit contracts and acceptance tests. Implementation tasks route to a cheaper coding model with strict diff limits. Review tasks run in parallel across quality, security, performance, and product fit, usually read-only. Classification and dedupe use the cheapest reliable model available.
Externalize state. A long loop should not depend on one swelling chat context remembering what happened six hours ago. Store decisions, diffs, logs, failing tests, model calls, token counts, and reviewer findings in Postgres or another durable store. Feed each agent the minimum slice of current state plus the relevant repo files. Fresh contexts between research, plan, implement, and review reduce drift, and make routing cleaner because each phase has a measurable contract.
Verify Before Escalating
The router should be stingy with escalation. Most failures are not proof that the task needs a better model. They are proof that the loop lacks a tighter verifier, smaller scope, or better error compression. Before moving from a workhorse model to a frontier model, force the harness through deterministic checks: format, typecheck, unit tests, integration tests, visual snapshots, lint rules, migration dry runs, benchmark thresholds, and static analysis.
When a cheap model fails, do not send the whole transcript upward. Compress the failure into a repair packet: intended behavior, exact command, failing output, touched files, suspected invariant, and allowed patch surface. Try high-volume retries only when attempts are independent and cheap to verify. Fast models are valuable here because retry loops are latency-sensitive. A model that generates imperfect patches at high speed can still win if tests reject bad attempts automatically.
Escalate when the failure pattern is conceptual, not mechanical. Examples: three attempts break the same invariant differently, tests pass but product behavior is wrong, the codebase boundary is unclear, or reviewers disagree on the correct architecture. The frontier model should then adjudicate the invariant or redesign the decomposition, not merely produce another patch. This preserves scarce reasoning for the point where the loop is stuck for a real reason.
Parallelize With Isolation
Parallelism changes the routing problem because the expensive resource becomes contention, not just tokens. Ten agents sharing one branch, one database, and one context will create cleanup work that destroys the savings. Use worktrees, branch databases, isolated containers, and per-task state records. Let sub-agents inspect broadly, but write narrowly. For many loops, the best pattern is parallel read-only analysis followed by a single supervised implementation stream.
Codebase-wide work benefits from fan-out. Run separate agents for API migration candidates, dead code, security concerns, performance hot spots, visual regressions, and test coverage gaps. Dedupe findings with a cheap classifier, rank by severity and dependency order, then create implementation tasks. If every file needs a mechanical change, split by ownership boundary or package, not by arbitrary file count. Merge only after each shard passes local verification and a final cross-shard integration run.
Visual and frontend tasks need their own gate. A model can satisfy tests and still ship an unusable interface. Route initial visual architecture, animation systems, or complex responsive layouts to the model with the best observed design fidelity. Then downshift for copy swaps, spacing tweaks, small CSS repairs, and component repetition. Require screenshots, viewport checks, and where relevant canvas or pixel-level assertions before the router marks visual work complete.
Budget the Loop, Not the Call
Cost control should be enforced at the loop level. Track tokens by phase, model, task type, attempt number, and outcome. The useful metric is not dollars per million tokens. It is dollars per accepted artifact and dollars per verified project completion. A cheap model that causes five failed branches may be more expensive than a frontier model used once to set the plan. A frontier model that writes verbose patches for routine changes may be burning budget with no completion advantage.
Set budgets as policies: frontier model only for planning above a risk threshold, architecture changes touching shared contracts, failed invariants after bounded retries, final review for high-blast-radius diffs, and specialized visual generation. Workhorse models handle ordinary implementation. Fast cheap models handle classification, log summarization, dedupe, and retry generation. Keep prompts, skills, repo rules, and eval scripts in the repository so the same harness can move across agent products as pricing, rate limits, and model quality shift.
The router is finished when it can explain every expensive call in terms of avoided failure cost. That is the standard advanced teams should hold it to. Prestige models are not the operating system for autonomous engineering. They are scarce components inside one. The harness earns its keep by deciding when their judgment changes the trajectory, and when cheaper parallel loops can grind safely toward the same verified result.
Key takeaways
- Route on real completion cost: token spend, retries, latency, human review, failed branches, and scarce model availability.
- Use frontier models before branching for architecture, invariants, decomposition, high-risk review, and specialized visual work.
- Keep long-loop state outside the chat context so each phase can run in a fresh, scoped context with measurable contracts.
- Retry cheaply only when verification is deterministic and the attempts are independent enough to be useful.
- Parallel agents need isolated worktrees, branch databases, and narrow write authority, with cheap dedupe before implementation.
- Judge the router by cost per accepted artifact and cost per verified project completion, not benchmark rank.