The Cheapest Model That Can Safely Do the Job
How do you start using bigger models for the complex tasks without driving the cost through the roof?
My last article described a delivery pipeline end to end and gave its routing table exactly one section. That section said what the table does. It did not say how you would build one, why mine is deterministic, or where a light LLM assessment genuinely earns a place. So this is the zoom-in.
The short version, up front. The router is the least clever component in my entire system, and that is precisely why I trust it with autonomous work. Everything difficult happens before the routing decision. By the time a unit of work reaches the router, choosing a model is a lookup, a length check and a keyword scan. About forty lines of Python. No model call, no latency, no new failure mode.
Routing is downstream of specification
The instinct, when people start thinking about model routing, is to reach for a classifier. Something clever that reads the task and divines how hard it is. I think that instinct gets the dependency backwards.
You cannot route on complexity until complexity is legible, and complexity only becomes legible when the work is specified. In my pipeline, every unit of work arrives as a blueprint with one binding property: it must be executable by a weaker model without any further design decisions. The planning step, run by a strong model with tools and a read-only clone of the repository, has already made every judgement call. What remains is construction.
That constraint is what makes cheap models safe at all. A weak model given vague intent does not fail loudly. It fills the specification gap with guesses, and its guesses are worse than a strong model’s guesses, which is exactly the quality difference you were trying to route around. A weak model given a complete blueprint is doing the one thing it is genuinely good at, which is carrying out instructions.
So rule zero of routing: the router looks like the interesting part, but the specification discipline upstream is what does the work. If your work items are not specified tightly enough to route, the fix is a planning step, not a smarter router.
A router you can read in one sitting
Here is the real mechanism, lightly simplified from the code that runs my builds:
BUILDER_FOR_COMPLEXITY = {"high": "claude-opus", "medium": "gemini-pro",
"low": "glm", "trivial": "local-mistral"}
def infer_complexity(unit):
explicit = unit.get("complexity") # a human said so
if explicit:
return explicit
body, deps = unit["body"], unit["deps"]
if len(body) > 2500 or len(deps) >= 2:
return "high"
if len(body) > 900 or deps:
return "medium"
return "low"
def recommend_builder(unit):
if unit.get("builder"): # explicit override always wins
return unit["builder"]
tier = BUILDER_FOR_COMPLEXITY[infer_complexity(unit)]
risk = f'{unit["id"]} {unit["title"]} {unit["spec_ref"]} {unit["body"][:4000]}'
if tier != "claude-opus" and COMPLEX_SURFACE.search(risk):
return "claude-opus" # erred upward, on purpose
return tier
Six design decisions are hiding in there, and each one matters more than the code suggests.
A human override always wins. A builder: or complexity: field on the unit itself beats everything automatic. The router recommends. It never overrules a person who has looked at the work.
The heuristic is deliberately coarse. Complexity falls out of body length and dependency count. Over 2,500 characters of specification, or two or more dependencies, and it is high. Over 900 characters, or any dependency at all, and it is medium. Otherwise low. That sounds too crude to trust, and on its own it would be. It only has to be roughly right, because of the clamp that sits on top of it. Resist the urge to make this bit clever. Every hour spent tuning a complexity classifier is an hour that would be better spent on the guard rail above it.
The table maps complexity to an engine. For me right now that is Opus 5 running headless for high, Gemini 3.1 Pro for medium, GLM 5.2 for low, and a local mistral-small-4 on hardware in my house for trivial. The names will rot. The shape will not: one frontier route, one or two capable mid routes on cheap or free capacity, one free local route for bounded work.
The risk scan overrides everything automatic. Before any recommendation is honoured, the unit’s id, title, spec reference and body are scanned for a short list of danger words. Mine include consent, privacy, RLS, migration, ledger and commission. Yours will differ, but there will be about a dozen words in your domain that mean “if this goes quietly wrong, it really matters”. If the scan hits, the recommendation is discarded and the unit goes to the strongest model, however simple it looked.
Two details in that scan took real debugging to learn. First, scan depth is asymmetric on purpose. The dangerous unit is not the hard one that looks hard. It is the one whose title reads like a bounded pure transform, a normaliser or a formatter, while the acceptance criteria three paragraphs down quietly touch consent data. So questions about the work’s shape read the headline fields, and questions about risk read the body too. Second, regex word boundaries treat an underscore as a word character, so a naive \bconsent\b never fires inside cognitive_profile_consent, and those snake_case identifiers are exactly the surface the guard exists for. The boundaries have to be alphabetic. That is the kind of bug you only find by looking at what the scan missed.
The floor is a whitelist, not a default. The heuristic can never output trivial. The weakest model is reachable in exactly two ways: a human explicitly tags the unit for it, or the unit’s shape matches a whitelist of bounded pure transforms (parsers, adapters, normalisers, pollers, mappers, extractors) and the risk scan comes back clean. Reaching the cheapest tier requires positive evidence of a safe shape, not the mere absence of complexity signals. And the config loader refuses outright to let the complex class of work be assigned to the weakest engine, because one line of config must never be able to invert that guarantee.
Failure escalates instead of failing. A unit that fails on a cheaper engine is re-dispatched to a stronger one rather than marked failed and left for a human. The escalation target is set per project, so adding a route to one product cannot change the behaviour of another. This is what makes the coarse heuristic affordable in the other direction too: the cost of an over-optimistic down-route is a retry on a better model, caught by validation and review gates that run regardless of who built the thing.
Every automatic mistake must be the affordable one
If you take one line from this piece, take this one. A unit sent up a tier only wastes money. A unit sent down a tier risks a silently wrong pull request. Those two errors are not remotely comparable, so the router is built to make the cheap mistake freely and the expensive one never. Every automatic decision errs upward.
The clamp in one picture. Doubtful work leaves the cheap path and joins the strong one. The reverse journey does not exist.
That asymmetry is also why the router can stay deterministic. A deterministic router is auditable: every decision has a reason you can print. It is replayable: the same unit routes the same way tomorrow. It is testable: the whole policy sits under ordinary unit tests in CI. It costs nothing and adds no latency. And the only influence a work item’s own text has over it is to make its build more expensive, because matching the danger list routes it up. You give all of that away the moment the routing decision itself becomes a model call, and what you get back is the ability to handle unstructured input, which is worth having exactly once in the pipeline. I will come to where.
The same posture applies to new engines. When I added a second mid-tier engine, it had no end-to-end track record, so the risk scan routed sensitive work past it to the frontier model even where the table said otherwise. Engines earn access to risky work from bench data, not from a launch benchmark chart.
When the work is too messy for a regex
The deterministic router works because units of work have structure: a title, a body, dependencies, metadata. Plenty of real intake has none of that. A pile of customer emails. Feedback from a user who describes symptoms, not causes. A screenshot. This is where a light LLM assessment earns its place, and the placement matters: at the intake, not at the dispatch.
Two patterns I run live.
The supervisor. In my assistant, every incoming message hits a small local model first. If the message is simple, the small model answers it instantly. If it is complex or needs tools and multiple steps, the small model acknowledges it and dispatches the work to a stronger tier asynchronously. The conversation stays responsive, the routing decision costs nothing per message, and the expensive model runs only when something actually needs it.
The extractor. On my legal platform, incoming feedback can be a hundred thousand tokens of messy email threads, documents and images. A free local multimodal model reads the pile and produces a compact structured digest of discrete items, each anchored to a verbatim quote from the source. A frontier model then plans from that digest, with the quotes and the full source available as ground truth. Cheap models build what it plans. Reviews gate the result.
The rule that makes both patterns safe is the same rule: the light model is a router, not a judge. It classifies, structures and forwards. It never makes a design decision, never gets to summarise away the ground truth, and never decides what should be built. The moment your cheap model’s opinion becomes load-bearing, you have quietly routed the hardest work in the system to the weakest model in the system, which is the exact failure the router exists to prevent.
Where a light model earns its keep: at the intake. It straightens and tags the messy pile. It does not judge it.
And you have to measure it. My extractor’s accuracy is not a feeling, it is a recall number against a hand-built golden set of items it should have found, currently sitting around 77 per cent with the misses mapped item by item. Getting that number taught me the measurement lesson the hard way: my first judge was itself an LLM, and run-to-run variance swamped the signal until I replaced it with a deterministic matcher and averaged over multiple runs. Determinism is not an aesthetic preference. It is what makes the numbers mean anything.
If you want the best of both, combine them. Let the light model assess once at intake and write its verdict into the unit’s metadata as a complexity tag. From then on, the deterministic table consumes the tag. You pay for assessment once per unit rather than once per routing decision, the human can override the tag, and dispatch stays replayable forever.
The second axis: who is waiting
Complexity decides how smart the model needs to be. There is a second, independent axis that decides how fast and how paid it needs to be: whether a human is waiting.
The convention I have run since April is simple. Interactive work goes to a fast paid API, because a person is sitting at the terminal and seconds matter, and at a fraction of a penny per call the cost is irrelevant at interactive volumes. My pre-commit code review works this way, at roughly £1.50 a month for a review on every commit. Asynchronous and overnight work goes to the free local model, because nobody is waiting, and a sixty-second response time is invisible at three in the morning. Pure classification goes to the smallest thing that can do it.
Fold the two axes together and the routing principle reads: the cheapest model that can safely do the job, on the fastest tier the waiting human requires.
It is also worth being deliberate about where your meter is. My frontier route runs on a flat-rate subscription, the mid routes on cheap per-token capacity or free allowances, and the local route on electricity. Because the expensive tier is flat-rate, the asymmetric clamp is affordable: erring upward costs capacity, not marginal spend. If your frontier route were metered per token, the same clamp would still be correct, but you would feel every conservative decision, and you would be tempted to loosen exactly the guard that keeps the system safe. Structure the billing so the safe choice is also the comfortable one.
What the table does in practice
In the twelve days to 26 July the ledger records 88 autonomous builds across ten projects: seventy on the frontier route, eleven on the mid route, seven on the local model. The local route had a median build time of about three minutes against roughly twenty-four for the frontier route, and two of those local builds became merged production pull requests. A free model on hardware in my house shipped real product, because by the time work reached it there were no decisions left to make.
Read the split honestly, though. Eighty per cent of builds still went to the strongest route, because the clamp errs upward and most of my current backlog genuinely touches surfaces I have named as risky. That is the correct starting position. The share on cheaper engines should drift up over time as bench data earns them exemptions, and if it drifts up because someone relaxed the guard rather than because the evidence did, that is not optimisation, it is the expensive mistake on an instalment plan.
Building yours
The distilled version, in the order I would build it:
- Specify units of work until complexity is legible. If your intake is unstructured, add a triage step that structures it. That step is where a light LLM belongs.
- Start with a lookup table and a coarse heuristic over size and dependencies. Do not build a classifier.
- Name your danger words. About a dozen domain terms that mean “quietly wrong is expensive here”. Scan shape shallow, scan risk deep, and make the word boundaries survive snake_case.
- Make every automatic error the affordable one. Any doubt routes upward.
- Make the cheapest tier a whitelist of provably bounded shapes, never a default.
- Escalate failures up a tier automatically instead of parking them.
- Let a per-unit human override beat everything.
- Keep engines as configuration, a name-to-launcher lookup, so a new model is an entry rather than a refactor.
- If you do use an LLM assessor, use it once at intake, write its verdict into metadata, and keep it a router, never a judge.
- Measure the cheap tiers against a golden set with a deterministic judge before you widen what they are allowed to touch.
The clever parts of an agent pipeline get all the attention: the planner exploring a repository, the reviews, the memory. The router is a table with a guard rail on top, and that is exactly why the whole thing can run unattended. Route work to the cheapest model that can safely do it, and make sure every mistake the automation can make on its own is one you can afford.
Related Dendro Logic writing
- The Drawing Office and the Yard – the full pipeline this router sits inside, and the handoffs between its stages.
- Closing the Loop to Full Autonomy – why independent gates make the merge ceremony, which is what makes cheap builders safe to trust.
- Agile Is the Bottleneck Now – why a delivery process built around the cost of writing code measures the wrong thing once that cost collapses.
The router described here runs inside the foreman on an always-on Nvidia Spark, dispatching headless builds across Claude Opus 5, Gemini 3.1 Pro, GLM 5.2 and a local mistral-small-4. Figures are drawn from the build ledger for the twelve days to 26 July 2026 and from bench runs against hand-built golden sets.