The cheapest way to cut an AI bill is not a discount. It is sending each request to the cheapest model that can handle it.
That is model routing: a cheap model handles the bulk, and hard or high-value requests escalate to a stronger one.
For current prices, see latest AI models. For the full comparison, see the AI model comparison for 2026.
Why routing works
Most production traffic is routine. Support replies, classification, extraction, summaries and simple tool calls do not need a flagship model.
A typical distribution looks like this:
| Share of traffic | Difficulty | Model tier |
|---|---|---|
| 60–70% | Simple | Haiku 5.5, GPT-6.1 Terra, Gemini 3.6 Flash |
| 20–30% | Normal | Sonnet 5.5, GPT-6.1 Sol |
| 5–10% | Hard | Claude Fable 5.1, GPT-6.1 Sol |
| 1–2% | Critical | Flagship plus human review |
Sending everything to the flagship pays flagship prices for work a cheap model handles.
The three routing patterns
1. Rule-based routing
Route by task type. Refunds go to the strong model; order-status lookups go to the cheap one. Simple, predictable and easy to audit.
Best for: workflows with clearly different task classes.
2. Confidence-based escalation
The cheap model answers and returns a confidence signal. Below a threshold, the request escalates.
Best for: support and classification, where most cases are easy.
Risk: models are often confidently wrong. Calibrate the threshold on real data.
3. Verify-then-escalate
The cheap model produces an answer, a validator checks it against schema and rules, and failures escalate.
Best for: extraction, structured output and anything with a deterministic check.
This is the most reliable pattern because the escalation trigger is measurable.
A practical router
if task.type in HARD_TASKS:
model = flagship
elif cache_hit:
model = cheap
else:
result = cheap(prompt)
if not valid(result) or result.confidence < 0.7:
result = flagship(prompt)Add a maximum escalation rate. If more than 20% of traffic escalates, the cheap model is wrong for the job, not the router.
The cost maths
Take 1 million requests a month, 1,000 input and 300 output tokens each.
| Strategy | Monthly cost |
|---|---|
| All flagship ($10 / $50) | 1M × (1,000 × $10 + 300 × $50) / 1M = $25,000 |
| All cheap ($0.10 / $0.50) | $250 |
| 70% cheap, 25% mid ($2 / $10), 5% flagship | $2,675 |
Routing cuts the bill by roughly 90% against all-flagship while keeping quality where it matters. The exact numbers depend on your mix, but the shape holds.
What routing costs you
- Engineering time. A router is code, tests and monitoring.
- Two integrations. Every model needs its own client, limits and error handling.
- Evaluation. You must know which model is good enough for which task.
- Observability. Without logs, you cannot tell whether quality dropped.
- Prompt drift. A prompt tuned for one model may fail on another.
Routing is not free. It pays when volume is high or quality requirements vary widely.
How to measure it
Track four numbers:
- Escalation rate by task type.
- Quality score on a fixed evaluation set.
- Cost per successful task, not per token.
- p95 latency, including the escalation path.
If cost falls but quality falls faster, the router is a false economy.
Common mistakes
- Routing on model name instead of task type.
- Using confidence scores that were never calibrated.
- Escalating silently without logging the reason.
- Sending everything to the flagship “to be safe”.
- Forgetting that a cheaper model may need a different prompt.
- Not testing the fallback path when the primary provider is down.
A simple starting point
- Classify your traffic into three buckets.
- Send the easy bucket to a cheap model.
- Send the hard bucket to a flagship.
- Add verify-then-escalate for the middle.
- Measure cost per successful task for two weeks.
- Adjust the thresholds, not the architecture.
FAQ
What is AI model routing?
Sending each request to the cheapest model that can handle it, with escalation for hard or high-value cases.
Does routing hurt quality?
Only if the cheap model is used outside its competence. Measure on a fixed evaluation set and cap the escalation rate.
Which models should I route between?
A cheap tier such as Haiku 5.5, GPT-6.1 Terra or Gemini 3.6 Flash, and a flagship such as Fable 5.1 or GPT-6.1 Sol.
How much can routing save?
Typically 60–85% against sending all traffic to a flagship, depending on your mix.
Next step
If you are building an AI product and want the cost model right before launch, contact 24Bit System. Our AI app development cost guide covers the build side, and best AI model for agents covers the agent layer.
