Model Routing

Model routing means choosing, per task, which model handles it, instead of sending everything to one model by default. The obvious version routes by price: easy requests to a small cheap model, hard ones to the frontier model, because the gap per token is large.

The unit that matters is cost per successful task, not cost per token. Add what the work costs in full, retries, human review and rework included, count the outputs that met the bar, divide. On that arithmetic the routing table changes: a cheap model that needs a second attempt and an edit loses to an expensive one that lands the task in one pass, even on routine work.

The routing decision is therefore an eval. Define what done means for the task, run your actual prompts against the candidate models, and compare cost per passed task. Start cheap and upgrade where your tests expose gaps, or start strong and step down, keeping every downgrade the evals still pass.