Which LLM for Which Job: A Routing Cheat Sheet for SRE Agents
"Which model and which level do you use?" is a question that keeps coming back in a large chat of developers and DevOps engineers who run coding agents every day. The answers are rarely one model. People pair them: one model to plan, a second to write the code, a third to review it. One user summed it up: a tandem always gives a better result than one model alone.
That advice comes from coding work. An incident pipeline has the same shape, with different jobs: group alerts, investigate, check the conclusion, write the update. This post maps each job to the kind of model it needs, using what practitioners reported and what we measured on SRE Bench. The short version: the expensive model is rarely the one that should do the most work.
What Practitioners Report
These are reports from the chat, not measurements. They are useful because they show which property of a model people pick it for.
- Planning and analysis go to a strong, slow model. Several users plan with the most capable model they have. One noted that the same model often "does what you did not ask" and adds its own ideas. That is good for analysis and bad for an executor.
- Review goes to a careful model, or to the strong model again. A typical split: one model plans, another writes the code, and the final review returns to a stronger model.
- Bulk work goes to a cheap model, with limits. One user gives a cheap model large questions about an unknown codebase of 100K+ lines, because its input is very cheap. The same user tried it as the main code writer and gave up: it missed things and solved problems head-on, without much thought.
- Speed matters more than people expect. A slow model forces you to queue tasks. One engineer said that with a slow model they end up with a stack of ten tasks and forget what each one was about. Another said speed does not matter when you know exactly what to build and build it once.
The pattern is clear. People do not route by price alone. They route by the failure mode they can tolerate in each job: a planner may wander, a reviewer must not, an executor must follow the plan.
The Jobs in an Incident Pipeline
An AI SRE agent does at least four different jobs on one alert. They need different things from a model.
- Group the alert. Is this a new incident or a repeat of a live one? The input is short, the answer is yes or no, and it sits on the alert path, so latency counts.
- Investigate. A loop of tool calls: queries, log reads,
kubectlcommands. It needs reliable tool use and the discipline to run the boring check before it guesses. - Check the conclusion. Does the evidence support the root cause, or did the agent stop at the first broken component?
- Write the update. A title, a Slack summary, a postmortem draft. Short input, short output, any competent model.
What SRE Bench Says About the Investigator
The common instinct is to give the biggest job, the investigation, to the biggest model. Our data does not support that. On the current SRE Bench table, 11 models found the root cause in all twelve incidents. Their cost per identified root cause goes from $0.003 to $0.85, a spread of more than 280 times, for the same accuracy:
| Model | Root causes found | Cost / root cause | Median time |
|---|---|---|---|
| GPT-6 Luna | 100% | $0.003 | 91 s |
| GPT-6 Sol | 100% | $0.020 | 49 s |
| Claude Opus 5.5 | 100% | $0.093 | 75 s |
| Claude Fable 5 | 100% | $0.85 | 95 s |
| Claude Haiku 5.5 | 92% | $0.0033 | 37 s |
| DeepSeek V4.1 Flash 1017 | 92% | $0.019 | 35 s |
Two lessons come from this table.
A cheap model can be a perfect investigator. On these incidents, the cheapest perfect model costs a third of a cent per root cause. A stronger model adds cost, not accuracy.
A 92% row is one missed incident, and the miss is the thing to read. Claude Haiku 5.5 solved our hardest multi-hop incidents and then missed the easiest one because it skipped one command. For an executor, that kind of slip matters more than raw intelligence. It is also the slip a review step catches.
The table is one run per task, and prices change often. Use it to pick candidates, then measure them on your own incidents.
Do Not Think on the Alert Path
Grouping alerts looks like a job for a small, fast model. In practice, the reasoning setting matters as much as the model size. In Akmatori's alert correlator, a reasoning model thinks for 1,000 to 1,900 tokens over a prompt with 25 candidate incidents. On one production install that took 13 to 20 seconds per call, so a 15-second timeout failed most calls and we had to raise it to 45 seconds.
For a yes-or-no job on every alert, that is the wrong trade. Two fixes help:
- Use no model when a rule works. Akmatori first tries a deterministic fingerprint match: an alert with the same source, alert name and host as an alert on a live incident (active in the last 48 hours) joins that incident without an LLM call. The model sees only the alerts the rule cannot decide.
- Turn thinking down for classification. Fewer thinking tokens mean a faster answer. Try a low thinking level, or a non-reasoning model, and check that the verdicts stay the same.
Route per Job, Not per Turn
A prompt cache belongs to one model. If a harness sends turn 12 to a cheap model and turn 13 to a strong one, each model reads the whole context again without a cache hit. Our post on where agent tokens go shows how a cache miss on a 150K context costs 25 times more than a hit. Model switches inside one investigation give you the same miss on every switch.
So split the work at clean boundaries. The investigator runs one session on one model. The reviewer starts a new, short session with only the conclusion and the evidence. The summary writer gets only the final report. Each job then has a small context and a warm cache.
The Cheat Sheet
| Job | What it needs | Model to use | Trap |
|---|---|---|---|
| Group repeat alerts | Speed, yes or no | A rule first, then a small model with low thinking | A reasoning model took 13 to 20 s per alert |
| Investigate | Tool discipline, follows the plan | The cheapest model that is perfect on your incidents | A model that skips the boring check |
| Check the root cause | Skepticism, depth | A strong model in a new, short session | Re-reading the whole investigation |
| Plan a remediation | Analysis, options | A strong model; a human approves | Strong models add steps you did not ask for |
| Title, Slack update, postmortem draft | Clear writing | Any competent cheap model | None worth paying for |
| Read a large unknown codebase or log dump | Cheap input | A cheap model with a large context | Its summary is a lead, not a verdict |
How to Choose for Your Installation
Measure cost per solved incident, not price per token. A model that needs a second investigation, or a human, costs more than one that gets it right once. Take five to ten real incidents with known root causes, replay them on two or three candidate models, and keep the cheapest model that finds all of them.
Akmatori supports OpenAI, Anthropic, Google, OpenRouter, NVIDIA NIM, MiniMax and custom OpenAI-compatible endpoints such as Ollama or vLLM. You can save several model configurations, each with its own thinking level, and switch the active one in Settings → AI models. Today one active configuration serves every job, so the easiest win is to pick a cheap model that is perfect on your incidents. Setup is in the getting started guide.
Conclusion
Route models by the failure each job can tolerate. Give the investigation to the cheapest model that you measured as reliable, give the review to a strong model with a short context, and give grouping to a rule before any model. Split jobs at session boundaries so each model keeps its cache warm.
If your team wants AI-assisted incident workflows on your own infrastructure, Akmatori helps SRE teams investigate alerts, coordinate response, and automate safe infrastructure actions. Powered by Gcore for global infrastructure reliability.