DeepSeek V4.1 Flash 1017: Same Score, a Third of the Cost
SRE Bench gives each model a live incident in a real broken environment: a real alert, real tools, and a fresh install of Akmatori with no accumulated memory. A pinned judge grades every investigation against written ground truth. All three runs below used the same twelve scenarios, the same Akmatori release, and the same judge.
The Numbers
| Checkpoint | Root causes found | Cost per root cause | Median investigation |
|---|---|---|---|
| V4 Flash 0731 | 83% | $0.042 | 32 s |
| V4.1 Flash (September) | 92% | $0.062 | 78 s |
| V4.1 Flash 1017 | 92% | $0.019 | 35 s |
V4.1 Flash bought accuracy and paid for it in speed. The 1017 checkpoint keeps the accuracy, wins the speed back, and costs less than either predecessor.
Each Checkpoint Fell for a Different Trap
Our scenarios hide the true cause one hop behind a plausible wrong answer. The scores look flat, but the misses moved around.
- 0731 lost two trap scenarios. On the 502 incident it blamed long-exited Docker containers instead of finding the crash-looping backend with a corrupted
PORTvalue. - V4.1 Flash fixed both traps, then slipped on a Kubernetes selector mismatch. It saw a Service with zero endpoints and concluded the Redis backend did not exist. The healthy pods were there, labeled
app: rediscache. - 1017 solved the selector mismatch and the 502, stale-endpoint, and inode traps, then took the disk-full bait. It found the 2.4 GB of
session-cache-*.datfiles under/var/bench-dataand even noted they were being written during the incident. It still ranked them third, behind 8.9 GB of old containerd snapshots and the systemd journal.
That last miss is instructive. The model had the decisive clue, files growing right now, and let the biggest number on the screen outrank it.
Where the Savings Come From
Be careful with the cost column. The 1017 run used about 12 percent fewer tokens than the September run, and 35 percent fewer output tokens. That is real, but it does not explain a 3x gap.
Most of the difference is the serving path. We ran 1017 through OpenRouter, where 89 percent of input tokens were served from the prompt cache at a fraction of the normal price. The September run used a dedicated endpoint that reported no cache hits. The speed gain likely mixes model and provider effects in the same way.
Pricing also varies by time of day. Our run fell in the weekday peak window. Off-peak, the same investigations would cost about $0.009 per root cause.
What Operators Should Take From This
- Check prompt caching before you compare prices. An agent re-sends a long context on every step. Cache support can matter more than the per-token list price.
- Same score is not same behavior. Three checkpoints at 83 to 92 percent each missed a different scenario. If you upgrade, re-test your own incidents.
- Single runs have limits. One run cannot tell character from dice. The next SRE Bench edition adds repeated runs, so every score carries a reliability figure.
Conclusion
DeepSeek V4.1 Flash 1017 matches its predecessor's accuracy at under two cents per root cause with 35-second investigations. That makes it one of the cheapest strong options on the board, as long as your provider caches prompts. The full table and methodology are at akmatori.com/srebench.
Akmatori helps SRE teams connect alerts, metrics, logs, traces, and runbooks into governed incident workflows. Powered by Gcore, Akmatori gives operators the control they need when production behavior needs a fast explanation.