Skip to main content
November 20, 2026

Which Tasks to Give an AI Agent: A Practical Taxonomy for Coding and SRE Work

Which Tasks to Give an AI Agent: A Practical Taxonomy

Two requests can sound the same and go very differently. "Port this service from Python to Go and keep the tests green" usually goes well. "Make our alerting adequate" usually stalls, loops, or comes back with something confident and useless.

The difference is not difficulty. It is who decides when the task is done. Below: a taxonomy by that test, a checklist, and why adding one more prompt rule per failure makes things worse.

The Dividing Line: Who Decides "Done"?

An agent works in a loop: act, observe, adjust. The loop only converges when the agent can observe something that says pass or fail. Anthropic's Building effective agents names this as the reason coding suits agents: "Code solutions are verifiable through automated tests" and "Agents can iterate on solutions using test results as feedback."

The Claude Code best practices put it plainly: "If you give it something that comes back pass or fail, Claude can close that loop itself."

So the first question is: what is the oracle? A compiler, a test suite, a coverage counter or a metric means delegate. "I'll know it when I see it" means you hold the oracle, and the agent cannot run you in a loop.

Tasks Agents Handle Well

Each of these has a judge outside the agent.

Porting and migrations. The old code is the specification and the existing tests are the judge. Google's paper Migrating Code At Scale With LLMs At Google covers 39 migrations by three developers over twelve months: 595 code changes, with 74.45% of the changes generated by the LLM, and an estimated 50% cut in total time compared with earlier manual migrations. Claude Code ships this pattern as /batch, which splits "one big mechanical change across many files" into 5 to 30 units, each in its own worktree.

Fuzzing. A fuzz target either compiles, runs, and reaches new code, or it does not. Google's oss-fuzz-gen scores generated targets on compilability, runtime crashes, runtime coverage, and line coverage against the human-written targets. Its README lists 30 new bugs and vulnerabilities found by generated targets, including CVE-2024-9143 in OpenSSL, that "were not reachable with existing OSS-Fuzz targets."

Writing tests. Coverage is a number. Meta's TestGen-LLM paper reports that 75% of generated tests built, 57% passed reliably, and 25% increased coverage. Filters that "assure measurable improvement over the original test suite" discarded the rest, and engineers accepted 73% of the surviving recommendations. A machine threw out most candidates, not a reviewer.

Fuzzy search. "Find past incidents that look like this alert" has no exact spec, but each hit is cheap to check. Fuzzy inputs are fine when verifying an output is cheap.

Goal-driven search. When the goal is a number, the agent can search for it. DeepMind's AlphaEvolve "verifies, runs and scores the proposed programs using automated evaluation metrics" and helps most "where progress can be clearly and systematically measured." At ops scale, "cut p99 of this query below 50 ms without changing results" has the same shape.

Tasks That Stall on "Good" and "Adequate"

Words like "good", "clean", "adequate", "reasonable" and "production-ready" hide the oracle in your head. The agent then guesses, and it rarely says it is guessing.

The Ambig-SWE paper (ICLR 2026) measured this. The authors took SWE-bench Verified tasks, removed details from the issue text, and compared agents on the full and underspecified versions. Agents did worse without the details. Letting agents ask clarifying questions recovered a lot, "up to 74% over the non-interactive settings", but not all of it: with interaction, Claude Sonnet 4 reached about 89% of its full-spec performance, while Deepseek-v2 and Llama 3.1 70B reached 59% and 54%. And models struggled to tell a well-specified task from an underspecified one. An agent does not reliably notice that it is missing the definition of "adequate".

Decision Table

Task Oracle Delegate? What to add first
Port a module to another stack Existing tests, build Yes Run the old tests against the new code
Write unit tests Coverage, pass rate Yes Reject tests that do not raise coverage
Fuzz a parser Crashes, coverage Yes A harness that builds and runs targets
Find incidents similar to this alert Human confirms each hit Yes, read-only A short list of candidates, not a verdict
Tune a query or config to a target A metric Yes The metric, the limit, and a rollback
Summarize an incident timeline Facts in logs and chat Yes, with citations Require a source for every line
"Make alerting adequate" You Not yet Turn "adequate" into checks (see below)
"Clean up this code" You Not yet A linter config or a target structure
Decide whether to fail over You and the business No Agent gathers evidence, human decides

The Rule-Per-Edge-Case Trap

When an agent gets a vague task wrong, the reflex is a new rule: "Always check disk before memory." Then "Unless it is a Kubernetes node." Then "IMPORTANT: never restart payments." Six months later the prompt is a page of exceptions nobody dares to delete.

Anthropic's context engineering guide names the pattern: "teams will often stuff a laundry list of edge cases into a prompt", and "We do not recommend this." It warns against "hardcoding complex, brittle logic" in prompts and recommends "a set of diverse, canonical examples" instead.

The cost is measurable. IFScale tested 20 models with up to 500 instructions in one prompt; "even the best frontier models only achieve 68% accuracy" at that density, with a bias toward earlier instructions. Rule 40 competes with rules 1 to 39.

The Claude Code docs say every CLAUDE.md line "takes a little of Claude's attention on every task", and if Claude keeps ignoring a line, "adding IMPORTANT probably won't fix it": the file is too long, or the rule belongs in a hook.

Each rule patches one output instead of fixing the missing oracle. Better moves:

  1. Turn the adjective into a check. If "adequate" means "every paging alert has a runbook link", write a script that fails when one does not.
  2. Move hard rules into code. "Never restart payments" is a denylist in the tool layer, not a sentence.
  3. Replace exceptions with examples. Two worked investigations teach more than ten "unless" clauses.
  4. Delete rules when you add a check. A rule the check enforces is noise.

Worked Example: Making "Adequate Alerting" Delegable

The vague version: "Review our alerting and make it adequate."

The delegable version:

  • For every alert that paged in the last 30 days, list whether it has a runbook link, an owner label, and a severity.
  • Flag alerts that fired more than 20 times without a linked incident.
  • Flag services in the inventory with no alert on error rate.
  • Output a table and a draft change per gap. Do not apply anything.

Each line has a yes or no answer the agent can compute from data. Whether 20 is the right threshold is a human call, made once in the task instead of guessed on every run. It is the same move as giving each plan phase an exit check, as in our post on agent plan granularity.

Checklist Before You Delegate

  • Oracle: can a test, build, metric, diff or script say pass or fail?
  • Cheap verification: if not, can a human check each output in seconds?
  • Adjectives: does the task say "good", "clean", "adequate" or "safe"? Replace each with a check or threshold.
  • Blast radius: is the first pass read-only, with changes as drafts?
  • Evidence: will the agent show the command it ran and its output?
  • Rules: about to add a prompt rule? Make it a check, a hook, or an example.
  • Hidden constraints: do structural rules exist that tests miss? See our post on constraint decay.

Where This Fits in Akmatori

Incident investigation sits closer to "fuzzy search plus cheap verification" than to "make it adequate". As the architecture docs describe, every Akmatori incident run searches runbooks, then cross-incident memory, then recent change events, and only then calls infrastructure tools. SSH commands pass the connector's denied and allowed command patterns and then read-only rules: hard rules in code, not sentences in a prompt. An agent's causal claim is saved as memory graded unverified, a lead to check against live data.

Conclusion

Give an agent tasks where something other than you decides "done": ports with tests, fuzzing with coverage, tests with filters, search with cheap checks, tuning against a metric. For everything else, define "done" first. When the agent fails, skip the next rule: add a check, move the rule into code, or show an example.

If your team wants AI-assisted incident workflows with clear checks and read-only first steps, Akmatori helps SRE teams investigate alerts, coordinate response, and automate safe infrastructure actions. Powered by Gcore for global infrastructure reliability.

Automate incident response and prevent on-call burnout with AI-driven agents!