MAI Code 2 vs Opus 5.5: How to Test the 96% at 5% Claim on Your Own Tasks

On November 17, 2026, at Ignite, Microsoft AI introduced MAI Code 2 and MAI Code 2 Flash. The headline: MAI Code 2 delivers "96%" of Claude Opus 5.5's performance at "just 5%" of the cost, averaged over four coding benchmarks. Against GPT-6.1 Sol, Microsoft claims 98% of the performance at 40% of the cost.
For an agent that picks a model per task, a 20x cost gap is a routing decision. This post takes the claim apart, shows what changes at list price, gives a checklist to reproduce it, and looks at the MAI Licensed Weights offer and lock-in.
How Microsoft Built the Number
The announcement and its footnotes give the method:
- Four benchmarks: DeepSWE 1.1, Terminal Bench 2.1, SWE-Atlas and FrontierCode v1.1. Microsoft says they "target the everyday enterprise coding tasks we see in GitHub Copilot."
- Microsoft's harness: the first three were "run on MSBench (Microsoft's internal eval platform) with the GitHub Copilot CLI without subagents." FrontierCode scores and costs come from Artificial Analysis.
- Max effort: "All models were run at max reasoning effort."
- Discounted price: "MAI Code 2 cost is shown at its 50% promotional price."
The post does not say how many trials ran per task or how cost per task was computed. It also says the models are "explicitly not yet tuned for harder problems, like fundamental science research and very long-horizon agentic work."
InfoWorld's report lists Microsoft's per-benchmark scores and per-task costs behind the averages:
| Benchmark | MAI Code 2 | Claude Opus 5.5 | MAI as % of Opus |
|---|---|---|---|
| DeepSWE 1.1 | 73.2% | 69.0% | 106% |
| Terminal Bench 2.1 | 84.6% | 78.7% | 107% |
| SWE-Atlas | 43.1% | 57.5% | 75% |
| FrontierCode v1.1 | 52.0% | 56.0% | 93% |
| Simple average | 63.2% | 65.3% | 96.8% |
| Average cost per task | $0.309 (promo) | $6.306 | 4.9% |
The averages are plain means of the four scores, and 63.2 / 65.3 is 96.8%, which Microsoft reports as 96%. But the average hides the spread. MAI Code 2 beats Opus 5.5 on two benchmarks and trails it by 14 points on SWE-Atlas. If your work looks more like SWE-Atlas than Terminal Bench, the number for you is closer to 75% than 96%.
What the 5% Becomes at List Price
The GitHub Copilot models and pricing page lists the token prices, per million tokens:
| Model | Input | Cached input | Output |
|---|---|---|---|
| MAI Code 2, promo until May 17, 2027 | $0.50 | $0.025 | $2.50 |
| MAI Code 2, after the promo | $1.00 | $0.05 | $5.00 |
| Claude Opus 5.5 | $4.00 | $0.20 | $20.00 |
The Opus 5.5 prices match Anthropic's own pricing page.
Two things follow from this table.
- At list price, the claim is about 10%, not 5%. Double $0.309 and you get $0.618 per task, or 9.8% of Opus. Budget for this number after May 17, 2027.
- Most of the gap is token volume, not token price. The promo price is exactly 12.5% of Opus on input, cached input and output. A per-task cost of 4.9% means that, in Microsoft's runs, MAI Code 2 billed about 40% of the tokens Opus 5.5 did, at max effort inside the Copilot CLI. In your harness and at your effort level, that ratio can differ. Measure it first.
Where It Is Available
The GitHub changelog lists MAI Code 2 for "Copilot Pro, Pro+, Max, Business, and Enterprise" and says both models are available "to eligible GitHub Copilot subscribers on annual plans." They are billed under usage-based billing, and Business and Enterprise admins control access through the model policy.
Outside Copilot, access is narrower. The Microsoft AI post says "Code 2 Flash is in limited-access preview in Microsoft Foundry" and that OpenRouter, Vercel, Hermes, OpenCode and Kilo will come "soon." If your agent calls models through an API, you may not be able to test MAI Code 2 in your own harness yet.
A Checklist for Reproducing the Claim
Run this before you move any traffic. It fits any "X% of model Y at Z% of the price" claim.
- Same prompts, harness and tools. Run both models through your production agent with the same system prompt, skills and tool definitions. Microsoft's numbers come from the Copilot CLI without subagents; if your agent differs, they do not transfer.
- Your effort level, not max. If production runs at
mediumto control cost, test atmedium. Scores and token counts both move with effort. - Several trials per task. Microsoft's announcement states no trial count. Run each task at least three times and count tasks that pass every time, not the best run.
- Count retries and tool calls. Log turns, tool calls, tool errors and timeouts. A model that needs twice the tool calls also needs more wall-clock time, and in an incident that matters.
- Cost per solved task, at two prices. Divide total spend, failed runs included, by tasks solved, at the promo price and at the price after May 17, 2027. Include cached input: an agent re-reads its context every turn.
- A held-out set. Public benchmarks may have leaked into training data, and a set you tune prompts on stops being a fair test. Keep 20 to 50 tasks from your own tickets and postmortems that you never tune on.
- Break the result down. Report per task type, like the table above, not one average.
- Price a failure. Add the cost of a wrong answer: engineer time, a bad change, a second incident.
The rule from our post on where agent tokens go applies here: judge models by cost per solved task, not by cost per token.
Worked Example: Cost per Solved Incident
Take an incident agent and value a failed investigation at $50 of on-call time. Use Microsoft's per-task costs as model cost, and two sets of success rates: Microsoft's average, and a hypothetical set for a team whose tasks look like SWE-Atlas.
| Scenario | Model | Model cost | Success | Expected cost per task |
|---|---|---|---|---|
| Microsoft average, promo | MAI Code 2 | $0.31 | 63.2% | $0.31 + 0.368 × $50 = $18.71 |
| Microsoft average | Opus 5.5 | $6.31 | 65.3% | $6.31 + 0.347 × $50 = $23.66 |
| SWE-Atlas-like, list price | MAI Code 2 | $0.62 | 43.1% | $0.62 + 0.569 × $50 = $29.07 |
| SWE-Atlas-like | Opus 5.5 | $6.31 | 57.5% | $6.31 + 0.425 × $50 = $27.56 |
On Microsoft's average, MAI Code 2 wins even when failures are expensive. On the task type where it trails, Opus 5.5 is cheaper per solved task despite costing ten times more per run. At $50 per failure, each point of success rate is worth $0.50 per task, so the $5.69 list-price gap equals about 11 points. Measure whether the gap on your tasks is larger.
Licensed Weights and Lock-In
Microsoft also announced MAI Licensed Weights, "giving organizations the ability to deploy and customize Microsoft AI models in their own Azure environments." Customers "retain control over how they're deployed, who can access them, and when they're upgraded," and Microsoft says "Your data and adaptations remain yours, with Microsoft only using them to train its models with your express agreement." The program "is coming soon through our Frontier Company engagements"; the page asks customers to contact their Microsoft account team. The customer results on the page "reflect customer-specific evaluations."
That is a real option for teams that want a model tuned on their own code. It is also a deeper commitment than a model picker:
| Path | What you can switch away from easily | What ties you in |
|---|---|---|
| MAI Code 2 in GitHub Copilot | The model: pick another in the model picker | The Copilot subscription and its annual plan |
| MAI Code 2 through an API, when it ships | The model and the provider, if your harness is provider-neutral | Prompts tuned to one model |
| Licensed weights tuned on your data | Little, until you have an exit plan | Azure hosting, the Frontier Company contract, a tuned model you cannot compare without your own evals |
The announcement does not say whether tuned weights can leave Azure, what happens to them when the contract ends, or how a tuning carries over to the next base model. Ask those three questions before you sign. Keep the held-out set and harness vendor-neutral: they are how you prove later that a tuned model still beats the next one off the shelf.
Where Akmatori Fits
Akmatori runs one active LLM configuration for all agent runs (PUT /api/settings/llm/:id/activate in the API reference), each with its own provider, model and thinking_level. GET /api/incidents/:uuid returns an incident's agent runs with token count and execution time: the data the checklist needs. Make one model active for a class of staging alerts, then the other, and compare cost and correct root causes per incident. MAI Code 2 is not on Akmatori's providers today; Microsoft lists OpenRouter as "soon." For model comparisons on real incidents, see SRE Bench.
Conclusion
Microsoft's numbers add up as stated: a simple average over four chosen benchmarks, run in Microsoft's own harness at max effort, with MAI Code 2 at half price. At list price the cost ratio is about 10%, most of the saving comes from token volume, and on SWE-Atlas MAI Code 2 reaches only 75% of Opus 5.5. Run the checklist on your own held-out tasks, decide by cost per solved task at the price after May 17, 2027, and get exit terms before you tune weights on Azure.
If your team wants AI-assisted incident workflows that you can route by cost per solved task, Akmatori helps SRE teams investigate alerts, coordinate response, and automate safe infrastructure actions. Powered by Gcore for global infrastructure reliability.
