Prices checked 28 October 2026: standard (pay-as-you-go) API tier, US dollars per million tokens, from each provider’s official pricing page unless stated otherwise.
Kimi K3.1 Pro costs half as much per token as Claude Opus 5.5: $2 per million input tokens and $10 per million output tokens, against $4 and $20. On Artificial Analysis’s Intelligence Index runs, Kimi still cost more per task: $2.11 against $1.82 for Opus 5.5 at High effort. It produced 55,000 output tokens per task to Opus’s 36,000.
That’s the mistake a price table invites. A per-token rate tells you what one unit of text costs. It doesn’t tell you how many units a model will spend on your job, or how often the job comes back usable. LLM cost per task captures both.
LLM cost per task is the total API spend on every attempt at a set of tasks, divided by the number of attempts that met your acceptance criteria.
The rest of this article is about calculating that number correctly, and about knowing when to trust it.
Key Takeaways
- Price per token and cost per task can rank models in opposite orders. Kimi K3.1 Pro is half Opus 5.5’s per-token price but cost about 16% more per Artificial Analysis task.
- Reasoning tokens are billed as output by OpenAI, Anthropic and Google, even when you never see them. Anthropic also bills earlier turns’ thinking as input on its newest models.
- Caching can swing the bill several-fold. In one public test, a broken cache made a single run cost $17.60 instead of $2.62.
- Divide by successes, not attempts. Failed runs cost money and produce nothing.
- Twenty runs can’t separate close results. An 80% and a 65% pass rate from 20 runs each have overlapping confidence intervals.
- Price changes can settle a close call. One developer test found Gemini 4 Argon roughly level with Opus 5.5 per successful fix. At Argon’s December 1 prices, it would cost about twice as much.
Quick Navigation
- The LLM cost per task formula
- From cost per attempt to LLM cost per successful task
- A public test: 20 runs, four models
- How caching and reasoning change LLM cost per task
- What published comparisons actually measured
- How to measure LLM cost per task on your own workload
- A decision rule for LLM cost per task
- Frequently asked questions
The LLM cost per task formula
For one attempt, the API cost is the sum of each token category times its rate, with token counts converted to millions:
Attempt cost = (fresh input tokens ÷ 1,000,000 × fresh-input rate)
+ (cache-read tokens ÷ 1,000,000 × cache-read rate)
+ (cache-write tokens ÷ 1,000,000 × cache-write rate)
+ (billable output ÷ 1,000,000 × output rate)
+ per-call charges (web search, code containers, grounding)Then, across a test set:
LLM cost per successful task = sum of all attempt costs ÷ number of successful attempts
Each line has its own billing rules, and they differ by provider. This is the buyer’s cost. What it costs a provider to serve those tokens, and its margin, is a separate question, covered in what inference actually costs per token.
What each line costs at OpenAI, Anthropic and Google
| Line item | OpenAI (GPT-5.6 and later) | Anthropic (Claude 5.5 models) | Google (Gemini 4 Argon) |
|---|---|---|---|
| Fresh input | List rate | List rate | List rate |
| Cache read | 0.1× input; 0.05× on GPT-6.1 Sol and Terra | 0.1× input; 0.05× on Opus 5.5 and Sonnet 5.5 | $0.10 per million (0.05× input) |
| Cache write | 1.25× input (30-minute) or 2.5× (12-hour) | 1.25× input (5-minute) or 2× (1-hour) | No separate charge listed; caching is implicit only |
| Reasoning tokens | Billed as output | Billed as output, including when thinking text is hidden | Billed as output (“incl. thinking”) |
| Search tools | $10 per 1,000 web searches | $10 per 1,000 web searches | $14 per 1,000 grounded queries after 5,000 free a month |
Sources: OpenAI pricing and prompt caching; Anthropic pricing and prompt caching; Gemini API pricing.
Two details catch people out. Anthropic’s thinking defaults to a display mode of “omitted” on Opus 5.5 and Sonnet 5.5, but its thinking documentation is explicit: “You’re still charged for the full thinking tokens. Omitting reduces latency, not cost.” On those models, previous turns’ thinking blocks also stay in context and are billed as input, so a long agent session pays for old reasoning again on every turn.
Reading usage fields without double-counting
A common error in home-made calculators is adding a field that already contains another one.
- Anthropic:
input_tokenscounts only the tokens after the last cache breakpoint. Total input iscache_read_input_tokens + cache_creation_input_tokens + input_tokens, and each part takes its own rate. - OpenAI:
cached_tokensexcludes tokens charged as writes, which appear incache_write_tokens. Reasoning tokens sit insideoutput_tokens, underoutput_tokens_details.reasoning_tokens. Don’t add them again. - Truncated responses: with reasoning models, a request that hits its output limit can bill input and reasoning tokens and return no visible answer, as OpenAI’s reasoning guide warns. That attempt still counts toward the cost.
Then reconcile your summed usage against the provider’s invoice for the same period. In July 2026, developers on OpenAI’s forum reported that GPT-5.6 output-token counts were several times higher than the text actually generated. OpenAI didn’t publicly confirm a defect in that thread. Whatever happened there, the invoice is the figure your finance team will pay.
From cost per attempt to LLM cost per successful task
The numerator is everything you paid to get results. That includes failed attempts, automatic retries, intermediate calls inside an agent loop, any model you use as a judge and per-call tool fees. Count each API call once, at the moment it happens. If your harness logs both an agent step and its parent run, sum only one of them.
The denominator is the number of attempts that passed a fixed acceptance test, written before the run. “The tests pass” is a test. “The output looked fine” is not. The same criteria must apply to every model, or the division compares different jobs.
There are two valid ways to define a task:
- Independent attempts. Each run is scored on its own, which is what most benchmark-style comparisons do.
- Tasks with a retry policy. A task succeeds if any attempt within, say, three tries passes. Then the cost per task is all spend divided by tasks completed. That matches production better, but pick one definition and use it for every model.
API cost is not the whole bill. Human review, retrieval infrastructure, observability and sandboxes belong in a fully loaded cost per task whenever models differ in how much of that work they create. A model whose failures are obvious and cheap to discard can beat one whose failures look plausible and need a reviewer to catch.
A public test: 20 runs, four models
On 14 October 2026, a developer writing at paddo.dev replayed six merged changes from a production monorepo against four models. Each model got 20 runs (three or four per change) at medium effort in the Claude Code harness. Gemini 4 Argon was routed through OpenRouter. A run passed if the tests that shipped with the original fix passed. Costs were calculated at list prices.
| Model | Runs | Passing runs | Total cost | Cost per run | Cost per passing run |
|---|---|---|---|---|---|
| Gemini 4 Argon | 20 | 16 | $57.68 | $2.88 | $3.61 |
| Claude Opus 5.5 | 20 | 13 | $45.81 | $2.29 | $3.52 |
| Claude Sonnet 5.5 | 20 | 11 | $11.76 | $0.59 | $1.07 |
| GPT-6.1 Sol | 20 | 11 | $6.73 | $0.34 | $0.61 |
The totals and pass counts come from the post. The last two columns are our arithmetic, and they check: $57.68 ÷ 16 = $3.605, and $45.81 ÷ 13 = $3.524.
Argon and Opus are close, nine cents apart per passing run. Argon’s per-token prices matched Sonnet 5.5’s at the time ($2 input, $10 output), yet it cost five times as much as Sonnet per run. It wrote 1.37 million output tokens across the 20 runs, 2.8 times as many as Opus.
GPT-6.1 Sol had the lowest cost per passing run, but the post also records five Sol runs that broke a previously passing test, against none for the others. It doesn’t say whether those runs are among Sol’s 11 passes. If “breaks nothing” is part of your acceptance test, Sol’s denominator may shrink, which is why the test has to be written down before anyone divides.
How much a 20-run test can tell you
Not much, at the margin. A 95% Wilson confidence interval on 16 passes out of 20 is about 58% to 92%; for 13 out of 20, about 43% to 82%. The ranges overlap heavily, so a rerun could reverse the order. And six changes from one codebase, with correlated runs on each change, say little about a different stack.
What the December repricing does
Google’s pricing page doubles every Argon token rate on 1 December 2026: input from $2 to $4, cached input from $0.10 to $0.20, output from $10 to $20. If the runs used identical tokens, Argon’s total would become $115.36, or $7.21 per passing run, about twice Opus’s $3.52. We covered the change itself in Gemini 4 Argon pricing doubles Dec 1.
A simple break-even check shows how far the decision moves. One model is cheaper per success than another when its pass rate exceeds the other’s pass rate multiplied by the ratio of their per-attempt costs. Before December, Argon needed a pass rate above 65% × 1.26 ≈ 82% to beat Opus; it scored 80%. After December, it would need 65% × 2.52 ≈ 164%, which is impossible. Even at the bottom of Opus’s confidence interval, Argon would lose. On this test, the price change settles a question the sample size couldn’t.
How caching and reasoning change LLM cost per task
The same post found a stark caching failure. The first Argon run cost $17.60 because it read 8.4 million input tokens and only 59,000 came from cache. The harness inserted system messages that changed the prompt prefix on every call, so Gemini’s implicit cache never matched. After a fix, the same run cost $2.62, with 94% of input served from cache.
The second example is illustrative. The token counts and pass rates are assumptions, not measurements of any real model. Two models attempt the same agent task:
- Model X: $1 input, $0.05 cache read, $1.25 cache write, $5 output, per million tokens. Pass rate 60%.
- Model Y: $4 input, $0.20 cache read, $5 cache write, $20 output. Pass rate 85%.
- Each attempt sends 300,000 input tokens over its turns. When caching works, 270,000 are cache reads and 30,000 are cache writes.
- Model Y produces 20,000 output tokens per attempt, including reasoning. Model X produces 60,000.
| Scenario | Cost per attempt | Pass rate | Cost per successful task |
|---|---|---|---|
| Model X, cache broken (all 300,000 input tokens fresh) | $0.600 | 60% | $1.00 |
| Model X, cache working | $0.351 | 60% | $0.585 |
| Model X, cache working, reasoning doubles to 120,000 output tokens | $0.651 | 60% | $1.085 |
| Model Y, cache working | $0.604 | 85% | $0.711 |
The calculation for the second row: 270,000 × $0.05 + 30,000 × $1.25 + 60,000 × $5, each divided by one million, gives $0.0135 + $0.0375 + $0.30 = $0.351. Dividing by 0.60 gives $0.585.
With caching working, the cheaper model wins. Break the cache, or let it reason twice as long, and the expensive model wins instead. Neither change shows up in a price table.
What published comparisons actually measured
Before using a published cost-per-task figure, check the effort setting, what counted as a task, and whether success was part of the number.
- Effort settings. Put Gemini 4 Argon’s 36,000 reasoning tokens per task next to Opus 5.5’s 84,000, and Argon looks far leaner. Both figures appear on Artificial Analysis, but they come from different settings: Argon at High effort and Opus 5.5 at Max. At matched High effort, the comparison reverses: Argon used 36,000 reasoning tokens per task to Opus’s 18,000, and 62,000 output tokens in total to Opus’s 36,000. Argon still cost less per task, $1.62 against $1.82, because its tokens were half the price at the time.
- Vendor claims. OpenAI launched GPT-6.1 Terra on 26 October at $1 input and $5 output. DataCamp’s write-up of the announcement says that at max effort Terra is over three times as token-efficient as Opus 5.5 and a quarter of the price per token, and it puts the combined effect at “9x lower cost per task”, attributed to OpenAI. No benchmark is named, and we couldn’t open OpenAI’s post to check. Three times the efficiency at a quarter of the price would multiply to about 12 times, not 9, so the figure depends on a token breakdown that hasn’t been published. Treat it as a vendor claim to test, not a result.
- Cost per task is not cost per success. Artificial Analysis computes cost per task from token counts at list prices. It doesn’t divide by how many answers were right. A model that is cheaper per task but scores lower may still cost more per correct answer. As we argued in why benchmark scores don’t transfer to your workload, these numbers are good for building a shortlist, not for making the final decision.
How to measure LLM cost per task on your own workload
- Pick 30 to 50 real tasks from production traffic, covering easy, typical and hard cases. More tasks beat more runs of the same task.
- Write the acceptance test first: unit tests, a schema check or a fixed-judge rubric, identical for every model.
- Fix the configuration: prompts, tools, effort setting, context limits and retry policy.
- Log raw usage for every call, including retries and sub-agents, with each token field stored separately.
- Price the logs offline against a dated copy of each pricing page, so you can reprice them when rates change.
- Check cache hit rates per model. A low rate in a repetitive session usually means the prompt prefix changes between calls.
- Reconcile with the invoice for the test window.
- Report ranges: pass rate with a confidence interval, cost per attempt, cost per successful task and latency.
Run at least two attempts per task. That shows whether failures are random or the same task fails every time.
A decision rule for LLM cost per task
Compare models on cost per successful task, after deciding what a success is worth. Then apply three checks.
- If the confidence intervals on pass rate overlap, treat the models as tied on quality and choose on cost per attempt, latency and operational fit.
- If one model’s cost per attempt is k times another’s, it needs a pass rate more than k times higher to win. When that would take a pass rate above 100%, stop testing.
- If a success is worth far more than the API cost, as with a merged fix that saves an engineer an hour, pass rate and reviewer time outweigh token prices. Optimise for the fewest failures, then for cost.
Per-token prices still have a job. They tell you where to look first. LLM cost per task tells you what you’ll actually pay.
Frequently asked questions
What is LLM cost per task?
It is the total API spend on all attempts at a set of tasks, divided by the number of attempts that met a fixed acceptance test. Failed attempts, retries, intermediate agent calls and judge-model calls stay in the total, because you paid for them. A broader, fully loaded version adds review time and infrastructure.
How do I calculate LLM API cost for one request?
Multiply each token category by its rate per million tokens: fresh input, cache reads, cache writes and output, including reasoning. Then add per-call tool fees such as web search. Take the counts from the response’s usage object, and check that one field doesn’t already include another before adding them.
Are reasoning tokens billed?
Yes. OpenAI, Anthropic and Google bill reasoning or thinking tokens at the output rate, even when the reasoning text isn’t returned. On Anthropic’s Opus 5.5 and Sonnet 5.5, previous turns’ thinking blocks are also billed as input when they stay in the conversation.
How much does prompt caching save?
Cache reads cost between 5% and 10% of the input rate at OpenAI, Anthropic and Google for current models. OpenAI and Anthropic charge 1.25 times the input rate or more to write to the cache. Savings depend on how much of each request repeats an earlier prefix, so measure the hit rate rather than assuming it.
Is the cheapest model per token ever the cheapest per task?
Often, yes. On Artificial Analysis’s figures, Gemini 4 Argon at High effort cost less per task than Opus 5.5 at High, while its tokens were half the price. The ranking flips when the cheaper model uses many more tokens, misses the cache or fails more often. Only measurement on your own tasks tells you which case you’re in.
Keep reading
Here are the latest posts from the blog.
Enterprise AI Spending Hit $177B. Agents Are Driving the Bill
Machine Unlearning: Can We Really Trust an Edited AI Model?
Your Robot May Pass Every Test. Its Simulator Could Still Be Lying
