The seductive number is the hourly rental rate. An H200 rents for roughly $3.10 to $3.80 per GPU-hour from the cheaper providers, which works out to about $2,300 to $2,800 a month running continuously.
Set that against a five-figure API bill and the conclusion looks obvious.
The conclusion is usually wrong, and it is wrong for a specific reason: the GPU rate prices one input to a system that has several. Nobody bills you separately for the rest, so they do not appear on any invoice you can point at.
That is what makes this failure mode persistent. An API bill is a single line item that captures the entire cost of the capability. A self-hosting bill is a single line item that captures perhaps a third of it, with the remainder distributed across salaries, unused capacity and outages that never get attributed back to the decision.
The per-token arithmetic underneath all of this — what a token costs to serve, and why that differs from what you are charged — is covered in what inference actually costs per token.
Key Takeaways
- Published break-even points for self-hosting range from 2 million tokens per day to 11 billion tokens per month. Both figures are defensible, because each assumes a different comparison API that the article usually does not name.
- Against a frontier API, one H200 breaks even around 0.5 billion tokens per month. Against a budget API, break-even requires 33 billion — more than four times what that GPU can physically produce.
- Self-hosting does not have a break-even point. It has a break-even point against a specific alternative, and the spread between them is roughly 123×.
- The raw GPU rate covers 20–40% of true cost. Credible estimates of the full multiplier cluster between 2.5× and 3×, with a defensible range of 1.3× to 5×.
- A GPU at 10% utilisation costs ten times as much per token as the same GPU at full load. Utilisation moves the answer more than hardware choice does.
Quick Navigation
- The 5 Hidden Layers of Self-Hosted LLM Cost
- What the Self-Hosted LLM Cost Multiplier Really Is
- Why Self-Hosted LLM Cost Break-Evens Disagree
- The Comparison API Decides Your Self-Hosted LLM Cost
- When Break-Even Is Physically Unreachable
- Utilisation and Self-Hosted LLM Cost
- When Self-Hosted LLM Cost Actually Wins
- The Hybrid Option Most Comparisons Ignore
- Calculating Your Own Self-Hosted LLM Cost
- Primary sources
- Frequently Asked Questions
The 5 Hidden Layers of Self-Hosted LLM Cost
Five categories sit outside the GPU line and account for most of the gap.
Layer 1 — Engineering time. Somebody configures the serving stack, tunes batching, manages model weights, handles version upgrades, and debugs the memory error at 2am. On a loaded engineering salary, a fraction of one full-time role can exceed the GPU rental itself. Teams without GPU operations experience typically need consulting or managed support through the first quarter.
Layer 2 — Idle capacity. APIs cost nothing when nobody is using them. A rented GPU bills identically at 3am on a Sunday and at peak load on a Tuesday. Production traffic is never flat, and the trough is billed at the same rate as the peak.
Layer 3 — Redundancy. One GPU is a single point of failure. Production reliability means a second instance, which doubles the hardware line before you have served a single additional token. APIs include redundancy in the quoted price.
Layer 4 — The surrounding infrastructure. Load balancing, monitoring, logging, model storage, networking egress, and the observability stack that makes any of it debuggable. Raw GPU costs represent roughly 30–40% of true infrastructure investment.
Layer 5 — Model refresh. Open-weight models improve every few months. Evaluating, migrating and re-tuning against a new release is recurring engineering work. On an API, the provider absorbs it and you get the improvement in a version string.
None of these are exotic. All of them are routinely omitted from the comparison that drives the decision.
What the Self-Hosted LLM Cost Multiplier Really Is
Published estimates of the total multiplier vary, and the variance is narrower than you might expect.
| Source estimate | Multiplier on raw GPU cost |
|---|---|
| Conservative | 1.3× |
| Hidden costs adding 20–40% | ~1.4× |
| Common mid-range | 2.0× |
| Infrastructure stack analyses | 2.5–3.0× |
| Full TCO with DevOps and downtime | 3.0–5.0× |
Applied to an H200 at $3.50 per hour — roughly $2,555 per month raw:
| Multiplier | All-in monthly cost |
|---|---|
| 1.3× | $3,322 |
| 2.0× | $5,110 |
| 2.5× | $6,388 |
| 3.0× | $7,665 |
| 5.0× | $12,775 |
The rest of this article uses 2.5× — $6,388 per month — because it sits in the middle of the credible range. Substitute your own multiplier; the structure of the argument does not change.
The 5× figure typically reflects deployments with dedicated engineering, redundancy and low utilisation. The 1.3× figure typically reflects a well-utilized single GPU run by a team that already had the skills. Both are honest; they describe different situations.
Why Self-Hosted LLM Cost Break-Evens Disagree
Here is the finding that motivated this article.
Search for the break-even point and you will find, from credible 2026 sources: 2–5 million tokens per day. 5–10 million tokens per month. 100–256 million tokens per month. 500 million tokens per month. 11 billion tokens per month.
That is roughly a 2,000× spread across published figures.
They are not contradicting each other. They are answering different questions and rarely saying so.
Self-hosting does not have a break-even point. It has a break-even point against a specific alternative.
An API bill scales linearly with volume. A self-hosted bill is fixed. Break-even is where the line crosses the constant, and the slope of that line is entirely determined by which API you picked as the comparison.
Any article stating a break-even without naming the comparison model has left out the variable that determines the answer.
The Comparison API Decides Your Self-Hosted LLM Cost
Run the arithmetic. Fixed self-hosted cost of $6,388 per month, divided by each API’s blended rate at a typical 1,000-in/500-out request shape.
| Comparison API | Blended rate | Break-even volume |
|---|---|---|
| Claude Fable 5 | $23.33/M | 274M tokens/month |
| GPT-5.6 Sol | $13.33/M | 479M tokens/month |
| Claude Opus 5 | $11.67/M | 547M tokens/month |
| Claude Sonnet 5 (Sept) | $7.00/M | 912M tokens/month |
| Claude Haiku 4.5 | $2.33/M | 2.74B tokens/month |
| GPT-5.6 Luna | $0.53/M | 12.05B tokens/month |
| DeepSeek V4-Flash | $0.19/M | 33.62B tokens/month |
A 123× spread in break-even volume, driven entirely by the comparison choice. Same GPU, same cost model, same arithmetic.
This resolves the published disagreement completely. Analyses reporting low break-evens compared against frontier models. Analyses reporting high break-evens compared against budget or open-weight hosted APIs. Both were right about their own question.
The practical implication is uncomfortable for the usual framing. The decision is rarely “self-host or use an API.” It is “self-host, or use the cheapest API that meets our quality bar.” Managed open-weight providers occupy that middle tier, and they are the comparison that actually threatens the self-hosting case.
When Break-Even Is Physically Unreachable
Two rows in that table are worse than expensive. They are arithmetically impossible.
A single H200 sustaining 3,000 tokens per second at 100% utilisation produces about 7.88 billion tokens per month. That is a generous ceiling — it assumes continuous high-batch operation with no idle time, which no production workload achieves.
Compare that ceiling to the break-even requirements:
| Comparison API | Break-even needs | One H200 can produce | Verdict |
|---|---|---|---|
| Claude Opus 5 | 0.55B | 7.88B | Reachable |
| Claude Haiku 4.5 | 2.74B | 7.88B | Reachable |
| GPT-5.6 Luna | 12.05B | 7.88B | Impossible |
| DeepSeek V4-Flash | 33.62B | 7.88B | Impossible |
Against a budget API, one GPU cannot break even at any volume, because the volume required exceeds what the hardware can physically emit. Adding GPUs does not help — it raises the fixed cost proportionally, so the ratio holds.
The throughput ceiling is not a tuning problem. It follows from bandwidth divided by bytes moved per token, as set out in memory bandwidth and the limits of AI chips.
This is the single most useful check available before a self-hosting decision, and it takes two minutes: compute your break-even volume, compute your hardware’s physical ceiling, and confirm the first is smaller than the second.
Utilisation and Self-Hosted LLM Cost
Every figure above assumes the GPU stays busy. That assumption fails routinely.
| Utilisation | Effective cost multiple |
|---|---|
| 100% | 1× |
| 50% | 2× |
| 25% | 4× |
| 10% | 10× |
A GPU at 10% load inflates per-token cost tenfold, converting an asset into a liability billed by the hour.
Production traffic has daily peaks, weekend troughs and quiet nights. Unless you are backfilling the gaps with offline batch work, average utilisation on a dedicated instance is frequently below 30%.
Note how this compounds with the multiplier. At 2.5× hidden costs and 30% utilisation, your effective cost is roughly 8× the raw GPU rate. That is the number to compare against an API bill, and it is not what appears on the rental invoice.
The honest test for self-hosting has never really been about model quality or hourly rates. It is whether you can keep the GPU busy.
When Self-Hosted LLM Cost Actually Wins
Three situations where self-hosting is the right answer, and they are narrower than the discourse suggests.
High, predictable volume against premium models. If you genuinely need frontier-class quality and run above roughly 0.5 billion tokens a month with steady traffic, the arithmetic favors you. One worked scenario: $36,000 of hardware against $7,500–15,000 of monthly frontier API spend breaks even in six to seven months.
Data residency and regulatory constraints. Healthcare under HIPAA, financial services under SOC 2, government contracts, and any deployment where data cannot leave your infrastructure. Here cost is not the deciding variable, and self-hosting can be correct at any volume.
Latency floors an API cannot meet. Network round-trip becomes material in a tight interactive loop. Local inference removes it.
Outside those three, the arithmetic usually points the other way. One analysis put it starkly: at 50 million tokens per day, a budget API cost around $2,250 per month while the same workload self-hosted on four A10G GPUs cost $5,175 — the “cheaper” route costing 2.3× more.
The sensible default sequence: start on APIs, move to a managed open-weight provider as volume grows, and consider owning hardware only when volume is predictable and the comparison at your quality bar still favors it. The layers this decision sits on top of are covered in the AI compute stack.
The Hybrid Option Most Comparisons Ignore
The debate is usually framed as a binary. It rarely is one in practice, and the middle options change the arithmetic more than any hardware choice.
Managed open-weight APIs. Providers serving Llama, Qwen, Mistral and similar models sit between frontier pricing and owned hardware, with blended rates commonly quoted around $0.09 to $0.44 per million tokens. You get open-weight economics without operating anything.
This tier is the reason so many self-hosting business cases collapse under scrutiny. Teams compare owned hardware against a frontier API, find a favourable result, and never test it against the managed provider serving the exact same open model they intended to host.
Complexity-based routing. Send roughly 70% of queries to a budget tier, 20% to mid, and 10% to frontier. Reported savings run above 80% with limited quality impact, because the hard queries still reach the strong model. This changes your blended rate, which changes your break-even, which may remove the case for self-hosting entirely.
Split by workload rather than by volume. Self-host the steady, predictable baseline where utilisation stays high. Burst to an API for peaks. This directly targets the utilisation problem — the fixed asset serves the flat portion of the curve, and the variable-cost provider absorbs the spikes that would otherwise sit idle between them.
Self-host only what needs it. Data residency requirements often apply to one workflow, not the whole product. Running a single regulated pipeline on owned hardware while everything else uses APIs is usually cheaper than treating one constraint as a mandate for the entire stack.
The sequencing that follows from all of this is unglamorous and reliably correct: prove the product on APIs, move to managed open-weight as volume grows, then own hardware only for the specific workloads where the arithmetic still favors it after you have named the real comparison.
Calculating Your Own Self-Hosted LLM Cost
Six steps. An afternoon’s work, and it beats any published break-even figure because it uses your numbers.
- Name your comparison API. Not the most expensive one. The cheapest that clears your quality bar. This single choice moves the answer by two orders of magnitude.
- Compute your blended rate. Take your actual input-to-output ratio and apply it to that API’s pricing. Headline input price will understate your bill by 1.4× to 2.7×.
- Estimate your multiplier honestly. 1.3× if you have GPU operations skills in-house and will run one well-utilized instance. 3× or more with redundancy, dedicated engineering and variable load.
- Check the physical ceiling. Bandwidth divided by bytes per token, times your expected utilisation. If break-even exceeds this, stop — the decision is already made.
- Model your real utilisation curve. Not peak capacity. The average across a week including nights and weekends.
- Re-run quarterly. Prices moved twice in the last six weeks alone on the API side, and GPU rates move with supply. A model built in February is stale by August.
One thing worth stating plainly: if the calculation comes out close, choose the API. A narrow margin does not survive the first outage, the first model refresh, or the first month someone leaves the team.
Primary sources
All break-even figures above are computed from the stated formulas using a $3.50/hour GPU rate and a 2.5× multiplier, and are shown in full so readers can substitute their own inputs. Published multiplier estimates vary; the range is shown rather than a single value.
Frequently Asked Questions
Is self-hosting an LLM cheaper than using an API?
It depends entirely on which API you compare against. Against a frontier model, break-even can arrive around 0.5 billion tokens per month. Against a budget API, break-even may exceed what the hardware can physically produce.
What is the true multiplier on raw GPU cost?
Credible estimates run from 1.3× to 5×, clustering around 2.5–3×. Raw GPU cost typically represents 30–40% of true infrastructure investment once engineering, redundancy, idle time and surrounding infrastructure are included.
Why do published break-even figures vary so much?
Because they compare against different APIs and rarely say which. Figures from 2 million tokens per day to 11 billion tokens per month can all be arithmetically correct for their unstated comparison model.
How does utilisation affect self-hosted LLM cost?
Linearly and severely. A GPU at 10% load costs ten times as much per token as one at 100%, because rental is billed by the hour regardless of use.
When should I self-host regardless of cost?
When data residency or regulatory constraints prohibit sending data to a third party, or when network latency in an interactive loop is unacceptable. In both cases cost is not the deciding variable.
Keep reading
Here are the latest posts from the blog.
Self-Hosted LLM Cost: The 5 Hidden Fees in Your Bill
Egress Control: The 7 Hidden Paths Out of Your Agent
Agent Observability: The 4 Signals Your Stack Must Emit
