AI Infrastructure & Chips

The physical layer under every AI product: accelerators, memory, interconnect, power and what it all costs. Coverage of Nvidia, AMD, TSMC, Google TPU and AWS Trainium, plus the constraints that decide who can actually serve a frontier model — HBM supply, datacentre power, and the gap between a chip’s spec sheet and its cost per token.

AI Coding Agents Are Getting Expensive. Who’s Paying for the Tokens?

by
AI Coding Agent

Pricing checked on official vendor pages on 1 November 2026. All USD. Cost scenarios are illustrative and labelled as such.

On 28 October, TechCrunch published a number from Jellyfish, the engineering analytics company, that should interest any finance team: spending on Claude Code and other Anthropic-powered coding tools has grown 10x since the start of the year and now equals about 2% of total engineering cost. “Even if you’re saying 2% is still small,” Jellyfish chief scientist Nicholas Arcolano told the publication, “that 10x growth is growth that most CFOs didn’t plan for.”

That invoice line is the visible part of AI coding agent cost. The rest shows up in review queues, rework and the hours senior engineers spend supervising agents. The tools now bill like utilities, but their output still has to pass through people. What it costs to generate code and what it costs to deliver accepted software are two different numbers.

Key Takeaways
  • Jellyfish’s figures measure proposed changes and spend, not delivery. Anthropic-powered tools account for nearly a third of proposed changes, and spending on them is up 10x this year to about 2% of engineering cost.
  • Seat prices are floors. Claude Code, Codex and Copilot all bill heavier use at token-based rates, so heavy users cost similar amounts whichever tool they use.
  • Per-developer spend has a long tail. Jellyfish’s May 2026 mean was $248 a month, and the 99th percentile was $2,452.
  • Model choice drives cost more than product choice, because tokens are priced at each model’s rate once allowances run out.
  • Review time can exceed the tool bill, but it’s an opportunity cost on existing salaries, and more generated code adds little delivered software if review capacity stays fixed.
  • Budgets work best with a central floor, pooled team allowances and visible review capacity.

Quick Navigation

What Jellyfish’s data says about AI coding agent cost

Each Jellyfish figure in the TechCrunch report measures something different. First, Claude Code and other tools built on Anthropic’s models were responsible for “nearly a third of all proposed code changes submitted for review.” The denominator is proposed changes, essentially pull requests opened, not merged code or completed work. Second, spending on those tools rose 10x since January, which is a growth rate, not a level. Third, that spending equals about 2% of total engineering cost, which includes salaries. TechCrunch describes the base as 276,000 of Jellyfish’s customers. Jellyfish’s own September release describes its dataset as more than 275,000 software engineers at over 1,000 companies, so the figure most likely counts individual engineers rather than companies.

The same report captured a shift between vendors. “It took a long time for us to see Codex really pop in our data at all,” said Nikolas Albarran, Jellyfish’s AI product lead, “and it has come up with Astra,” OpenAI’s GPT-6 model. He added that for teams standardised on Anthropic, “there’s not a big enough reason to switch.”

Jellyfish’s State of AI in Software Engineering report for the first half of 2026 fills in the per-person picture. At the median company, the typical user’s monthly AI coding spend rose from $5 in August 2025 to $81 in May 2026. The mean across users in May was $248. The 99th percentile was $2,452. More than half of users spent under $100.

An average per-developer number hides a long tail, and the tail is where budgets break.

Why the seat price is only the floor of AI coding agent cost

A year ago, most coding assistants were priced like software, with a fixed monthly seat. In 2026, all three major tools charge a seat fee that includes a usage allowance, then bill heavier use by the tokens consumed.

Tokens are the units models read and write, roughly three-quarters of a word each. An agent’s token use grows quickly because every step carries context: files read, project instructions, tool results and the conversation so far. Anthropic’s Claude Code cost documentation puts it plainly: “Token costs scale with context size.” It also notes that each Claude Code turn carries file contents, tool calls and multi-step reasoning, “so one debugging session can consume more than a day of chat.”

So the seat price works as a floor. Light users stay inside it. Heavy users pass through it, and from there the model’s token rate sets the marginal cost. Our analysis of cost per completed task explains why the finished task, not the token, is the useful unit.

Claude Code, Codex and Copilot: three billing models

The three products reach similar economics by different routes. The table compares business plans, because that’s where most team spending sits.

Claude Code (Anthropic)Codex (OpenAI)GitHub Copilot
Individual plansPro $20/month ($17 annual), Max $100, Ultra $200Plus $20/month; Pro from $100 (also $200, $500)Pro $10, Pro+ $39, Max $100
Team or business seatTeam: $25/month standard ($20 annual); premium $125 ($100 annual) with 5x usageBusiness: $25/user/month ($20 annual), 2+ usersBusiness: $19/user/month
Enterprise$20/seat/month (annual, 20-seat minimum) plus usage at API ratesContact salesEnterprise: $39/user/month
What the seat includesUsage allowance on five-hour and weekly windowsMessage limits per five-hour window that vary by modelAI credits equal to the seat price (1,900 for Business, 3,900 for Enterprise; 1 credit = $0.01), pooled across the organisation
Beyond the allowanceUsage credits at standard API ratesCredits priced per million tokens; dollar rate per credit set by plan or agreementAdditional credits at model API rates, enabled by default
Admin spend controlsUser, group and org spend limits; pooled group budgets on EnterpriseWorkspace credits; usage and cost review for adminsBudgets at user, cost-centre, org and enterprise level

Sources: Claude pricing, Codex pricing and GitHub’s usage-based billing documentation, all checked on 1 November 2026.

Three charts comparing AI coding agent cost under a fixed subscription

Some details change the arithmetic. GitHub moved Copilot to token-based AI credits on 1 June, with annual individual subscribers switching at renewal. Completions and next-edit suggestions aren’t billed, but chat, the CLI and agents are. Business and Enterprise customers had promotional allowances of $30 and $70 a month until 1 September, so included usage fell this autumn.

OpenAI’s Codex page lists credit rates per million tokens for each model, but not a dollar price per credit, which “depend[s] on your plan or agreement.” Model choice matters a lot: on Plus or standard Business, the page estimates 5 to 45 local messages per five hours on GPT-6 Astra, against 350 to 3,000 on GPT-6 Luna. Anthropic’s Enterprise plan is the plainest version of the structure: a $20 seat, then usage at API rates that “scales with model and task.”

None of these seat prices is comparable to an API price list on its own, because a seat buys an allowance measured in usage windows or credits, not tokens. Per-token rates are in our API pricing comparison.

An AI coding agent cost model for one developer

The model below uses three illustrative usage levels. They aren’t measured data for any company. They’re anchored to two published figures: Anthropic’s statement that enterprise Claude Code deployments average “around $13 per developer per active day and $150-250 per developer per month,” with 90% of users below $30 per active day, and Jellyfish’s May 2026 median-company figure of $81 a month.

Assumptions

  • Usage is metered spend at list API rates, which applies to any tool billing beyond its allowance at those rates.
  • Light: 10 active days at $8, for short tasks and occasional agent sessions.
  • Typical: 15 active days at $13, for daily assistance, debugging and tests.
  • Heavy: 21 active days at $30, for long sessions, large codebases and parallel agents.
  • Seat examples: Claude Enterprise ($20 plus usage) and Copilot Business ($19 including $19 of credits).
  • Review and supervision time: 2, 6 and 15 extra hours a month, valued at an illustrative fully loaded engineering cost of $100 an hour.
Per developer, per monthLightTypicalHeavy
Metered usage at API rates$80$195$630
Claude Enterprise (seat + usage)$100$215$650
Copilot Business (seat covers first $19)$80$195$630
Extra review and supervision time2 h = $2006 h = $60015 h = $1,500
Total attributable cost (Claude example)$300$815$2,150

The method is simple: tool cost = seat + max(0, usage − included credit). Review cost = extra hours × hourly cost. Codex isn’t in the table because OpenAI doesn’t publish a dollar price per credit. For API-key use, Codex follows standard API pricing, so the usage row applies.

Once usage exceeds the allowance, the products converge, because they all charge for tokens at model rates. The model a team runs, and how long its sessions last, matters more than the product.

The review row is larger than the tool row at every level. Those hours aren’t new spending: they come out of salaries the company already pays, so they’re an opportunity cost, not an invoice. Treating them as cash would double-count labour. The model also excludes time the agent saves, which may exceed these hours. It measures gross attributable cost, not net return.

Generated code is not delivered software

Jellyfish’s August analysis of about 16.9 million pull requests found total volume grew 37.6% between January and July. The same study found merge rates for high-impact changes broadly similar across tools: 88.3% for pure-human work, 89.5% for Copilot and 86.7% for Claude Code. Its author cautions that merge rate measures organisational acceptance, not correctness, and that the results are correlational.

Workflow diagram from AI-generated code

Faros AI, which sells engineering analytics, reports a sharper picture in its Q2 2026 telemetry from 22,000 developers. Comparing high and low AI adoption within the same organisations, it found task throughput up 33.7% and PR size up 51%. Median review time rose fivefold, bugs per developer rose 54%, and deployments per week fell 11.7%. These are vendor-reported correlations, not causal estimates, but they point at the same constraint.

Here’s an illustrative example of how that constraint plays out. A ten-person team opens 50 pull requests a week, each taking 45 minutes to review, and has 40 hours a week of review capacity. The 37.5 hours it needs fit. The team adopts coding agents at the typical level: ten seats at $215, or about $2,150 a month. Pull requests rise to 70 a week, and each is 51% larger. If review time scales with size, the team now needs about 79 review hours a week, roughly double its capacity.

Unless engineers move time from building to reviewing, about 35 of those larger PRs get reviewed each week. Measured in units of work, generated output has roughly doubled while reviewed output has risen about 6%. The company pays more, generates twice the code and delivers only slightly more software.

The hidden AI coding agent cost: review, supervision and rework

Review capacity is the obvious bottleneck. Several other costs sit around it.

  • Context switching. A reviewer reading an agent’s 600-line change has to rebuild reasoning the agent never wrote down.
  • Test validation. Someone has to check that agent-written tests test the right thing.
  • Security review. Generated dependencies and infrastructure changes still need human sign-off.
  • Rejected changes. A PR closed unmerged still consumed tokens and review time.
  • Rework. Higher bug rates mean follow-up work that lands weeks later, under another ticket.
  • Maintenance. Every merged line becomes code someone owns.

Supervision is the least visible. Jellyfish found the longest Claude Code sessions, the top 0.1%, ran past 94 minutes, and that low-supervision long runs produced about 11 times the net lines of heavily supervised runs of the same length. More lines with less oversight is efficient if the lines are right and costly if they aren’t. Multi-agent setups multiply the effect: Anthropic’s documentation says agent teams use about seven times the tokens of a standard session when teammates run in plan mode, a pattern our piece on multi-agent token cost examines.

Controlled evidence is thinner. METR’s randomised study of 16 experienced open-source developers, published in July 2025, found early-2025 AI tools made them 19% slower, though they believed they were 20% faster. In February 2026, METR said a follow-up couldn’t produce reliable results, partly because developers wouldn’t work without AI. Neither settles whether today’s agents raise productivity, but both show how easily perceived speed and measured delivery diverge.

How to budget for AI coding agent cost

Large organisations are already finding the shape of this. In October, 404 Media reported, and Forbes followed up, that Accenture centrally funds $29 per employee per month for Claude usage from 1 October. Spending above that is charged to an approved project or team budget. Some employees had reportedly been spending $500 to $1,000 a month. Accenture said reporting that it had “put a firm limit” on access was “incorrect.” The figure, it said, describes one part of its funding approach.

The structure, a central floor with spending above it charged to whoever benefits, is more useful than the number. A consulting firm has billable projects to charge; a product company may not.

Vendors are building the controls. On 28 October, Anthropic made pooled group budgets generally available for Claude Enterprise. Teams share a monthly budget with individual limits as guardrails, and billing admins get alerts at 50%, 75%, 95% and 100%. GitHub pools Copilot credits across an organisation, allows budgets per user, cost centre or organisation, and leaves additional usage on by default, so a heavy month gets billed unless someone turns that off.

A workable budget usually has four parts:

  1. A per-seat baseline for light and typical users.
  2. A pooled allowance for heavy users, owned by a team lead.
  3. Model defaults: the cheaper model for routine work, the frontier model for hard problems.
  4. A review budget in hours, so leaders can see where generated work is waiting.

What to measure before scaling up

Before expanding seats or raising caps, measure these:

  • Spend per developer at the mean, 90th and 99th percentiles.
  • The share of AI-assisted PRs that merge, and how long they wait for review compared with other PRs.
  • Review hours per merged change, before and after adoption.
  • Reverts, escaped defects and incidents linked to AI-assisted changes.
  • Cycle time from first commit to production.
  • Which models are used for which tasks, and what each costs per merged PR.

If tool spend rises while cycle time and review backlog stay flat or worsen, the constraint is review, and more tokens won’t help.

Frequently asked questions

How much does an AI coding agent cost per developer in 2026?

Usually around $100 to $250 a month in tool spend, with a long tail above. Anthropic reports $150 to $250 for enterprise Claude Code users; Jellyfish measured a $248 mean in May 2026.

Is Claude Code more expensive than GitHub Copilot?

At the seat level, Copilot Business is cheaper ($19 against Claude Team’s $25 monthly standard seat). For heavy agent use, both bill tokens at model rates beyond the allowance, so the model used matters more than the product.

How does OpenAI Codex charge for usage?

Codex is included in ChatGPT plans with message limits per five-hour window that vary by model. Extra usage is bought as credits priced per million tokens. Used with an API key, Codex is billed at standard API rates.

What is the difference between AI coding subscriptions and API pricing?

A subscription buys an allowance measured in usage windows or credits. API pricing charges per token with no allowance, so comparing them needs usage assumptions.

Why does AI coding agent cost rise so quickly with heavy use?

Agents resend context on every step, and long sessions, large codebases and parallel agents multiply it. Thinking tokens bill as output.

How can engineering teams control AI coding agent cost?

Set per-user and pooled budgets, default to cheaper models for routine work, clear sessions between tasks, and track spend at the 90th and 99th percentiles.

Does AI-generated code reduce engineering costs?

The evidence is mixed: telemetry shows higher throughput alongside longer reviews and more bugs, and a 2025 randomised study found experienced developers slower. Measure merged work and cycle time in your own team.

How should companies budget for AI coding agent cost?

Combine a per-seat baseline, a pooled allowance for heavy users, model defaults and an explicit review-capacity budget. Treat examples such as Accenture’s $29 central funding as structures, not benchmarks.

Keep reading

Here are the latest posts from the blog.

Types of AI Models

Types of AI Models in 2026: A Field Guide From LLMs to World Models

Model details checked against official announcements, model cards and documentation on 1 November 2026. Line up four AI models that were current at the end …

Read more

AI Coding Agent

AI Coding Agents Are Getting Expensive. Who’s Paying for the Tokens?

Pricing checked on official vendor pages on 1 November 2026. All USD. Cost scenarios are illustrative and labelled as such. On 28 October, TechCrunch published …

Read more

generative AI architecture

Generative AI Architecture: The 7 Layers That Are Hard to Undo

Published 30 October 2026. API prices checked 28–30 October 2026 on each provider’s official pricing page. Cost figures below are labelled as illustrative where they …

Read more

Indirect Prompt Injection

Gemini 4 Carbon Hit 0% on Indirect Prompt Injection. Is That Enough?

When Google announced Gemini 4 Carbon on 28 October, one line in the safety section stood out. Carbon scored 0.0% on Gray Swan’s Indirect Prompt …

Read more

SpaceX’s $40B GPU Lease: The Customer Sets the Price

by
SpaceX GPU

Published 28 October 2026. This is a developing financing story. As of publication, SpaceX had signed a binding commitment letter (24 October), but the master finance lease hadn’t been signed, the fixed rate hadn’t been set, and the deal hadn’t closed. We will update this article when SpaceX discloses pricing or closing.

SpaceX announced on 26 October that it had secured “the largest committed GPU financing”: roughly $40 billion of Nvidia Vera Rubin equipment, to be bought by an Apollo-managed entity and leased to SpaceX. The Form 8-K filed the same day is more careful. It describes a binding commitment letter signed on 24 October, a lease that has not yet been signed, and a fixed rate that will be set “at pricing.”

That missing rate is the most interesting part of the deal. In October, CoreWeave’s unsecured bonds traded at yields near 15%. Earlier in the year, the same company’s best-protected GPU loan carried a fixed rate of about 5.9%. Both are claims on one company’s GPU business. The difference came from who was expected to pay. SpaceX GPU financing will be priced by the same logic, and the documents suggest the investors are underwriting SpaceX far more than they are underwriting the chips.

Key Takeaways
  • Committed, not closed or priced. SpaceX signed a binding commitment letter on 24 October. The fixed rate, the master lease and closing come later, with closing expected before the end of 2026.
  • $40 billion is principal only. Interest and ongoing costs come on top. On illustrative assumptions, each percentage point on the rate adds roughly $1 billion of interest.
  • Rent repays the full hardware cost. Each quarterly payment covers one-twentieth of a tranche’s cost, so investors depend on SpaceX paying far more than on what used GPUs fetch.
  • SpaceX’s contracts end before the lease does. Only about 6% of its $63 billion backlog falls more than three years out. The lease runs to roughly 2032.
  • CoreWeave shows the customer setting the price. Its contract-backed, A3-rated GPU loan carried about 5.9% fixed. Its unsecured notes yielded near 15%. The instruments differ, so the gap reflects structure, not chips.
  • SpaceX’s earlier GPU leases look costlier than its bonds. Our estimate from its 10-Q puts the Valor lease debt at about 10.7% a year, against 5.35% to 6.65% coupons on its unsecured bonds.

Quick Navigation

How the SpaceX GPU financing actually works

The 8-K describes a lessor “managed by Apollo Global Management” that will acquire “graphics processing unit pods and related equipment” and lease them to SpaceX for use at data centres SpaceX selects. Apollo and Goldman Sachs lead the financing parties, with Goldman as sole structuring agent. Funds managed by BlackRock, Goldman Sachs Asset Management and KKR-advised accounts made anchor commitments.

The investors won’t hold a single loan. Bloomberg reported that the debt will take the form of private bonds issued by an entity that sits off SpaceX’s balance sheet. Pimco bought about $4 billion of an “A2” tranche that sits between more senior and more junior slices, according to Bloomberg’s sources. So the risk is layered before it reaches anyone’s portfolio.

The rent formula is the most revealing disclosure. Equipment arrives in tranches, and each tranche gets its own lease schedule of roughly five to five and a quarter years. Rent is paid quarterly in advance and “will generally consist of” one-twentieth of the tranche’s financed cost, plus a fixed-rate charge on the outstanding balance, plus certain ongoing costs.

Twenty payments of one-twentieth each return the full cost of the equipment. The lessor is counting on SpaceX’s payments, not on selling used Rubin racks in 2032. The 8-K is silent on end-of-term ownership, purchase options, guarantees and prepayment. Elon Musk called it on X “primarily callable debt”, but the filing describes no call feature.

The issuing entity may sit off SpaceX’s balance sheet, but the lease won’t. Under US accounting rules, a lessee records a finance lease as a liability once it commences.

SpaceX GPU
What the $40 billion does and doesn’t include

SpaceX “expects aggregate principal payments under the lease schedules over their respective terms to total approximately $40 billion.” That is the equipment cost being financed. The fixed-rate charge and the ongoing costs come on top, and neither has a published figure yet.

The scale matters. SpaceX reported $38.8 billion of long-term debt and capital lease obligations at 30 September in its third-quarter Form 10-Q. Once every tranche has commenced, principal alone runs to about $2 billion a quarter. For context, SpaceX generated $13.1 billion of operating cash flow in the first nine months of 2026 and held $89.9 billion of cash and marketable securities at quarter-end.

Four dates that are easy to confuse
MilestoneDateStatus
Commitment letter signed24 October 2026Done
Announcement and 8-K26 October 2026Done
Pricing (fixed rate set)Before the master lease is signedNot yet disclosed
Master lease signed and deal closedExpected before end of 2026Subject to unspecified conditions
Lease schedules beginOn delivery, expected first half of 2027Expected

Musk said on 25 October that the financing had closed. The company’s own filing, made a day later, does not say that. Where the two differ, this article follows the filing.

Whose credit backs SpaceX GPU financing

SpaceX has investment-grade ratings: Baa1 from Moody’s, BBB+ from Fitch and BBB from S&P, all with stable outlooks, according to Reuters on 18 June. Musk’s claim that “we’re investment grade and have Starlink & launch revenue” is accurate on the first point. Reuters also reported that S&P viewed the AI segment as the uncertain part, because of its capital needs and competition.

Bond investors have been less generous than the agencies. A Bloomberg Opinion column in July calculated that SpaceX’s average spread over Treasuries was slightly wider than the average for BB-rated, sub-investment-grade debt. On 21 October, the spread on SpaceX’s 6.6% notes due 2046 widened to 2.64 percentage points, and its five-year credit default swaps were near record highs, according to Bloomberg and ICE data relayed by Seoul Economic Daily.

These four measures answer different questions. A rating is an agency’s opinion of default risk. Musk’s description is an issuer’s claim. A bond’s yield is what investors pay today for an existing security with its own maturity and liquidity. The lease rate will be a negotiated price for a new, tranched instrument. None of them substitutes for the others.

The operating picture is strong but concentrated. Third-quarter revenue was $14.6 billion: $7.56 billion from the Super Intelligence (SI) segment, $5.25 billion from Starlink and $1.77 billion from launch. One SI customer supplied 34.5% of it. Nine-month capital spending of $48.1 billion far exceeded operating cash flow. SpaceX’s CFO cites payback periods under a year on SI infrastructure, without publishing the calculation.

A backlog that ends before the lease does

SpaceX reported a backlog of $63.0 billion at 30 September. It expects 67% of that to be recognised within a year, 27% in one to three years and 6% after that. Six per cent is about $3.8 billion. The lease payments, by contrast, run from 2027 to roughly 2032.

The contracts are also shorter than they look. Google’s agreement to pay $920 million a month runs to June 2029, but either side can end it on 90 days’ notice after 31 December 2026, TechCrunch reported from SpaceX’s filing. Anthropic’s $1.25 billion-a-month Colossus 1 agreement runs through 2029.

So the later years of rent are not covered by any contract SpaceX has disclosed. Investors are lending against SpaceX’s ability to keep selling compute, and behind that, against Starlink and launch. That is the opposite of how CoreWeave structures its cheapest debt.

CoreWeave: one GPU business, two prices

CoreWeave is the cleanest test of the idea that the customer, not the hardware, sets the price of GPU debt. The company borrows in two very different ways, and the price gap between them is wide.

The first is the delayed-draw term loan, or DDTL, raised inside a special-purpose subsidiary. CoreWeave’s March 2026 overview of DDTL 4.0 describes an $8.5 billion facility priced at SOFR plus 2.25 percentage points, with a fixed tranche at about 5.9%. Moody’s rated it A3. Lenders hold a first-priority claim on the equipment, the data-centre leases and the cloud contract with an unnamed investment-grade customer, and the loan is non-recourse to CoreWeave itself. CoreWeave says that cost is about 1.75 points below DDTL 3.0, which was priced at SOFR plus 4% in 2025 against an OpenAI contract.

The second is ordinary unsecured bonds. In mid-October, CoreWeave’s 9.75% notes due 2031 traded at a yield to maturity of about 15.8%, as we covered in our analysis of CoreWeave’s secured loans and unsecured bonds. No specific assets back those notes.

Several things differ between the two. One is a rated, secured loan priced at origination in March; the other is a market yield on an unsecured bond in a much worse October. Neither is backed by the chips in the SpaceX deal. The comparison still shows something useful: an A3 rating on a GPU loan came from the investment-grade customer’s contract and the security package around it. CoreWeave’s own credit, priced near 15%, was largely kept out of the loan.

SpaceX’s lease points the other way. The 8-K doesn’t mention any customer contract pledged to the lessor, and SpaceX’s contracts are shorter than the lease. The rent comes from SpaceX’s general resources. That puts the lease closer to SpaceX’s own corporate credit than to a contract-backed CoreWeave loan, with tranching and the equipment adding protection on top.

SpaceX GPU financing already has a costlier precedent

The Apollo deal is not SpaceX’s first GPU lease. Fortune reported from SpaceX’s IPO prospectus that entities of Valor Equity Partners, led by SpaceX director Antonio Gracias, signed three GPU leases with an xAI subsidiary in October 2025, January 2026 and April 2026. SpaceX guarantees the payments. The obligations total close to $20 billion over their terms, and SpaceX’s auditor treated the arrangements as loans, with the GPUs as collateral.

SpaceX’s quarterly filings let us estimate what that money costs. Related-party debt stood at $12.9 billion on 30 September. Related-party interest expense was $864 million for the first nine months and $513 million for the first half, which leaves $351 million for the third quarter. Annualised against the quarter’s average balance, that is about 10.7%. This is our calculation, not a disclosed rate, and the interest line may include fees or other related-party items.

For comparison, SpaceX’s $25 billion of unsecured bonds, priced in June, carry coupons from 5.35% on the 2031 notes to 6.65% on the 2056 notes. If our estimate is close, the company’s existing GPU leases cost several points more than its plain unsecured debt.

There are good reasons for some of that gap. The first lease was signed when xAI was a separate company whose own secured notes carried a 12.5% coupon, according to the same Fortune report. Related-party terms are also not a clean market price. That makes the Apollo rate a useful test. A rate near SpaceX’s bond curve would show that SpaceX’s credit now carries its GPU leases. A rate near the Valor level would suggest investors still price SpaceX’s AI hardware as a separate risk.

Where the chips still matter in SpaceX GPU financing

The hardware doesn’t disappear from the analysis. It decides how much investors recover if SpaceX stops paying, and it shapes how large the senior and junior tranches can be. The question is what a fleet of used Vera Rubin systems would fetch in the circumstances where SpaceX can’t pay.

Nvidia’s Rubin announcement describes the Vera Rubin NVL72 rack as 72 Rubin GPUs and 36 Vera CPUs, and claims up to ten times lower inference cost per token than Blackwell for mixture-of-experts models. That is a vendor claim, but it shows the lender’s problem: each generation is sold on making the last one cheaper to replace, and Rubin won’t stay Nvidia’s newest platform for five years.

That is the residual-value problem in AI chip financing, and it bites hardest when borrower stress and collateral value move together. If AI demand falls, SpaceX’s SI revenue and used-GPU prices fall at once. Book value won’t help, for the reasons in how accelerator depreciation changes the economics of AI infrastructure.

SpaceX GPU

SpaceX’s answer is that its rent doesn’t depend only on AI. Musk wrote that debt markets had “almost entirely shut off access” for companies without substantial revenue outside AI data centres. Starlink and launch are that revenue. The lessor is buying exposure to SpaceX’s whole business, with the GPUs as a backstop.

Location matters too. CoreWeave’s DDTL 4.0 lenders hold a claim on the data-centre leases. The SpaceX 8-K says nothing about site access, and racks that need specific power and liquid cooling are hard to repossess.

The missing number in SpaceX GPU financing

The fixed rate will be set at pricing, which comes before the master lease is signed. Bloomberg reported that a volatile week for AI debt had disrupted pricing discussions. Until SpaceX discloses the rate, nobody outside the deal can say how investors weighed SpaceX’s credit against the hardware.

The rate also determines the cost. Because the balance falls by one-twentieth each quarter, the average balance over a five-year schedule is roughly half the original cost. On that basis, each percentage point on the rate adds about $0.95 billion to $1.05 billion of interest across $40 billion of principal. The range depends on whether the charge is calculated before or after each payment, which the 8-K doesn’t specify. These are illustrative figures that ignore delivery timing and ongoing costs.

A worked example: one tranche, three lessees

Take a $1 billion tranche of identical racks on the same rent formula: twenty quarterly principal payments of $50 million, with interest on the balance left after each payment. Only the lessee and the protections change. The rates are borrowed from the instruments discussed above. They are not quotes for the SpaceX deal.

Lessee and protectionIllustrative rateInterest over the leaseTotal paid
GPU cloud SPV with a pledged contract from an investment-grade customer (like DDTL 4.0)5.9%$140 million$1.140 billion
Subsidiary lease guaranteed by an investment-grade parent, related-party lessor (SpaceX’s estimated Valor cost)10.7%$254 million$1.254 billion
GPU cloud’s own unsecured credit (CoreWeave 2031 notes yield)15.0%$356 million$1.356 billion

The hardware is the same in every row. The interest bill more than doubles between the first and last rows because the source of repayment changes. The first row also shows that a company whose own bonds yield near 15% can borrow cheaply when a stronger customer stands behind the contract.

Three ways SpaceX GPU financing could play out

These are conceptual scenarios, not predictions. They show who absorbs the damage as conditions change.

Strong AI demandWeaker AI demandSector-wide downturn
SpaceX SI revenueGrows with capacityFalls; customers like Google can exit on 90 days’ notice after 2026Falls sharply
Rent capacityCovered by SI aloneCovered with help from Starlink and launchDepends heavily on Starlink and launch
Used Rubin valuesFirmSofter as buyers wait for newer chipsWeak, with few buyers
Who takes the loss firstNobodySpaceX shareholders, through lower returns on computeShareholders, then junior lease tranches if SpaceX defaults

In the first two columns, the lease investors are mostly protected by SpaceX’s balance sheet. Only the third column, where SpaceX itself is in trouble and used GPUs are hard to sell at the same moment, tests the hardware’s value.

What to watch next in SpaceX GPU financing

  • The rate. Near SpaceX’s unsecured bond yields would support the argument here. Well above them would mean investors are charging extra for AI or equipment risk.
  • The master lease. If filed, look for guarantees, purchase options, prepayment rights, default terms and any pledge of customer contracts or data-centre leases.
  • Closing, expected before the end of 2026 under conditions the filing doesn’t describe.
  • SpaceX’s spreads and default swaps. Wider spreads before pricing would likely mean a higher lease rate.
  • The Google contract after 31 December, when its termination right becomes live.
  • Backlog length in the annual report, and first deliveries in 2027, when rent starts.

Three findings would change this conclusion: a customer contract pledged to the lessor, a third-party guarantee, or a rate far above SpaceX’s own debt.

What SpaceX GPU financing says about AI credit

The question was how much of the price of AI hardware financing reflects the hardware and how much reflects the customer. On the documents available, the customer dominates. CoreWeave got investment-grade pricing by putting a stronger customer’s contract between lenders and itself. SpaceX is the customer in its own deal, and the rent formula repays the full equipment cost from SpaceX’s cash flows.

The chips still set the floor on recovery. That matters most in the scenario where SpaceX and the AI market weaken together. The rate, when SpaceX discloses it, will show how much investors charged for that scenario.

Frequently asked questions

What is SpaceX’s $40 billion GPU lease?

It is a planned master finance lease. An Apollo-managed lessor will buy Nvidia Vera Rubin GPU pods and related equipment and lease them to SpaceX for its data centres. SpaceX expects principal payments of about $40 billion across lease schedules of roughly five years each. Interest and certain ongoing costs are charged on top of that.

Has SpaceX’s GPU financing closed?

Not according to SpaceX’s filings as of 28 October 2026. SpaceX signed a binding commitment letter on 24 October. The master lease will be signed after pricing, and closing is expected before the end of 2026, subject to conditions the filing doesn’t describe. Musk said on X that the deal had closed, but the 8-K describes a commitment.

What interest rate will SpaceX pay?

It hasn’t been disclosed. The 8-K says a fixed rate will be set at pricing, before the master lease is signed. Bloomberg reported that pricing talks were still under way on 25 October. On illustrative assumptions, each percentage point on the rate adds roughly $1 billion of interest across the $40 billion.

Who owns the GPUs under the lease?

The Apollo-managed lessor acquires the equipment and leases it to SpaceX, according to the 8-K. The filing doesn’t say what happens at the end of the term or whether SpaceX can buy the equipment. Because rent repays one-twentieth of each tranche’s cost every quarter, the lessor recovers the full cost through rent if SpaceX pays as scheduled.

Is SpaceX investment grade?

Yes, by the agencies. Moody’s rates it Baa1, Fitch BBB+ and S&P BBB, all with stable outlooks as of June 2026. Bond markets have been more sceptical. SpaceX’s spreads and credit default swaps widened through October, and a Bloomberg Opinion column argued in July that its bonds traded more like high-yield debt.

Why compare SpaceX with CoreWeave?

CoreWeave borrows against GPUs in two ways that are priced very differently. Its A3-rated loan, secured by an investment-grade customer’s contract, carried a fixed rate of about 5.9%. Its unsecured notes yielded about 15.8% in October. The instruments differ, so the comparison shows how structure and customer credit drive price. It doesn’t show that the chips are equivalent.

What happens to the GPUs if AI demand falls?

SpaceX would still owe rent, which it could pay from Starlink and launch revenue. The equipment’s value matters mainly if SpaceX defaults. Then investors depend on resale prices for used Vera Rubin systems, which would probably be weak in the same downturn. Junior tranches would take losses first.

What should readers watch next?

Watch for the pricing disclosure and the fixed rate, the master lease terms if SpaceX files them, and closing before year-end. Also watch SpaceX’s bond spreads, the status of the Google contract after 31 December 2026, and the first Vera Rubin deliveries in 2027.

Keep reading

Here are the latest posts from the blog.

Types of AI Models

Types of AI Models in 2026: A Field Guide From LLMs to World Models

Model details checked against official announcements, model cards and documentation on 1 November 2026. Line up four AI models that were current at the end …

Read more

AI Coding Agent

AI Coding Agents Are Getting Expensive. Who’s Paying for the Tokens?

Pricing checked on official vendor pages on 1 November 2026. All USD. Cost scenarios are illustrative and labelled as such. On 28 October, TechCrunch published …

Read more

generative AI architecture

Generative AI Architecture: The 7 Layers That Are Hard to Undo

Published 30 October 2026. API prices checked 28–30 October 2026 on each provider’s official pricing page. Cost figures below are labelled as illustrative where they …

Read more

Indirect Prompt Injection

Gemini 4 Carbon Hit 0% on Indirect Prompt Injection. Is That Enough?

When Google announced Gemini 4 Carbon on 28 October, one line in the safety section stood out. Carbon scored 0.0% on Gray Swan’s Indirect Prompt …

Read more

Kimi K3.1 Pro Hardware Requirements: GPUs, Memory, Cost

by
Kimi K3.1 Pro hardware requirements

Before anyone argues about which GPU to rent, a smaller question settles most of the argument: how many bytes have to sit in accelerator memory before the first token comes out? For Kimi K3.1 Pro hardware requirements, that number already exists, even though the model’s own weights don’t.

Moonshot AI launched Kimi K3.1 Pro on its API on 23 October and describes it as a post-trained version of Kimi K3. K3’s weights have been public since July, and they occupy 1.561 TB. Unless Moonshot changes the format when it publishes K3.1 Pro, that’s the floor: about 1.56 TB of weights before a single cached token, activation buffer or draft model.

A model name doesn’t give you a GPU count. Weight format, memory per accelerator, the engine’s reservation, context length and concurrency do. This guide works through each with the arithmetic shown, so you can swap in your own numbers.

Quick Answer
  • Weights: 1.561 TB (1.420 TiB) for the native MXFP4 K3 checkpoint. A plain 8-bit copy would be 2.78 TB; BF16 would be 5.56 TB.
  • Beyond weights: about 14 KB of compressed attention cache per token at FP8, plus a fixed recurrent state of roughly 0.2 to 0.4 GB per active request, plus runtime buffers.
  • GPU floor: eight 288 GB accelerators (NVIDIA B300 or AMD MI355X class) in one node: 2,304 GB raw. That’s also the minimum the vLLM recipe states.
  • Single GPU: no. The largest current accelerators hold 288 GB, under a fifth of the weights.
  • Multi-node: required on 141 to 192 GB parts, and advised for production on anything; Moonshot recommends 64 or more accelerators for efficient serving.
  • API or self-host: one on-demand 8x B300 node on AWS runs about $102,540 a month. To beat $2/$10 per million tokens on a 20k-in, 2k-out workload, it must sustain roughly 13,000 input and 1,300 output tokens per second, all day, every day. Start with the API.
Key Takeaways
  • Kimi K3.1 Pro’s weights are promised by 4 November 2026; until then, size against K3’s 1.561 TB MXFP4 checkpoint and expect to re-check.
  • “2.8 TB at 8-bit” is arithmetic for a file that doesn’t exist. The native release is already 4-bit, trained that way.
  • Eight 288 GB GPUs in one node are the floor. B200, H200 and H100 fleets need two or more nodes just to hold the weights.
  • Every request costs a fixed 0.2 to 0.4 GB of KDA state plus about 14 KB per token of FP8 MLA cache, so short requests aren’t as cheap as their length suggests and million-token contexts are cheaper than on dense models.
  • A single on-demand AWS node costs about $102,540 a month before redundancy, staff or storage.
  • Matching the $2/$10 API on a 20k-in, 2k-out workload takes roughly 13,000 input and 1,300 output tokens per second, sustained around the clock. Measure whether your hardware can do that before you sign anything.
  • The K3 licence isn’t MIT: model-as-a-service businesses above $20 million in total company revenue need a separate agreement.

Quick Navigation

What is confirmed about Kimi K3.1 Pro

The model is real and callable today as kimi-k3.1-pro, with a 1,048,576-token context window. Almost everything a hardware plan needs, though, comes from its predecessor’s repository.

ItemStatus on 26 Oct 2026Basis
Model exists, API liveConfirmedMoonshot release post, API docs
2.8T total, 104B active parametersInferred for K3.1 ProK3 model card; “post-trained version of K3”; Artificial Analysis listing
896 experts, 16 routed + 2 shared per tokenConfirmed for K3 onlyK3 config.json
Native MXFP4 weightsConfirmed for K3 onlyK3 config and model card
Weights downloadableNo; promised by 4 Nov 2026Moonshot release post
LicenceUnpublished; K3’s used as referenceKimi K3 License

Two architectural facts drive everything below. K3 is a mixture-of-experts (MoE) model: a router sends each token through 16 of 896 expert networks plus 2 shared ones, so only about 104 billion parameters do work per token. All 2.8 trillion still have to be resident, because any token can be routed to any expert.

The second fact is the attention design. Of 93 layers, 69 use Kimi Delta Attention (KDA), a linear-attention variant that keeps a fixed-size state per request instead of a growing cache. The other 24 use gated multi-head latent attention (MLA), which caches one compressed vector per token. That split makes K3’s memory behaviour unlike a conventional transformer’s.

Calculating Kimi K3.1 Pro weight memory

Kimi K3.1 Pro hardware requirements

The formula is plain: weight bytes = parameters x bits per parameter ÷ 8. Using K3’s 2,779,931,837,184 parameters:

PrecisionBits per parameterDecimal TBBinary TiBWhat it represents
BF16165.5605.057Theoretical; no such release
8-bit (FP8/INT8)82.7802.528Theoretical
4-bit, no scales41.3901.264Theoretical minimum
MXFP4 everywhere4.251.4771.343Adds one 8-bit scale per 32 weights
Actual K3 checkpoint4.49 (effective)1.5611.420Published repository, 96 shards

All rows are weights only, with no runtime overhead. The brief’s “2.8 TB at 8-bit, 1.4 TB at 4-bit” is correct arithmetic for the wrong question. Moonshot trained K3 to run in MXFP4 from supervised fine-tuning onward, so there is no 8-bit release to plan around.

The gap between 1.39 TB and 1.561 TB has two sources. MXFP4 stores a shared 8-bit scale for every block of 32 weights, adding about 0.09 TB. And the config leaves attention, shared experts, the dense first-layer MLP, the output head and the vision encoder unquantized. The “5.6 TB” figures circulating for K3 come from assuming BF16.

Watch your units. TB is 10^12 bytes and TiB is 2^40, about 10% larger. NVIDIA lists a DGX B300’s total GPU memory as “2.1 TB”, yet eight 288 GB GPUs hold 2,304 GB, which is 2.10 TiB. Compare a TiB capacity with a TB weight file and you’ve lost a tenth of your headroom on paper.

How many GPUs Kimi K3.1 Pro needs

The lower bound is: GPUs = ceiling(weight bytes ÷ (memory per GPU x usable fraction)). vLLM’s Kimi K3 recipes run at 0.88 to 0.92 of GPU memory; this guide uses 0.90.

AcceleratorMemory per GPULower boundPractical layoutFits one 8-GPU node?
NVIDIA B300 / GB300288 GB78Yes, ~513 GB left after weights
AMD MI355X288 GB78Yes, same arithmetic
NVIDIA B200180 GB1016 (two nodes)No: 1,440 GB total
NVIDIA H200141 GB1316 (two nodes)No: 1,128 GB total
NVIDIA H10080 GB2224 to 32 (three to four nodes)No

Memory per GPU from AMD’s MI355X page (which also lists B200 at 180 GB) and NVIDIA. Seven 288 GB GPUs clear the bound, but tensor parallelism splits the 96 attention heads across GPUs, and 96 doesn’t divide by seven. Eight is the real floor, and it’s what the vLLM recipe states: at least 8x GB300, or 8x MI355X/MI350X on ROCm, with “multi-node for real production traffic”.

Eight GPUs are not one 2.3 TB pool. Each holds its own shard, and every layer exchanges activations between them. Inside a DGX B300 that traffic rides NVLink at 14.4 TB/s aggregate. Between nodes it crosses InfiniBand or Ethernet at up to 800 Gb/s per port, roughly 100 GB/s, which is why two-node recipes split the model into pipeline stages (TP8 x PP2) and why NVIDIA’s Dynamo recipes target GB200 and GB300 racks with NVLink spanning trays: 16 GB200s aggregated, or 32 split between prefill and decode.

Hopper is the awkward case. H100 and H200 lack native FP4 tensor cores, so MXFP4 experts run through dequantising kernels. The recipe page builds commands for H200, but its stated minimum is Blackwell or MI355X, and we found no published throughput for K3 on Hopper.

Can a single GPU or workstation run it?

No single GPU comes close. The workstation route is llama.cpp with the experts in system RAM. One published test loaded a ~554 GB 1-bit GGUF of K3 on a 160 GB VRAM machine with the MoE weights on CPU. Nobody has published quality figures for that quant. The 4-bit GGUF is 1,508.7 GB, barely smaller than the native file, because the native file is already 4-bit. It’s a way to poke at the model, not to serve users, and K3.1 Pro quants can’t exist until the weights do.

The KV cache changes the Kimi K3.1 Pro calculation

Weights are a one-off cost. Every active request adds its own memory, and K3’s hybrid design splits that into two parts with very different behaviour.

  1. MLA cache, which grows with context. Per token: 24 layers x (512 latent + 64 positional) = 13,824 values. That’s 13.8 KB at FP8, the precision vLLM’s recipes use, or 27.6 KB at BF16. The config sets mla_use_nope, so if the positional part isn’t stored the cache is about 11% smaller; treat 13.8 KB as an upper bound.
  2. KDA state, which doesn’t. Each of 69 layers holds a 128 x 128 state matrix per head across 96 heads: 108.5 million values per request, 0.22 GB in BF16 or 0.43 GB in FP32, whether the prompt is 50 tokens or 500,000. That size assumes the Kimi Linear formulation Moonshot published for KDA; check your engine’s reported cache allocation.
Context per requestMLA cache (FP8)KDA state (BF16)Total per request
8K tokens0.11 GB0.22 GB0.33 GB
32K tokens0.45 GB0.22 GB0.67 GB
128K tokens1.81 GB0.22 GB2.03 GB
1M tokens14.50 GB0.22 GB14.71 GB

Two things fall out. Below about 16,000 tokens the fixed state outweighs the growing cache, so short chat traffic carries a per-request floor most sizing guides ignore. At the other end, long contexts are cheap: a 70B dense model with grouped-query attention needs about 160 KB per token, as our breakdown of AI memory costs works out, or roughly 172 GB for a million tokens against K3’s 14.7 GB.

On one 8x B300 node, 0.90 of 2,304 GB leaves about 2,074 GB, and weights take 1,561 GB. Set aside an assumed 110 GB for activations, CUDA graphs, communication buffers and the speculative draft model the recipes load, and roughly 400 GB remains for requests. That’s an upper bound of about 600 concurrent requests at 32K, 197 at 128K, or 27 at the full million.

Those counts assume the cache is spread across GPUs without duplication. vLLM’s decode context parallelism does that for MLA; without it, tensor-parallel ranks can each hold a copy and the numbers fall sharply. The recipes also spill to host RAM: one AMD prefill configuration budgets 1,799 GB of host DRAM for cache offload. Memory scales linearly with concurrency and context. Throughput at those concurrencies is a separate question that only measurement answers.

Quantisation trade-offs

Weight quantisation and cache quantisation are separate levers, and the first one is mostly pulled already.

LeverMemory effectWhat you risk
Keep native MXFP4 weightsBaseline 1.561 TBNothing; this is how the model was trained
Up-cast to FP8 or BF161.8x to 3.6x moreHardware cost, with no documented quality gain
Community 1-bit or 2-bit quantsAbout 0.55 TB for the smallest published 1-bit GGUFUnmeasured quality loss; K3-only until K3.1 Pro weights ship
FP8 KV cacheHalves the MLA cache versus BF16Small accuracy risk; recipes use it by default
FP32 vs BF16 KDA stateDoubles or halves the fixed per-request stateSome low-latency backends require FP32

Because K3 went through quantization-aware training, its 4-bit weights aren’t a lossy copy of some better original that’s available to you. Going below 4 bits is a real compression, and nobody has published K3 or K3.1 Pro quality numbers for those quants. Run your own evaluation against the hosted API before trusting one.

Hardware matters here too. Blackwell and AMD’s CDNA 4 parts execute MXFP4 natively; AMD lists 10.1 PFLOPs of MXFP4 matrix throughput for the MI355X. Hopper doesn’t, which is a second reason the recipes start at Blackwell. The cache side has its own knobs: the latency-tuned GB300 profile switches the KDA state to FP32, doubling that per-request floor to about 0.43 GB.

The real cost of self-hosting Kimi K3.1 Pro

Kimi K3.1 Pro hardware requirements

Two dated price points for one 8x B300 node, on demand, 30 days at 24 hours:

ItemAWS p6-b300.48xlargeMedian specialist cloud
Hourly, 8x B300$142.42$65.44 ($8.18 per GPU)
30 days continuous (720 h)$102,540$47,117
Effective rate at 50% useful utilisation$284.83/h$130.88/h
Effective rate at 25% useful utilisation$569.66/h$261.76/h

AWS rate: us-east-1, Linux, on demand, per Vantage’s EC2 tracker, read 26 October 2026; the instance includes 4,096 GiB of host RAM and 8 x 3.84 TB local NVMe. Specialist rate: median across 10 providers per getdeploying, 8 September 2026; those offers may not match AWS on networking, storage or availability.

The effective-rate rows are the ones that matter. A node you pay for around the clock but keep busy half the time costs twice as much per useful hour, and real traffic has nights and weekends.

The GPU line also isn’t the bill. One node has no redundancy, so a production service needs a second, doubling the fixed cost before it serves an extra token. Add engineering time, monitoring, storage for 1.56 TB of weights plus staging on every node, egress, and the work of swapping in K3.1 Pro weights when they arrive. Published estimates of that total put it at 1.3 to 5 times the raw GPU rate, a range we unpack in the hidden fees in self-hosted LLM cost.

Kimi K3.1 Pro API vs self-hosting: the break-even

Moonshot’s API price for standard kimi-k3.1-pro is $2 per million input tokens, $10 per million output and $0.20 per million cached input, confirmed by Artificial Analysis. The Highspeed variant costs $4/$20. Cache writes are billed separately at a rate that wasn’t displayed when we checked.

The formula: API cost = (input tokens ÷ 1,000,000 x input rate) + (output tokens ÷ 1,000,000 x output rate).

Illustrative workload, an assumption: an agentic coding call with 20,000 input tokens and 2,000 output tokens, reasoning included. Uncached, that’s $0.04 + $0.02 = $0.06 per call. With 90% of input served from cache, it’s $0.0076 + $0.02 = $0.0276, before cache-write fees.

Break-even calls per month = monthly self-hosting cost ÷ API cost per call. Spread over 2,592,000 seconds in 30 days:

Scenario (GPU cost only)Calls/month to match APISustained calls/sInput tokens/sOutput tokens/s
AWS, no cache1.71M0.6613,1871,319
AWS, 90% cache hits3.72M1.4328,6672,867
Specialist cloud, no cache785K0.306,059606
Specialist cloud, 90% cache hits1.71M0.6613,1721,317

Caching makes the API cheaper, so it pushes break-even further away. And these are averages held for 720 hours straight. Peaks need headroom above them, so a fleet that just breaks even on average is running below capacity most of the day.

Can one node sustain those rates at an acceptable latency? There’s no independent published throughput for K3.1 Pro on 8x B300, so that’s the number to measure, not assume. Remember too that the right comparison is the cheapest API that meets your quality bar. Third-party hosts listed K3 below Moonshot’s own price on OpenRouter, and the same is likely once K3.1 Pro weights circulate. Our Kimi K3.1 Pro vs Sonnet 5.5 comparison covers per-task token use, which moves the API side of this equation more than the rate card does.

Self-hosting earns its place when data can’t leave your infrastructure, when load is high and steady enough to keep a node busy, when you need to fine-tune, or when you already own Blackwell capacity sitting idle.

Licence and operational constraints

K3.1 Pro’s licence isn’t published. K3’s, the likely template, is a custom “Kimi K3 License”. Its grant borrows MIT’s wording (use, copy, modify, distribute, sell, fine-tune), but it isn’t MIT. The licence text adds:

  • A model-as-a-service clause. If you or your affiliates run a business giving third parties inference or fine-tuning access with meaningful control over inputs, parameters or training data, and your total revenue with affiliates exceeds $20 million over any 12 consecutive months, you need a separate agreement with Moonshot before commercial use. The threshold is company revenue, not revenue from the model.
  • A branding clause. Products above 100 million monthly active users or $20 million in monthly revenue must display “Kimi K3” prominently.
  • Exemptions. Internal use that exposes neither the model nor its outputs to third parties, and use through Moonshot’s own products or certified inference partners.

That’s a reading of K3’s terms, not legal advice. Have counsel read K3.1 Pro’s licence when it ships.

The operational constraints are less visible. The vLLM image is a CUDA 13 build needing an r580 or newer NVIDIA driver. Recipes raise engine start-up timeouts to between one and two hours, because loading 1.56 TB across GPUs and autotuning kernels is slow; that’s your recovery time after a crash. Every node needs its own copy of the weights. And the recipe warns that K3 occasionally emits tool calls its own parser doesn’t expect, so validate and retry.

A practical Kimi K3.1 Pro deployment decision

  • Use the hosted API when traffic is small, spiky or unproven, or nobody on the team runs GPU clusters. Moonshot offers zero data retention to API customers, which settles some privacy objections without any hardware.
  • Rent one 8x B300 or MI355X node for evaluations, fine-tuning experiments or steady internal batch work you can keep busy. Accept that it has no redundancy.
  • Plan multi-node or rack-scale for production with latency targets, long contexts at high concurrency, or uptime commitments. Moonshot’s 64-plus accelerator guidance describes this tier.
  • Benchmark before you sign. Replay 50 to 200 real requests, measure tokens per second at your p95 latency target, and compare any quantised variant’s outputs with the hosted API.

What would change this estimate? The K3.1 Pro file size and precision; the KDA state dtype your engine uses; your measured throughput per node; GPU rental prices, which roughly doubled for B300 through 2026 according to price trackers; and the K3.1 Pro licence terms.

Conclusion

Three figures here are solid: 1.561 TB of K3 weights, 288 GB per B300 or MI355X, and the $2/$10 API price. The per-request memory numbers are calculations from K3’s published config and should hold if K3.1 Pro keeps the architecture. What nobody can give you yet is throughput, and throughput decides whether any of the cost arithmetic works.

So don’t rent for Kimi K3.1 Pro hardware requirements yet. Use the API, log your real token mix and concurrency, and when the weights land, check the file size against 1.56 TB, then spend a week on one rented node measuring tokens per second at your latency target. If that number can’t clear the break-even rates in the table above with room for peaks and a second replica, the API was the cheaper machine all along.

Frequently asked questions

How much VRAM does Kimi K3.1 Pro need?

Provisionally, at least 1.56 TB just for weights, based on the Kimi K3 checkpoint that K3.1 Pro is post-trained from. Add memory for each active request and for runtime buffers. In practice that means about 2.3 TB of GPU memory, such as eight 288 GB accelerators. The figure will be confirmed when Moonshot publishes K3.1 Pro weights, promised by 4 November 2026.

Can Kimi K3.1 Pro run on one GPU?

No. The largest current accelerators, NVIDIA B300 and AMD MI355X, hold 288 GB each, under a fifth of the 1.56 TB weight file. Even heavily compressed community quants of K3 run to roughly 550 GB, which still needs several GPUs or large amounts of system RAM.

How many GPUs are needed for a model of this size?

Eight 288 GB GPUs are the practical minimum and match the vLLM recipe’s stated floor. With 180 GB B200s or 141 GB H200s you need 16 across two nodes. Moonshot recommends 64 or more accelerators for efficient production serving.

Does 4-bit quantisation reduce memory requirements?

For this model it’s already applied. Kimi K3 ships in MXFP4, a 4-bit format with shared scales, trained that way from fine-tuning onward. Converting to 8-bit would roughly double memory. Going below 4 bits shrinks the file further, but no published quality data exists for those quants.

How much does self-hosting cost per month?

One 8x B300 node costs about $102,540 for 30 days on AWS on demand, or about $47,117 at the median specialist-cloud rate (prices checked September to October 2026). Redundancy, staff, storage and idle time add substantially to that GPU line.

Is an API cheaper than running the model locally?

For most workloads, yes. At $2/$10 per million tokens, a self-hosted AWS node only matches the API if it sustains about 1.7 million 20k-token calls a month. Self-hosting can win on data control, steady high load or fine-tuning needs.

Can the model run on a workstation?

Only experimentally. Testers have loaded a 1-bit K3 quant with experts held in system RAM on a machine with 160 GB of VRAM. Output quality at that compression is unmeasured, and K3.1 Pro quants can’t exist until its weights ship.

What licence applies to commercial deployment?

K3.1 Pro’s licence is unpublished. K3 uses the custom Kimi K3 License, which permits commercial use but requires a separate agreement for model-as-a-service businesses above $20 million in 12-month company revenue, and “Kimi K3” branding above 100 million monthly users or $20 million monthly revenue.

Keep reading

Here are the latest posts from the blog.

Types of AI Models

Types of AI Models in 2026: A Field Guide From LLMs to World Models

Model details checked against official announcements, model cards and documentation on 1 November 2026. Line up four AI models that were current at the end …

Read more

AI Coding Agent

AI Coding Agents Are Getting Expensive. Who’s Paying for the Tokens?

Pricing checked on official vendor pages on 1 November 2026. All USD. Cost scenarios are illustrative and labelled as such. On 28 October, TechCrunch published …

Read more

generative AI architecture

Generative AI Architecture: The 7 Layers That Are Hard to Undo

Published 30 October 2026. API prices checked 28–30 October 2026 on each provider’s official pricing page. Cost figures below are labelled as illustrative where they …

Read more

Indirect Prompt Injection

Gemini 4 Carbon Hit 0% on Indirect Prompt Injection. Is That Enough?

When Google announced Gemini 4 Carbon on 28 October, one line in the safety section stood out. Carbon scored 0.0% on Gray Swan’s Indirect Prompt …

Read more

AI Infrastructure Spending: What $145 Billion and a Sold-Out AWS Actually Buy

by
AI capex

Published 25 October 2026 · Last verified 25 October 2026 · By the UniverseBlend editorial desk

On 21 October, Meta set the latest marker for AI infrastructure spending, telling investors it expects to spend $135 billion to $145 billion on capital expenditure in 2026, a figure that includes principal payments on finance leases.

Side by side, the two statements describe one problem from opposite ends. Meta shows how much money is going in. Amazon shows how little of that money has yet become compute that a customer can rent.

In this story, AI infrastructure spending means cash spent on long-lived assets — land, buildings, electrical and cooling plant, servers, accelerators, memory and networking — that hyperscalers expect to run AI and cloud workloads.

Key Takeaways
  • Meta’s 21 October guidance narrowed its 2026 range to $135–145 billion; the $145 billion ceiling has stood since April. With $85.8 billion spent by September, the range implies $49–59 billion of fourth-quarter capex, 41–70% above the third quarter.
  • Amazon has not raised its ~$220 billion forecast since 30 July. Jassy’s 23 October remarks described capacity commitments and an appetite to spend more, not a new budget.
  • “Sold out” applies to large, committed blocks of AWS capacity. Jassy said Amazon still keeps capacity back for on-demand EC2 and its Bedrock inference service.
  • Estimates of 2026 hyperscaler spending run from about $794 billion (consensus cited by Goldman Sachs, all capex, US) to about $900 billion (J.P. Morgan, AI capex, five US firms) to $957 billion (TrendForce, nine global cloud providers). They measure different things and cannot be added together.
  • Memory prices are inflating the dollar figures. Jassy attributed most of Amazon’s $20 billion increase to memory costs, and TrendForce estimates DRAM and NAND will take 47% of large cloud providers’ capex this year.
  • AWS’s $496 billion backlog equals roughly 2.9 years of its current revenue run rate. Its duration, pricing and margin are not disclosed.
  • Utilisation of new capacity, contract pricing, and price versus volume inside capex would show whether the spending works. None is disclosed.

Quick Navigation

The AI infrastructure spending ledger

The headline numbers come in three kinds: what companies have spent, what they say they will spend, and what analysts think the sector will spend. They should never share a column without labels.

WhoFigure and periodStatusWhat it does not establish
Meta$135–145bn, 2026, incl. finance-lease principalGuidance, 21 Oct 2026AI share; price inflation within it
Amazon~$220bn cash capex, 2026Management expectation, 30 Jul 2026; unchanged at 25 OctAWS share (includes retail, logistics, satellites)
Alphabet$195–205bn, 2026Guidance, 22 Jul 2026Split between Cloud customers and Google’s own use
Microsoft~$175bn, calendar 2026Guidance as reported, 29 Jul 2026Lower than April’s ~$190bn because of lease accounting, not cuts
Five US hyperscalers incl. Oracle~$900bn AI capex, 2026J.P. Morgan estimate, 19 Oct 2026Company disclosure; uses J.P. Morgan’s AI definition
US hyperscalers~$794bn total capex, 2026Consensus cited by Goldman Sachs, 7 Aug 2026AI share; Goldman says it overstates US AI investment by ~$200bn
Nine global cloud providers$957bn, 2026TrendForce estimate, 14 Oct 2026Includes ByteDance, Tencent, Alibaba, Baidu; not additive to the rows above

The “$700 billion” figure that circulated in July is out of date. Reuters used it on 30 July for Big Tech’s AI investment; by October the narrowest comparable estimate had passed $790 billion.

Meta’s guidance reads like a raise, but only the floor moved, from $130 billion to $135 billion. The $145 billion ceiling dates from 29 April, when Meta cited higher component prices. What the new floor establishes is that Meta expects to spend at least $49 billion in the fourth quarter. (Calculation: $135bn minimum guidance minus $85.82bn spent from January to September, using purchases of property and equipment plus finance-lease principal from Meta’s third-quarter cash-flow statement.)

Microsoft’s row shows why a lower number need not mean a smaller programme. CFO Dive reported that Microsoft extended the estimated useful life of its data centres from 15 to 25 years. Chief financial officer Amy Hood said the change moves some future data-centre leases from finance leases, which count as capex, to operating leases, which do not. The buildings are the same. The reported capex is lower.

The money arrives before the compute

A capex dollar buys assets that come online in a fixed order, and the earliest spending earns nothing.

In his April shareholder letter, Jassy listed what AWS pays for up front: land, power, buildings, chips, servers and networking gear. The cash goes out “typically 6-24 months before we start billing customers,” depending on the component. The same letter puts data-centre lives at 30-plus years and chips, servers and networking at five to six.

That timing suggests a framework for reading any capex announcement. Track the money through six states:

  1. Committed — approved budget or guidance. Meta’s $135–145 billion sits here.
  2. Procured — equipment ordered or paid for. TrendForce reports that some component suppliers now want payment in advance, which pulls cash forward.
  3. Installed — racks in a building. This needs a finished shell and cooling.
  4. Available — energised, networked and commissioned capacity that can take a workload. This needs a grid connection.
  5. Contracted — capacity sold under commitment. AWS’s $496 billion backlog lives here, and it can run ahead of state 4.
  6. Monetised — capacity billing at a price that covers its depreciation and running costs.

Companies disclose the first and fifth states and, in aggregate, the sixth. The states in between are where the time goes.

Power is the slowest of them. UniverseBlend’s analysis of transformer lead times and interconnection queues found transformers averaging about 128 weeks and US interconnection waits of three to seven years, longer than the buildings they serve. J.P. Morgan’s 19 October note puts data-centre construction at 18–30 months and names power as the primary bottleneck.

Amazon’s own metric points the same way. AWS added 3.9 gigawatts of power capacity in 2025 and expects to double total power capacity by the end of 2027, according to the shareholder letter. When the constraint is energised megawatts, capex can outrun capacity for a long time without anything having gone wrong.

That is why Meta’s fourth quarter deserves attention. Its guidance implies quarterly spending 41–70% above the third quarter’s $34.91 billion. (Calculation: implied fourth quarter of $49.2–59.2bn divided by the third-quarter actual.) A late-year surge of that size buys assets that will mostly enter service in 2027.

AI capex

The memory allocation problem

Memory changed the arithmetic more than any other component in 2026, because it raises the price of AI servers and ordinary cloud servers at the same time.

Two products are involved. Conventional server DRAM, sold as DDR5 registered modules (RDIMMs), sits beside CPUs in general-purpose servers. High-bandwidth memory (HBM) is also made from DRAM dies, but they are stacked, linked by through-silicon vias and packaged beside an accelerator to deliver very high bandwidth. An operator cannot plug HBM into a CPU server or fit DDR5 into a GPU package.

They compete upstream, for wafers. Micron chief executive Sanjay Mehrotra has said HBM3E needs about three times the silicon of DDR5 to produce the same number of bits. He expects that ratio to exceed 4-to-1 with HBM4E. Every wafer turned over to HBM yields fewer bits than it would as DDR5. UniverseBlend covered how HBM crowds out conventional DRAM wafers in September.

Amazon has tied this to its budget. On its 30 July earnings call, Jassy attributed most of the increase from about $200 billion to about $220 billion to higher memory costs, which J.P. Morgan’s research also records. He returned to the point on the 23 October podcast. According to several reports of the interview, he described splitting constrained memory between HBM for accelerators and DRAM for AWS’s core, non-AI services, both of which are growing.

The analyst figures show the scale. TrendForce estimates that server DRAM contract prices rose 64% in the second half of 2025 and will rise about 270% over 2026. It expects DRAM and NAND flash together to account for 47% of large cloud providers’ capex in 2026 and 68% in 2027. These are one research firm’s estimates, not company disclosures, but they point the same way as what Amazon and Meta have told investors.

Demand is coming from the CPU side as well. A 22 October TrendForce report expects server CPU shipments to grow 31% in 2026 and more than 40% in 2027, partly because agentic AI workloads need more CPU orchestration. It sees planned CPU-to-accelerator ratios moving from 1:8 toward 1:2 or even 1:1. Those CPUs need memory of their own, mostly DDR5 server modules, so AI demand also lifts conventional DRAM demand directly.

Part of 2026’s capex growth is therefore a price effect: a larger budget buys fewer servers per dollar than it did a year ago. No hyperscaler splits capex into price and volume, so outsiders cannot tell how much extra compute each additional billion buys.

AI capex

Training and inference compete for the same buildout

Accelerator counts say little on their own, because training and inference ask different things of the same hardware budget.

Training a frontier model needs thousands of accelerators working in lockstep for weeks, high-bandwidth interconnect between them and enough HBM to hold model state. Models are retrained and new ones begun, so a training cluster is a recurring claim on capacity, not a one-off cost.

Inference runs continuously and follows user demand. Generating each token means reading the model’s weights and its key-value (KV) cache from memory. Throughput is therefore usually limited by memory bandwidth, as UniverseBlend explained in why memory bandwidth, not FLOPS, caps inference throughput. The KV cache grows with context length and with the number of concurrent requests, so longer contexts cut how many users one accelerator can serve. Operators batch requests to raise utilisation, and pay for it in latency.

These differences turn into capex decisions. A cluster built for training carries a networking fabric that an inference fleet may not need. Hardware built for fast or cheap inference may be poorly suited to frontier training, and UniverseBlend has argued that specialised inference chips win only a narrow slice of the market. Capacity can move between workloads, though not always efficiently.

Meta and Amazon make this choice from different positions. Meta is its own customer. Its release says current and scheduled capacity gives it “flexibility to allocate resources across model training, core business growth” and new products. Its return arrives mainly through advertising: third-quarter ad revenue rose 26%, with impressions up 10% and price per ad up 14%. That growth cannot be credited wholly to AI infrastructure, although Meta says its AI investments are already delivering substantial incremental ad revenue.

Amazon sells capacity. Jassy said 2027 is “largely sold out all the big chunks,” according to Reuters, with some capacity held back for on-demand EC2 and Bedrock. Large commitments take long blocks; on-demand and managed-inference customers buy in small pieces at different prices. The same gigawatt can earn very different returns depending on who fills it.

Sold-out capacity versus profitable capacity

Sold-out capacity shows that demand at current prices exceeds deliverable supply. It does not show that the next dollar of capex earns an attractive return.

The disclosures establish three things. Customers have signed commitments: AWS’s backlog reached $496 billion at the end of June, up from $364 billion three months earlier. AWS is profitable now, with $16.6 billion of operating income on $42.2 billion of second-quarter revenue, according to Amazon’s second-quarter release. And Jassy wrote in April that Amazon already had commitments for a substantial portion of its 2026 AWS capex.

They do not establish duration, pricing or concentration. The backlog is about 2.9 years of AWS’s $169 billion annualised run rate (calculation: backlog divided by run rate), but Amazon does not say how long the contracts run, what margins they carry or how much comes from a few AI laboratories; OpenAI alone has committed more than $100 billion, according to the shareholder letter. Regional mix and utilisation are also undisclosed.

Costs are rising too. Meta’s depreciation and amortisation rose 44% year on year in the third quarter to $7.15 billion, and its free cash flow was negative $2.12 billion. Meta also announced a non-voting Class C share class, partly to give it a way to raise capital, effective 19 January 2027. J.P. Morgan notes that hyperscalers are increasingly funding the buildout with debt and equity rather than free cash flow.

Useful-life assumptions sit beneath all of this. Amazon depreciates chips and servers over five to six years. J.P. Morgan’s chief US economist, Michael Feroli, has warned that AI hardware may lose economic value faster than conventional equipment as each generation lowers the cost per unit of compute. If he is right, current depreciation schedules could overstate profits.

The strongest counterargument is Amazon’s. Jassy’s letter argues that data centres last 30-plus years and that returns become cumulatively attractive a couple of years after assets enter service. That holds well for buildings and power, the long-lived part. It says less about the five-year assets, which are where memory inflation lands.

What to watch next

Scheduled disclosures:

  • 28 October 2026 — Alphabet third-quarter results and Microsoft fiscal first-quarter 2027 results, both after the US close. Watch Alphabet’s $195–205 billion range and Microsoft’s capex including finance leases.
  • 29 October 2026 — Amazon third-quarter results, as reported by Reuters. Watch whether the ~$220 billion expectation changes, the AWS backlog and margins, and any memory commentary.
  • 19 January 2027 — Meta’s Class C stock takes effect, an indication of how Meta intends to fund 2027.

Not yet scheduled: Meta’s 2027 capex range, which management says it is still developing, and 2027 HBM contract prices, which TrendForce expects to rise 70–140%.

A scorecard for the quarters that follow:

IndicatorTypeDisclosed?
Backlog growth versus revenue growthLeadingYes (AWS, Google Cloud, Microsoft)
Capex growth versus cloud revenue growthLeadingYes
Price versus volume within capexLeadingNo
Utilisation of newly deployed capacityLeadingNo
Segment margin after depreciationLaggingPartly (AWS yes; Meta’s AI no)
Free cash flow and external financingLaggingYes
Return on invested capital for AI assetsLaggingNo

If backlog keeps outpacing revenue while margins hold after depreciation, the spending is converting. If capex keeps rising while margins fall and financing needs grow, it is not, whatever the backlog says.

Conclusion

Treat AI infrastructure spending in 2026 as a measure of intent. Meta’s range tells you how much it is prepared to put in; Amazon’s sold-out remark tells you how far delivery is lagging demand. Neither tells you about returns.

Three questions separate a credible announcement from a merely large one. How much of the increase is price? How long until the spending becomes energised capacity? And who has contracted to use it, at what margin? Today companies answer the third only in aggregate and the first two not at all. Until that changes, a rising capex number is evidence of demand and of component inflation, in proportions that nobody outside these companies can see.

FAQ

What is AI capex in 2026?

It is capital spending on data centres, power, cooling, servers, accelerators, memory and networking intended to run AI and cloud workloads. Estimates for 2026 range from about $794 billion (US hyperscaler consensus, all capex) to about $900 billion (J.P. Morgan, AI capex at five US firms), depending on scope.

Why is Meta spending up to $145 billion on capital expenditure?

Meta guided on 21 October 2026 to $135–145 billion, including finance-lease principal, to support its advertising business and its AI work. When it raised the range in April, it cited higher component prices. Its 2027 plans are not yet set.

Why is Amazon’s AWS capacity sold out through 2027?

Jassy said on 23 October 2026 that customers have committed to most large blocks of 2027 capacity because demand from AI labs and enterprises exceeds what Amazon can build. Some capacity is kept for on-demand EC2 and Bedrock, so not every service or region is unavailable.

How do HBM and conventional DRAM affect AI infrastructure costs?

HBM is stacked memory packaged with accelerators; conventional DRAM serves CPU servers. They are not interchangeable, but both come from DRAM wafers, and HBM uses about three times as much silicon per bit. Rising prices for both have pushed up capex budgets, including Amazon’s.

Does higher AI capex guarantee higher cloud profits?

No. Capex creates depreciation, over five to six years for chips and servers, and returns depend on utilisation and pricing. No company discloses margins on its new AI capacity.

How can businesses tell whether AI infrastructure spending is producing returns?

Compare backlog and revenue growth with capex growth, then check margin after depreciation and free cash flow. For a buyer, the test is utilisation: reserved but idle capacity earns nothing.

Editorial note: Figures are as disclosed or reported up to 25 October 2026. Alphabet and Microsoft report on 28 October and Amazon on 29 October; if those results change any figure here, this article will carry a dated update. Report errors through the contact page.

Keep reading

Here are the latest posts from the blog.

Types of AI Models

Types of AI Models in 2026: A Field Guide From LLMs to World Models

Model details checked against official announcements, model cards and documentation on 1 November 2026. Line up four AI models that were current at the end …

Read more

AI Coding Agent

AI Coding Agents Are Getting Expensive. Who’s Paying for the Tokens?

Pricing checked on official vendor pages on 1 November 2026. All USD. Cost scenarios are illustrative and labelled as such. On 28 October, TechCrunch published …

Read more

generative AI architecture

Generative AI Architecture: The 7 Layers That Are Hard to Undo

Published 30 October 2026. API prices checked 28–30 October 2026 on each provider’s official pricing page. Cost figures below are labelled as illustrative where they …

Read more

Indirect Prompt Injection

Gemini 4 Carbon Hit 0% on Indirect Prompt Injection. Is That Enough?

When Google announced Gemini 4 Carbon on 28 October, one line in the safety section stood out. Carbon scored 0.0% on Gray Swan’s Indirect Prompt …

Read more

Model Deprecation Tracker: Retirement Dates by Provider

by
model deprecation dates

A model retirement can break a production system that nobody has touched in months. The application code stays the same while the model ID it calls stops resolving, and the next request returns an error. Asking which model is newest doesn’t help much here. What matters is which identifier each environment is running, what its provider has published about that identifier, and how many working days remain before the date. This page collects the model deprecation dates that the major providers have published, with each row checked against the provider’s own documentation.

The timing is not abstract. On 23 October 2026, the day after this check, OpenAI is scheduled to shut down o1, o1-pro, o3-mini, o4-mini, gpt-4-turbo and several other legacy snapshots on its API.

Quick answer: what developers need to know
  • Model deprecation dates tell you when a provider stops supporting a model ID. Retirement (or shutdown) dates tell you when requests to it start failing. They are separate events, often months apart.
  • Deprecated usually still works. Anthropic defines a deprecated model as “still functional but no longer recommended”. Microsoft lets existing Azure customers keep using deprecated models, while blocking new customers.
  • The same model can retire on different dates on different platforms. Claude Sonnet 4.5 retires on the Claude API on 2026-11-30 but on Amazon Bedrock on 2027-04-08.
  • A missing date means the provider hasn’t published one. It doesn’t mean the model is safe indefinitely. “Not sooner than” dates are floors, not schedules.
  • The verified tracker is in section 1. Each row links to its primary source.
Key Takeaways
  • Keep an inventory of every model ID, platform and region you call. It’s the only way to know which deadlines apply to you.
  • Treat the deprecation date and the retirement date as two separate events. Plan to the retirement date, and start when the deprecation notice arrives.
  • Check the same model on every platform you use. OpenAI and Azure, and Anthropic and Bedrock, publish different dates for the same models.
  • Check the replacement’s own retirement date before migrating to it.
  • Evaluate replacements on your own work. A newer model can cost more, answer differently or need a different request format.
  • Treat a missing date as unknown, not safe.

The providers that publish the most detail, OpenAI and Anthropic, are also the easiest to plan around. The gaps sit with the providers that list models as “deprecated or retired” without dates, and with cloud platforms whose schedules drift from the model developer’s own. For teams running several models across several clouds, the inventory and the monthly check are the controls that actually prevent an outage.

Quick Navigation

Model deprecation dates: the current tracker

Each row below comes from the provider’s own lifecycle page, read on 22 October 2026. “Announced” is the date the provider published the notice, where it gives one. “Retirement” is the provider’s shutdown, retirement or end-of-life (EOL) date, in the provider’s own terminology. “Replacement” is shown only when the provider names one. “Last verified” is the date UniverseBlend checked the row, which is not the date the provider changed it.

Rows are grouped by urgency. Dates are YYYY-MM-DD.

A. Retirement scheduled within 90 days (by 2027-01-20)
ProviderModel / endpoint IDStatusAnnouncedRetirementRecommended replacementMigration notesLast verifiedSource
OpenAI APIo1, o1-2024-12-17, o1-pro, o3-mini, o4-mini, gpt-4-turbo, gpt-4, gpt-3.5-turbo, gpt-4.1-nano, gpt-4o-2024-05-13, gpt-image-1 and their fine-tunesRetirement scheduled2026-04-222026-10-23Per ID: gpt-5.6-sol, gpt-5.6-terra or gpt-5.6-luna; o1-pro → gpt-5.6-sol with reasoning.mode: proFine-tuned ft-gpt-4, ft-gpt-3.5-turbo and ft-o4-mini models shut down the same day. Fine-tunes can’t be carried across, so each needs re-evaluating on the replacement.2026-10-22OpenAI deprecations
Google Gemini APIdeep-research-pro-preview-12-2025 (managed agent, preview)Preview; retirement scheduledNot verified2026-10-23 (earliest possible)deep-research-preview-04-2026Agent IDs carry their own preview lifecycle.2026-10-22Gemini deprecations
Google Gemini APIveo-3.1-generate-preview, veo-3.1-fast-generate-preview, veo-3.1-lite-generate-preview, gemini-omni-flash-previewPreview; shutdown date reached on verification dayNot verified2026-10-22gemini-omni-1.1-flashVideo generation moves to a different model family, so prompts and output handling need retesting.2026-10-22Gemini deprecations
OpenAI APIv1/prompts (reusable prompts), Evals dashboard and API, Agent BuilderRetirement scheduled2026-06-032026-11-30 (Evals read-only from 2026-10-31)Prompts: move into application code. Evals: Promptfoo. Agent Builder: Agents SDK or ChatGPT Workspace AgentsThese are platform features, not models. Export prompt and eval definitions before the read-only date.2026-10-22OpenAI deprecations
AWS Bedrockanthropic.claude-3-5-sonnet-20240620-v1:0, anthropic.claude-3-5-sonnet-20241022-v2:0, anthropic.claude-3-7-sonnet-20250219-v1:0, anthropic.claude-3-sonnet-20240229-v1:0, anthropic.claude-3-haiku-20240307-v1:0 (some Regions)LegacyLegacy from 2026-10-10 (Claude 3 models 10, 13 or 15 Oct by Region)2026-11-13No official replacement identified on the lifecycle pageBedrock does not migrate automatically. Claude 3 Haiku appears with different EOL dates for different Regions, so check yours.2026-10-22Bedrock model lifecycle (Legacy)
Google Gemini APIgemini-2.5-flash-preview-tts, gemini-2.5-pro-preview-tts, gemini-3.1-flash-tts-previewPreview; retirement scheduledNot verified2026-11-17 (earliest possible)gemini-3.8-flash-tts or gemini-3.8-flash-lite-ttsVoices and audio formats may differ, so listen-test before switching.2026-10-22Gemini deprecations
Google Gemini APIgemini-3.1-flash-live-preview, gemini-2.5-flash-native-audio-preview-12-2025Preview; retirement scheduledNot verified2026-11-17 (earliest possible)gemini-3.8-liveLive sessions depend on latency and turn-taking behaviour, so test them under real-time load.2026-10-22Gemini deprecations
Azure OpenAIo3 (2025-04-16), o3-pro (2025-06-10), o3-deep-research (2025-06-26), o3-mini (2025-01-31), o4-mini (2025-04-16)DeprecatedNot verified2026-11-19gpt-5.6-sol (o3 family); gpt-5.6-terra (o3-mini, o4-mini)After retirement, requests return 410 Gone. Provisioned deployments are not auto-upgraded.2026-10-22Azure model retirement schedule
Anthropic (Claude API)claude-sonnet-4-5-20250929Deprecated2026-09-302026-11-30claude-sonnet-5-5This date applies to the Claude API only; Bedrock’s is 2027-04-08.2026-10-22Anthropic model deprecations
OpenAI APIgpt-image-1-mini, gpt-image-1.5, chatgpt-image-latestRetirement scheduled2026-06-022026-12-01gpt-image-2.5-sunburst or gpt-image-2.5-flarechatgpt-image-latest is a “-latest” alias, so anything pinned to it stops resolving on this date.2026-10-22OpenAI deprecations
Azure OpenAIgpt-4o (2024-05-13)DeprecatedNot verified2026-12-09gpt-5.6-solStandard deployments are auto-upgraded region by region; provisioned deployments must migrate manually.2026-10-22Azure model retirement schedule
OpenAI APIgpt-5-2025-08-07, gpt-5-mini-2025-08-07, gpt-5-nano-2025-08-07, gpt-5-pro-2025-10-06, o3-2025-04-16, o3-pro-2025-06-10Retirement scheduled2026-06-112026-12-11gpt-5.6-sol, gpt-5.6-terra, gpt-5.6-luna by size; pro variants use reasoning.mode: proThe pro models change from separate IDs to a reasoning mode, so request parameters change too.2026-10-22OpenAI deprecations
OpenAI APItts-1, tts-1-hd, gpt-4o-mini-tts-2025-03-20, gpt-4o-mini-tts-2025-12-15Retirement scheduled2026-10-012027-01-06gpt-realtime-2.1-miniThe replacement is a realtime model, so the integration pattern changes as well as the ID.2026-10-22OpenAI deprecations
AWS Bedrockanthropic.claude-opus-4-1-20250805-v1:0Legacy (public extended access since 2026-10-08)Legacy from 2026-07-082027-01-08No official replacement identified on the lifecycle pageAlready retired on the Claude API (2026-08-05); on Bedrock it runs five months longer.2026-10-22Bedrock model lifecycle (Legacy)
OpenAI APIgpt-4o-2024-08-06, gpt-4o-2024-11-20, gpt-4o-mini-2024-07-18, gpt-4.1-2025-04-14, gpt-4.1-mini-2025-04-14Retirement scheduled2026-10-142027-01-14gpt-6-lunaFine-tuned versions follow the base model’s schedule. Azure retires the same snapshots on 2027-04-14.2026-10-22OpenAI deprecations
OpenAI APIgpt-realtime, gpt-audio, gpt-4o-realtime, gpt-4o-audio, gpt-realtime-mini, gpt-audio-mini, gpt-4o-mini-realtime, gpt-4o-mini-audioRetirement scheduled2026-07-202027-01-20gpt-realtime-2.1, gpt-realtime-2.1-mini or gpt-audio-1.5 per IDVoice agents need end-to-end latency testing, not just text evals.2026-10-22OpenAI deprecations
B. Retirement scheduled after 2027-01-20
ProviderModel / endpoint IDStatusAnnouncedRetirementRecommended replacementMigration notesLast verifiedSource
Azure OpenAIgpt-5, gpt-5-mini, gpt-5-nano (2025-08-07)GA, retirement date publishedNot verified2027-02-09No official replacement identifiedOpenAI’s own API shuts the same snapshots on 2026-12-11, two months earlier.2026-10-22Azure model retirement schedule
OpenAI APIwhisper-1, gpt-4o-transcribe, gpt-4o-mini-transcribe, gpt-4o-transcribe-diarize, endpoint /v1/audio/translationsRetirement scheduled2026-08-262027-02-26gpt-live-transcribe or gpt-transcribe; v1/realtime/translations for the endpointThe endpoint path changes, so this needs code changes, not just a config edit.2026-10-22OpenAI deprecations
Google Gemini APIgemini-2.5-flash-imageRetirement scheduledNot verified2027-03-15 (earliest possible)gemini-3.1-flash-lite-imageImage output style changes with the model, so review outputs visually.2026-10-22Gemini deprecations
AWS Bedrockmeta.llama3-70b-instruct-v1:0, meta.llama3-8b-instruct-v1:0, meta.llama3-1-70b-instruct-v1:0, meta.llama3-1-8b-instruct-v1:0, meta.llama3-3-70b-instruct-v1:0, meta.llama4-maverick-17b-instruct-v1:0, meta.llama4-scout-17b-instruct-v1:0, mistral.mistral-7b-instruct-v0:2, mistral.mixtral-8x7b-instruct-v0:1, deepseek.r1-v1:0, deepseek.v3-v1:0, google.gemma-3-4b-it, google.gemma-3-12b-it, google.gemma-3-27b-itLegacyLegacy from 2026-09-302027-03-30No official replacement identified on the lifecycle pagePublic extended access date is “to be announced”. New Provisioned Throughput can’t be created during Legacy.2026-10-22Bedrock model lifecycle (Legacy)
OpenAI APIgpt-5.1, gpt-5.3-codex, gpt-5.4-nanoRetirement scheduled2026-10-012027-04-01gpt-6-sol (gpt-5.1, gpt-5.3-codex); gpt-6-luna (gpt-5.4-nano)Azure lists gpt-5.1 as the replacement for some gpt-4o versions, yet OpenAI retires it first. Check the replacement’s own date.2026-10-22OpenAI deprecations
AWS Bedrockanthropic.claude-sonnet-4-5-20250929-v1:0LegacyLegacy from 2026-10-082027-04-08No official replacement identified on the lifecycle pagePublic extended access begins 2027-01-08. Existing customers can lose access after 15 days of inactivity during Legacy.2026-10-22Bedrock model lifecycle (Legacy)
Azure OpenAIgpt-4o (2024-08-06, 2024-11-20), gpt-4o-mini (2024-07-18), gpt-4.1, gpt-4.1-mini, gpt-4.1-nano (2025-04-14)Deprecated (gpt-4o 2024-11-20: Legacy)Not verified2027-04-14gpt-5.1 for gpt-4o; none listed for the othersFine-tuned deployments of these models run to 2027-10-01 or 2027-10-14.2026-10-22Azure model retirement schedule
Google Gemini APIgemini-3.1-flash-liteRetirement scheduledNot verified2027-05-07 (earliest possible)gemini-3.5-flash-liteA GA model with a published one-year horizon.2026-10-22Gemini deprecations
Google Gemini APIgemini-embedding-001Retirement scheduledNot verified2028-05-14 (earliest possible)gemini-embedding-2Changing the embedding model means re-embedding the whole corpus, so plan the storage and compute for it.2026-10-22Gemini deprecations
C. Retired in 2026 (selected)
ProviderModel / endpoint IDStatusAnnouncedRetirementRecommended replacementMigration notesLast verifiedSource
OpenAI APIchatgpt-4o-latestShut down2025-11-182026-02-17gpt-5.1-chat-latest (itself shut down 2026-07-23)A replacement that later retired too.2026-10-22OpenAI deprecations
Anthropic (Claude API)claude-3-7-sonnet-20250219Retired2025-10-282026-02-19claude-sonnet-5-5Requests to retired Claude models fail.2026-10-22Anthropic model deprecations
Google Gemini APIgemini-3-pro-previewShutdown date passedNot verified2026-03-09gemini-3.1-pro-previewThe preview ran for under four months (released 2025-11-18).2026-10-22Gemini deprecations
Cohereembed-english-v2.0, embed-english-light-v2.0, embed-multilingual-v2.0, c4ai-aya-expanse-8b, c4ai-aya-vision-8bRetiredNot verified (entry dated 2026-04-04)2026-04-04embed-v4.0 or v3.0 embeddings; command-a-03-2025, command-r7b-12-2024 or command-a-reasoning-08-2025 for chatEmbedding replacements require re-indexing.2026-10-22Cohere deprecations
Anthropic (Claude API)claude-3-haiku-20240307Retired2026-02-192026-04-20claude-haiku-4-5-20251001Anthropic’s pricing page lists Haiku 4.5 at $1 input / $5 output per 1M tokens. Compare that with your current spend before switching.2026-10-22Anthropic model deprecations
Google Gemini APIgemini-2.0-flash, gemini-2.0-flash-001, gemini-2.0-flash-lite, gemini-2.0-flash-lite-001Shutdown date passedNot verified2026-06-01gemini-3.6-flash; gemini-3.1-flash-lite for LiteGoogle calls its table dates the “earliest possible”, so the actual cutoff wasn’t confirmed here.2026-10-22Gemini deprecations
Anthropic (Claude API)claude-sonnet-4-20250514, claude-opus-4-20250514Retired2026-04-142026-06-15claude-sonnet-5-5; claude-opus-4-8On Bedrock, Claude Sonnet 4 reached EOL later, on 2026-10-14.2026-10-22Anthropic model deprecations
Anthropic (Claude API)claude-opus-4-1-20250805Retired2026-06-052026-08-05claude-opus-4-8The Claude API gave 61 days’ notice; Bedrock keeps it until 2027-01-08.2026-10-22Anthropic model deprecations
OpenAI APIAssistants APIShut down2025-08-262026-08-26Responses API and Conversations APIAn entire API surface was removed, not just a model.2026-10-22OpenAI deprecations
OpenAI APIVideos API, sora-2, sora-2-pro and dated snapshotsShut down2026-03-242026-09-24No official replacement identifiedNo migration path was published.2026-10-22OpenAI deprecations
D. Active or preview, with published lifecycle context
ProviderModel / endpoint IDStatusAnnouncedRetirementRecommended replacementMigration notesLast verifiedSource
Anthropic (Claude API)claude-haiku-4-5-20251001ActiveNot applicableNot publicly announced (floor “not sooner than 2026-10-15” has passed)Not applicableA passed floor date doesn’t mean the model has retired. Expect a deprecation notice with at least 60 days’ warning.2026-10-22Anthropic model deprecations
Anthropic (Claude API)claude-opus-4-5-20251101ActiveNot applicableNot publicly announced (floor 2026-11-24)Not applicableThis is the next Claude floor date to watch.2026-10-22Anthropic model deprecations
Anthropic (Claude API)claude-mythos-previewDeprecated2026-06-09To be announcedNot listedDeprecated, but without a retirement date.2026-10-22Anthropic model deprecations
Google Gemini APIgemini-2.5-pro, gemini-2.5-flash, gemini-2.5-flash-liteActive (restricted)Not applicableNone announcedNot applicableGoogle says these “are not deprecated” but limits access to users who have used them before.2026-10-22Gemini deprecations
Google Gemini APIgemini-3.5-flash, gemini-3.7-flashAuto-routedNot verifiedNot applicableRouted to gemini-3.6-flash and gemini-3.8-flashThe ID still answers, but a different model serves it. Re-run evals.2026-10-22Gemini deprecations
Google Gemini APIgemini-3.1-pro-previewPreviewNot applicableNone announcedNot applicableEarlier Pro previews were shut down roughly four to nine months after release.2026-10-22Gemini deprecations
Mistral AIOlder IDs such as mistral-large-2411, mistral-medium-2505, mistral-small-2503, codestral-2501, pixtral-large-2411Deprecated or retired (not distinguished)Not publishedNot publishedNot published on the models pageMistral lists these together without dates, so check your ID with Mistral support.2026-10-22Mistral models

Model deprecation dates by provider

OpenAI

OpenAI uses “deprecation” for the process and “shutdown” or “sunset” for the moment a model “is no longer accessible”, in the words of its deprecations page. The page is the most complete schedule of any provider. Every entry has a heading date, exact snapshot IDs, a shutdown date and usually a named replacement. It also covers things that aren’t models: the Assistants API, reusable prompts, the Evals platform and Agent Builder. Several 2026 notices route replacements through a reasoning parameter, so o3-pro users move to gpt-5.6-sol with reasoning.mode: pro, which changes the request body as well as the model name.

Anthropic

Anthropic publishes a status table with four states: Active, Legacy, Deprecated and Retired. It commits to “at least 60 days’ notice before model retirement for publicly released models”, according to its model deprecations page. Its 2026 notices hit that window closely. Opus 4.1 was announced on 5 June and retired on 5 August. Active models carry a “not sooner than” date, which is a floor rather than a plan: Claude Haiku 4.5’s floor of 15 October 2026 has passed and the model is still listed as Active. From the 4.6 generation onward, Anthropic’s model ID documentation says a dateless ID such as claude-sonnet-4-6 “maps to a single, fixed model snapshot”.

Google Gemini API

Google labels its table dates as “the earliest possible dates on which a model might be retired” and announces exact dates separately, via release notes. Preview models turn over fast: gemini-3-pro-preview was released on 18 November 2025 and shut down on 9 March 2026. Google also auto-routes some retired IDs at the API level. Calls to gemini-3.5-flash are served by gemini-3.6-flash, so code keeps working while the model behind it changes. Google Cloud’s enterprise platform has its own lifecycle page, which did not render for this check and is not covered here.

Azure OpenAI

Azure OpenAI runs its own schedule for the same OpenAI models. Microsoft’s retirement policy defines deprecated as closed to new customers but still usable by existing ones, and retired as “All inference requests return 410 Gone“. GA models get 60 days’ notice and previews 30. Retirement dates “aren’t extendable”. Standard deployments can be auto-upgraded region by region, but provisioned deployments cannot. The dates diverge from OpenAI’s. gpt-5-2025-08-07 shuts down on OpenAI’s API on 11 December 2026 but stays on Azure until 9 February 2027.

Amazon Bedrock

Amazon Bedrock uses Active, Legacy and EOL. Its lifecycle documentation sets either a six-month or a 45-day Legacy period. It also states that “only the Amazon Bedrock dates on the model card apply, even if they differ from the provider’s dates”. Two clauses are easy to miss. During Legacy, existing customers “may lose access after 15 days of inactivity”. And after EOL, migration “will not happen automatically”.

Cohere and Mistral

Cohere and Mistral publish less. Cohere’s deprecations page defines its terms clearly (deprecated models stay available to anyone who used them within 90 days of the announcement), but some entries give no separate shutdown date. Mistral’s models page lists dozens of older IDs as “deprecated or retired” with no dates and no replacements, which is the largest gap in this tracker.

What a retirement date actually means

Providers use different words for similar events, and the differences decide what breaks and when.

  • Deprecation is a notice. Requests still succeed. On Azure and Cohere, deprecation also stops new customers from starting to use the model.
  • Legacy is a warning before deprecation, used by Anthropic, Azure, Bedrock and Cohere. On Bedrock, though, Legacy already comes with an EOL date.
  • Retirement, shutdown, sunset or EOL is the date requests start failing. Microsoft documents the failure as 410 Gone.
  • API or endpoint retirement removes a surface, not a model. OpenAI’s Assistants API shut down on 2026-08-26, and code built on it broke even where the underlying model was still available through the Responses API.
  • Alias changes and auto-routing keep the ID working but change the model behind it. OpenAI’s chatgpt-image-latest and Google’s routed Flash IDs both fall into this category.
  • Preview expiry follows a shorter cycle. Azure guarantees 30 days’ notice for previews instead of 60, and Google’s preview models often last only a few months.
model deprecation dates

A retirement date is also scoped to a platform. Claude Opus 4.1 stopped working on the Claude API on 2026-08-05 but runs on Bedrock until 2027-01-08. Teams that call the same model through two clouds have two deadlines.

Finally, a date in a table isn’t proof that the shutdown happened. Google’s dates are explicitly the earliest possible, and Bedrock’s page lists models under “reached EOL” by Region. The only reliable check is to call the model ID from your own account.

Why model deprecation dates matter for AI infrastructure and costs

The cost of a retirement is rarely the replacement’s token price. It is the work forced by the deadline: inventorying where the ID is used, re-running evaluations, rewriting prompts that relied on one model’s habits, and re-approving outputs in regulated workflows. UniverseBlend’s guide to what breaks when your model version retires maps that work week by week. A tracker only tells you when it has to be finished.

Replacement models have different economics even when they are described as more capable. Anthropic’s recommended successor to claude-3-haiku-20240307 is claude-haiku-4-5-20251001, which its pricing page lists at $1 input and $5 output per million tokens. A team moving off a small, older model should re-budget rather than assume the new one costs the same. Reasoning-mode replacements can produce more output tokens for the same task. Embedding replacements, such as Cohere’s v2 to v4 or Google’s gemini-embedding-001 to gemini-embedding-2, mean re-embedding every stored document. That is a compute bill and a re-indexing window, not a config change.

Serving and capacity change too. Bedrock blocks new Provisioned Throughput for Legacy models, and Azure does not auto-upgrade provisioned deployments, so reserved capacity has to be re-bought or re-created on the new model. Latency and throughput profiles differ between models, which can change how many replicas or how much rate-limit headroom a service needs.

Contracts and regions add another layer. A team on Azure may have more time than one on the OpenAI API, but Azure’s dates “aren’t extendable”. AWS lets a private arrangement continue access past EOL, but that depends on the provider. If your AI contract doesn’t state notice periods, the questions in our model vendor risk checklist are the ones to raise before the next renewal.

A practical model migration checklist

  1. Inventory every model ID in code, config, infrastructure templates, notebooks and scheduled jobs. Record the platform (direct API, Azure, Bedrock, Google Cloud), region and environment for each, because dates differ between them.
  2. Confirm the notice at the source. Read the provider’s lifecycle page and note both the announcement date and the retirement date. Check the replacement’s own retirement date, too.
  3. Shortlist the named replacement and one alternative. The provider’s recommendation is a starting point, not a guarantee that it fits your workload.
  4. Compare cost, latency and throughput on your real request mix, including output-token counts on reasoning models.
  5. Run your own evaluation set, 50 to 200 real, anonymised examples scored against a written rubric. Public benchmarks can screen candidates but can’t choose for you, as our LLM benchmark comparison explains.
  6. Validate the contracts the application depends on: tool-calling schemas, structured-output parsing, refusal behaviour, token limits, streaming and any reasoning parameters.
  7. Test the rollout and the rollback in staging. Keep the old ID callable until the cutover is proven, because after the retirement date there is nothing to roll back to.
  8. Update configuration, dashboards and alerts. Alert on 404, 410 and model-not-found errors by model ID, so that a missed reference shows up as an alarm instead of a customer complaint.
  9. Finish before the deadline, not on it. Rolling, region-by-region upgrades and timezone ambiguity make the final day unreliable.
  10. Record the migration date and the evaluation result, so the next retirement starts from a known baseline.
model deprecation dates

How the model deprecation dates in this tracker are maintained

Every row was checked by hand against the linked provider page on 22 October 2026. There is no automated monitoring behind this page, and coverage is selective, not exhaustive. The tracker includes models and endpoints that are likely to sit in production systems, plus examples that show how lifecycle rules work.

The planned review cadence is monthly, with an extra update whenever a provider publishes a notice that affects a listed row or a widely used model. The page keeps two dates separate. A row’s announcement date belongs to the provider, and the last-verified date belongs to UniverseBlend. The article’s modified date changes only when a row’s data changes, a row is added or removed, or the guidance is materially revised. Re-checking a page without finding a change updates the last-verified date but not the modified date.

Frequently Asked Questions

What are model deprecation dates?

They are the dates a provider publishes to mark lifecycle changes for a specific model ID: when it was deprecated, and when it will be retired or shut down. Providers use different terms, so read each one’s definitions.

Does deprecated mean a model is no longer available?

Usually not. Anthropic, Azure and Cohere all keep deprecated models working for existing users until the retirement date. Azure and Cohere close them to new customers on deprecation.

How can I check whether an AI model has been retired?

Check the provider’s lifecycle page for your platform, then call the model ID from your own account. On Bedrock, the modelLifecycle field in ListFoundationModels reports the state per Region. Azure’s Models API exposes a deprecationDate.

What happens when an AI model reaches its retirement date?

Requests to it fail. Microsoft documents a 410 Gone response, and Anthropic says requests to retired models fail. The exception is auto-upgrade or auto-routing, which some Azure deployments and some Google IDs use. There, requests succeed but a different model answers them.

Use the provider’s deprecation notice, which usually names one. Bedrock’s lifecycle page does not, and Mistral’s models page doesn’t either. In those cases, choose from the provider’s active model list and evaluate the candidates yourself.

Should production systems use model aliases or dated model identifiers?

Pinned identifiers are the safer default for production. Aliases such as -latest can change the model without a code change. Anthropic’s IDs from the 4.6 generation onward are fixed snapshots even without a date suffix. Check each provider’s convention.

How often should a model deprecation tracker be checked?

Monthly at minimum, and whenever a provider emails a notice. Short preview windows (30 days on Azure) mean that quarterly checks can miss a deadline entirely.

Can a provider change a retirement date?

The policies differ. Microsoft says its retirement dates “aren’t extendable”. Google presents its dates as the earliest possible and confirms exact dates separately. AWS allows continued access past EOL only under a private arrangement with the model provider. Assume a date can move earlier for previews, and don’t plan around it moving later.

Keep reading

Here are the latest posts from the blog.

Types of AI Models

Types of AI Models in 2026: A Field Guide From LLMs to World Models

Model details checked against official announcements, model cards and documentation on 1 November 2026. Line up four AI models that were current at the end …

Read more

AI Coding Agent

AI Coding Agents Are Getting Expensive. Who’s Paying for the Tokens?

Pricing checked on official vendor pages on 1 November 2026. All USD. Cost scenarios are illustrative and labelled as such. On 28 October, TechCrunch published …

Read more

generative AI architecture

Generative AI Architecture: The 7 Layers That Are Hard to Undo

Published 30 October 2026. API prices checked 28–30 October 2026 on each provider’s official pricing page. Cost figures below are labelled as illustrative where they …

Read more

Indirect Prompt Injection

Gemini 4 Carbon Hit 0% on Indirect Prompt Injection. Is That Enough?

When Google announced Gemini 4 Carbon on 28 October, one line in the safety section stood out. Carbon scored 0.0% on Gray Swan’s Indirect Prompt …

Read more

What the Model Context Protocol Actually Is, and What It Opens Up

by
Model Context Protocol

Connecting a language model to a company’s issue tracker used to mean writing a small integration layer by hand. Someone had to describe each operation to the model, handle authentication, map the model’s chosen action onto the right API call and turn the response back into text. Then the same work started again for the next model or the next application, because none of it was portable. With five AI applications and twenty internal systems, that is up to a hundred bespoke connectors, each with its own bugs and its own way of handling credentials.

The Model Context Protocol (MCP) is an attempt to collapse that grid. It defines one way for an AI application to discover what an external system offers and to call it. Anthropic described the problem in its November 2024 announcement: “Every new data source requires its own custom implementation.” MCP makes the connection standard. It does not make what sits on the other end of the connection trustworthy, and much of what follows is about the difference between the two.

Key Takeaways
  • The Model Context Protocol standardises how AI applications discover and call external capabilities. It replaces per-application integration code, not security design.
  • Hosts, clients and servers have distinct roles. The host sits between model and server, which makes it the natural place for policy.
  • Listing a tool and executing it are separate steps. Visibility through tools/list grants nothing, and execution is a host decision.
  • Tool descriptions and tool outputs are both untrusted inputs to the model. The specification says so, and tool-poisoning research has demonstrated the consequences.
  • The 2026-07-28 revision made MCP stateless and deprecated sampling, roots and logging. Check which version a server implements before relying on older documentation.
  • MCP provides interoperability, OAuth-based authentication for HTTP servers and consent guidance. Least privilege, isolation, validation and logging must be built around it.

Quick Navigation

What the Model Context Protocol actually is

The Model Context Protocol is an open specification for how AI applications exchange context and capabilities with external programs. In the words of the current specification, it gives applications a standard way to “share contextual information with language models” and “expose tools and capabilities to AI systems.” Its designers say it takes inspiration from the Language Server Protocol, which let one language server work with many code editors.

Several things it is often confused with are worth separating:

  • MCP is not a model. It doesn’t make a model smarter or better at choosing tools. That still depends on the model and the application.
  • MCP is not an agent framework. The architecture documentation says plainly that it “does not dictate how AI applications use LLMs or manage the provided context.”
  • MCP is not an API in the usual sense. A REST API defines one service’s operations. MCP defines how any service can describe its operations to any compatible AI application.
  • An MCP server is not the protocol. A server is one program that speaks MCP, written by anyone, with whatever quality and permissions its author chose.

Anthropic released the protocol in November 2024, along with SDKs and a set of open-source servers. In December 2025 it donated MCP to the Agentic AI Foundation, a fund under the Linux Foundation co-founded with Block and OpenAI. The existing maintainers kept control of its technical direction. The specification changes regularly. The current revision, dated 28 July 2026, made substantial changes, covered below.

Model Context Protocol architecture: hosts, clients and servers

MCP has three participants, and their responsibilities differ in ways that matter later.

  • The host is the AI application the user works in, such as a desktop chat app, an IDE or an agent runtime. It owns the conversation, the model and the user interface.
  • A client is a connector inside the host. The host creates one client for each server it connects to, and each client keeps a dedicated connection to its server.
  • A server is a program that provides context and capabilities: a filesystem server, a database server, an issue tracker’s server.

The model never talks to a server directly. When a model wants to use a tool, the host receives that request, decides whether to allow it, and has the relevant client send the call. The architecture documentation describes this step: the host “intercepts the tool call, routes it to the appropriate MCP server, executes it, and returns the results back to the LLM.” That interception point is where an application can enforce policy. Whether it does is up to the application.

Messages and transports

Clients and servers exchange JSON-RPC 2.0 messages, a simple request-and-response format. MCP defines two standard transports:

  • stdio, where the host launches the server as a local process and talks to it over standard input and output;
  • Streamable HTTP, where the server runs remotely and the client sends HTTP POST requests, optionally receiving streamed responses.

A local filesystem server typically uses stdio. A hosted service, such as the remote server Sentry operates, uses Streamable HTTP.

The 2026-07-28 revision made the protocol stateless. Earlier versions opened each connection with an initialize handshake that negotiated capabilities once per session. Now every request carries its protocol version and the client’s capabilities in its own metadata. Clients can call a new server/discover method to learn what a server supports. Servers that need state across calls must hand out explicit identifiers as ordinary tool arguments. The same revision deprecated sampling, which let servers ask the host’s model for completions, along with roots and logging. Articles written before mid-2026 describe the older design, so check which version a server implements.

Model Context Protocol

Tools, resources and prompts are different things

A server can offer three kinds of capability, and the MCP documentation is explicit about who is meant to control each.

CapabilityWhat it isWho typically controls itExample
ToolsFunctions the model can call; they may read data or change itThe model, with the host’s permissioncreate_issue, run_query, send_message
ResourcesRead-only data identified by a URIThe application, which decides what to include as contextA file, a database schema, a calendar
PromptsReusable templates for a taskThe user, who invokes them explicitlyA slash command such as /summarise-incident

The server concepts page uses this split: tools are model-controlled, resources application-controlled and prompts user-controlled. Servers declare which of these they support, and a client may use only some of them.

Discovery and execution are separate steps, and the gap between them matters. A client calls tools/list and receives each tool’s name, description and JSON Schema for its inputs. The host passes some or all of these to the model. Later, if the model asks for one, the client sends tools/call with arguments. Seeing a tool in a list grants nothing by itself. Execution is a second, distinct decision that the host makes.

What the Model Context Protocol opens up in real applications

Three patterns show what the protocol standardises and what it leaves to the application.

Querying documents or a database

A database server might expose its schema as a resource and a query function as a tool, and a filesystem server exposes file reading and searching. The official filesystem server is one of the project’s reference implementations. MCP standardises how the assistant finds those capabilities and calls them. The host still decides which directories or databases the server is configured for, and whether a query runs automatically or waits for approval. The integration needs read access to the data, and nothing more if it only answers questions.

Working with a code repository

GitHub’s official MCP server is a demonstrated case. It can run remotely, hosted by GitHub, or locally in Docker. It groups tools into toolsets, such as repositories, issues and pull requests. A --read-only flag skips write tools even when they’re explicitly requested. MCP standardises the interface, so a developer assistant in an IDE can list issues or open a pull request. The permissions come from the GitHub token or OAuth grant behind the server, and which toolsets are switched on is a deployment choice.

Retrieval combined with an action

This example is illustrative. An assistant reads a customer’s recent support tickets through one server, drafts a reply, and sends it through an email server. MCP lets both servers plug into the same host. Nothing in the protocol decides whether sending should happen without a person reviewing the draft. The integration needs read access to tickets and send access to email. That second permission turns a summarising assistant into one that acts on a customer.

The trust boundary hiding inside the tool description

A tool description is text written by the server’s author, and the model reads it to decide when and how to use the tool. That makes it an input to the model’s reasoning. If the author is careful and honest, the description helps the model choose correctly. If the author is careless, or the server is malicious, the description can push the model toward something the user never asked for.

The specification acknowledges this. It says descriptions of tool behaviour, including annotations, “should be considered untrusted, unless obtained from a trusted server.” In April 2025, Invariant Labs demonstrated “tool poisoning”. In its experiment, a tool presented as a simple add function carried hidden instructions in its description. Those instructions led the Cursor agent to read a configuration file and a private SSH key and pass them out through a tool argument. The confirmation dialog didn’t show the full arguments. Invariant also described “rug pulls”, where a server changes a description after the user has approved it. It showed a malicious server’s description altering how the agent used a different, trusted server’s email tool.

Tool outputs carry the same risk from the other direction. A tool that fetches a web page or reads an email returns text the model then reads, and that text may contain instructions. Our guide to the eight classes of prompt injection treats tool metadata and tool output as two of them.

Four layers, one of them yours

It helps to keep four layers distinct:

  1. Metadata: what the server claims a tool does.
  2. Model interpretation: what the model decides to do after reading it.
  3. Execution: what the server actually does when called, with whatever credentials it holds.
  4. Enforcement: what the host or surrounding infrastructure allows, regardless of the first three.

Only the fourth layer is under the deploying team’s direct control. Here is an illustrative example. A team connects a “project notes” server that also exposes a shell tool its author added for convenience, running with the developer’s own permissions. A poisoned page fetched by another tool now has a path to arbitrary commands. MCP carried every message correctly. The failure lay in what the host exposed and allowed.

The deeper threat model, covering token passthrough, cross-server shadowing, CVEs and supply-chain incidents, is in our analysis of where the model-to-tool boundary fails in MCP security. This article stays with what the protocol is.

Model Context Protocol

What the Model Context Protocol does not guarantee

Several common assumptions don’t survive a reading of the specification.

  • Interoperability is not semantic compatibility. Two servers can both speak MCP and still describe similar operations differently. A host that connects both may present the model with two search tools. The specification tells clients aggregating many servers to handle name collisions, for example by prefixing tool names with a server identifier.
  • Authentication is not authorisation. For HTTP servers, the authorisation specification builds on OAuth 2.1. Servers must validate that tokens were issued specifically for them, and must not pass tokens through to other services. That proves who is calling. Which actions that caller may take inside the server is the server’s own access-control logic. Authorisation is also optional in the protocol, and stdio servers are told to take credentials from their environment instead.
  • A successful response is not a trustworthy result. A tool result marked complete means the server answered. It doesn’t mean the answer is correct, safe to show the model, or free of injected instructions. The specification asks clients to “validate tool results before passing to LLM”, but how is left to them.
  • Human approval is not a complete boundary. The specification says there “SHOULD always be a human in the loop with the ability to deny tool invocations.” Invariant’s experiment shows an approval prompt is only as good as what it displays. A user approving a call they can’t fully see isn’t exercising much control.
Who is responsible for what

The specification is candid about where its responsibility ends: “MCP itself cannot enforce these security principles at the protocol level.” The work divides roughly as follows:

  • Server implementers validate inputs, enforce access control, rate-limit and sanitise outputs.
  • Host developers build consent flows, display tool calls accurately, validate results and log usage.
  • Deployment owners choose which servers to trust, with what credentials, inside what isolation.
  • Users decide what to approve, which works only if they’re shown enough to decide.

Before connecting a Model Context Protocol server: a checklist

These are the checks that change outcomes. The MCP security best practices cover the authorisation-level attacks in more depth.

  • Identity and provenance. Know who publishes the server, pin its version, and record a hash of its tool descriptions so a changed description is noticed.
  • Least privilege. Enable only the tools the task needs. Use read-only modes where they exist, as GitHub’s server offers.
  • Credentials. Give each server its own scoped, short-lived credential. Never share a broad personal token across several servers.
  • Scope of tools and resources. Restrict directories, repositories and databases at the server’s configuration, not in the prompt.
  • Approval for consequential actions. Gate writes, sends, deletes and payments. Make sure the approval shows the full arguments.
  • Isolation. Run local servers with limited filesystem and network access. The MCP best practices recommend sandboxing local servers, which run with the client’s privileges. Our guide to the four layers of sandbox isolation covers the options.
  • Validation. Check tool arguments against the schema before calling. Treat returned content as untrusted input to the model.
  • Logging. Record every tool call, its arguments, the result and the approving user, with a correlation ID.
  • Updates and configuration review. Re-review a server when its tool list changes. Updates can add tools.
  • Adversarial testing. Feed the integration poisoned documents and malicious tool outputs before production. Confirm the enforcement layer holds when the model is misled.

None of these removes prompt injection. Together they limit what a misled model can reach.

The engineering decision the Model Context Protocol leaves

MCP solves a real problem well. Connecting a new tool to a compatible host is now configuration rather than a project, and the same server can serve an IDE, a chat app and an agent runtime. The 2026 move to a stateless design also makes remote servers easier to scale behind ordinary HTTP infrastructure.

What MCP doesn’t decide is whether a given server deserves the access it asks for. Every server you connect adds a party whose text your model reads and whose code runs with credentials you granted. The question to ask of each integration is the same one you would ask of any third-party code with production access: what can it reach, and what stops it when it’s wrong?

FAQ

What is the Model Context Protocol?

It’s an open protocol that standardises how AI applications connect to external tools and data. A host application uses MCP clients to discover what an MCP server offers (tools, resources and prompts) and to call it. It doesn’t define how the model reasons or which tools are safe.

How does an MCP client communicate with an MCP server?

Through JSON-RPC 2.0 messages, over stdio for local servers or Streamable HTTP for remote ones. Since the 2026-07-28 revision, each request carries its own protocol version and capabilities. There’s no session handshake, and a client can call server/discover to learn what a server supports.

What is the difference between MCP and an API?

An API defines one service’s operations. MCP defines a common way for many services to describe their operations to AI applications. An MCP server often wraps an existing API, translating it into tools and resources that any MCP-compatible host can discover and call.

Is MCP secure?

The protocol includes OAuth-based authorisation for HTTP servers and security requirements for implementers. But the specification states that MCP “cannot enforce these security principles at the protocol level.” Security depends on which servers you trust, the permissions they hold, and the controls your host and infrastructure enforce.

When should a team use MCP?

When several AI applications need the same external capabilities, or when you want to use servers others maintain. For one application calling one internal API, a direct integration may be simpler. Either way, the same permission and isolation decisions apply.

Keep reading

Here are the latest posts from the blog.

Types of AI Models

Types of AI Models in 2026: A Field Guide From LLMs to World Models

Model details checked against official announcements, model cards and documentation on 1 November 2026. Line up four AI models that were current at the end …

Read more

AI Coding Agent

AI Coding Agents Are Getting Expensive. Who’s Paying for the Tokens?

Pricing checked on official vendor pages on 1 November 2026. All USD. Cost scenarios are illustrative and labelled as such. On 28 October, TechCrunch published …

Read more

generative AI architecture

Generative AI Architecture: The 7 Layers That Are Hard to Undo

Published 30 October 2026. API prices checked 28–30 October 2026 on each provider’s official pricing page. Cost figures below are labelled as illustrative where they …

Read more

Indirect Prompt Injection

Gemini 4 Carbon Hit 0% on Indirect Prompt Injection. Is That Enough?

When Google announced Gemini 4 Carbon on 28 October, one line in the safety section stood out. Carbon scored 0.0% on Gray Swan’s Indirect Prompt …

Read more

Chinese Open-Weight Models Compared: DeepSeek vs Qwen & More

by
Chinese open-weight models

Research cutoff: 20 October 2026. Prices, licences and legal status were checked on that date and change often.

On 19 October, MiniMax priced its new M3.1 Flash at $0.15 per million input tokens and $0.60 per million output tokens. Those are, to the cent, the off-peak rates DeepSeek charges for V4.1 Flash. The next day, the US Department of Homeland Security published an order removing DeepSeek products from federal civilian IT systems. That pairing sums up where Chinese open-weight models stand: quality at the top has converged enough that licence terms, serving cost, hardware and the identity of your customer now decide the purchase.

This comparison covers the current release from DeepSeek, Alibaba’s Qwen, Moonshot AI’s Kimi, MiniMax and Zhipu AI’s GLM (sold internationally as Z.ai). It does not name a winner, because the evidence does not support one.

Key Takeaways
  • “Open-weight” covers five different licences. Only DeepSeek V4.1 Flash (MIT) and Qwen3.8-27B (Apache 2.0) come with standard permissive licences. Kimi K3, GLM-5.3, MiniMax-M3 and Qwen’s 2.4-trillion-parameter model all add custom conditions.
  • The downloadable model is not always the model you tested. Qwen3.8’s open-weight flagship is text-only and lacks the hosted Max model’s 1M-token context. MiniMax’s newest model, M3.1 Flash, has no weights yet.
  • Hosted prices for the same workload differ by about 25×. On the example workload below, the bill runs from roughly $167 a month (DeepSeek, off-peak) to $4,110 (Kimi K3).
  • Self-hosting the flagships is a cluster decision. The largest checkpoints need 1.4–4.9 TB of accelerator memory for weights alone. Only the 27B Qwen model fits on a single GPU.
  • Vendor benchmark tables are not mutually comparable. DeepSeek’s and MiniMax’s October tables report different Humanity’s Last Exam scores for the same Anthropic model.
  • The US order on DeepSeek is narrow but real. It covers federal civilian agency systems and procurement, including third-party products “developed using DeepSeek implementations”. The public notice does not extend it to private companies.

Quick Navigation

What this comparison measures

“Open-weight” means the trained parameters can be downloaded. It does not mean the training data or training code is public (none of the five publishes its training data), and it does not mean the licence permits every commercial use. Each family is assessed on its current release, with the downloadable checkpoint and the hosted API treated separately where they differ. Vendor-reported figures are labelled as such.

Master comparison of Chinese open-weight models

Table 1 — Release, weights and licence (checked 20 October 2026)

FamilyCurrent release assessedWeightsLicenceCommercial use
DeepSeekV4.1-Flash-1017 (Oct 2026)DownloadableMITPermitted
QwenQwen3.8-27B; Qwen3.8-2.4T-A95B (Aug 2026)Downloadable27B: Apache 2.0 (reported). 2.4T: custom “Qwen3.8-Max License”Permitted; conditions above stated thresholds for the 2.4T model
KimiKimi K3 (late July 2026)DownloadableCustom “Kimi K3 License” (MIT-derived)Permitted; conditions above thresholds
MiniMaxMiniMax-M3 (June 2026); M3.1 Flash (Oct 2026)M3: downloadable. M3.1 Flash: not yet releasedM3: “MiniMax Community License”M3: commercial use requires notice or written authorisation
GLMGLM-5.3 (weights 28 Aug 2026)DownloadableCustom “GLM-5.3 License”Permitted; security review for hosts above $10 billion revenue

Table 2 — Architecture, context and price

ModelTotal / active parametersContextInputsAPI price per 1M tokens (input / output)
DeepSeek V4.1 Flash552B reported; 763B in repository metadata / 8B–16B active (reported)1MText, image$0.15 / $0.60 off-peak; double at peak
Qwen3.8-27B27B dense262K native, 1M with YaRNText, image, videoNot verified
Qwen3.8-2.4T-A95B2.4T / 95BBelow the hosted model’s 1M (exact figure not published)Text onlyHosted Max: $2 / $6 (reported, not verified)
Kimi K32.8T / 104B1MText, image, video$3.00 / $15.00 ($0.30 cached input)
MiniMax-M3~428B / ~23B1MText, image, videoNot published
MiniMax M3.1 FlashNot disclosedNot stated officiallyText, image (reported)$0.15 / $0.60 ($0.015 cached input)
GLM-5.3~744–753B / ~40B1MNot confirmed (GLM-5.2 was text-only)$1.40 / $4.40

Every model except Qwen’s 27B has converged on a very large mixture-of-experts design with a 1M-token context, so architecture no longer separates them. Hosted prices do: they span more than an order of magnitude, from the two Flash models to Kimi K3.

Chinese open-weight models

What the benchmark tables establish

Less than they appear to. The October model cards from DeepSeek and MiniMax both compare against Kimi K3, GLM-5.3 and Anthropic’s Claude Opus 5, and both use some of the same benchmarks: DeepSWE v1.1, Terminal-Bench 4.0, NL2Repo-Bench and Humanity’s Last Exam (HLE).

Table 3 — Vendor-reported scores on shared benchmarks (October 2026)

BenchmarkDeepSeek V4.1-Flash-1017MiniMax M3.1 FlashKimi K3 (in DeepSeek’s table)GLM-5.3 (in DeepSeek’s table)
DeepSWE v1.177.073.767.566.9
Terminal-Bench 4.042.338.112.637.9
NL2Repo-Bench68.261.258.058.0
HLE (no tools)42.643.443.542.0 (text-only subset)

Sources: DeepSeek V4.1-Flash-1017 model card; MiniMax M3.1 Flash announcement. DeepSeek ran DeepSWE with the Mini-SWE-V2 harness and Terminal-Bench with the Claude Code harness; MiniMax’s footnotes cite several harnesses. All figures are company-reported.

DeepSeek’s table lists Claude Opus 5 at 56.3 on HLE without tools; MiniMax’s lists the same model at 55.6. The gap is small, but it shows that run conditions differ even when the labels match, and the four HLE scores above sit within 1.5 points of each other.

The Kimi row is the clearest warning. In DeepSeek’s table, Kimi K3 scores 12.6 on Terminal-Bench 4.0. Moonshot’s own card reports 88.3 on Terminal-Bench 2.1, an earlier version. Neither figure is necessarily wrong, but they cannot be compared, and a reader skimming two announcements could draw opposite conclusions.

Independent measurement helps a little. Artificial Analysis, as reported by The Decoder, puts DeepSeek V4.1-Flash-1017 at 45 on its Intelligence Index and M3.1 Flash at 42; its own page lists GLM-5.3 at 45. It also found that M3.1 Flash answered 56% of AA-Omniscience questions correctly but was confidently wrong in 93% of the cases it missed, a reliability signal absent from MiniMax’s table. None of this is your codebase or your latency budget. Use benchmarks to decide what to test.

Five families, five different trade-offs

DeepSeek: lowest price, cleanest licence, biggest procurement question

DeepSeek released V4.1 Flash on 10 September and an updated checkpoint, V4.1-Flash-1017, on 17–18 October, under the MIT licence with FP8 weights and a 1M-token context. SiliconANGLE’s launch report describes a 552B-parameter mixture-of-experts model activating 8B parameters on prompts and 16B when generating; the repository metadata shows 763B. DeepSeek has not explained the gap, and the higher figure is the one that sets your memory budget.

Price is DeepSeek’s strongest argument. Since 17 August, rates double during weekday peak hours (09:00–12:00 and 14:00–18:00 Beijing time). Off-peak, V4.1 Flash costs $0.15 per million uncached input tokens and $0.60 per million output tokens, and since 14 September calls to V4-Pro have been served and billed as V4.1 Flash. Two caveats: a nearly five-hour API outage on 15 October, and the US federal order covered below.

Qwen: the only realistic single-GPU option, and a split flagship

Qwen3.8-27B is the only strong small model in any family’s current generation. It is dense and multimodal, handles 262K tokens natively (1M with YaRN scaling), and The Decoder reports an Apache 2.0 licence. Its weights need about 54 GB in BF16 or 27 GB at FP8, so it fits on one 80 GB accelerator.

The flagship, Qwen3.8-2.4T-A95B (2.4 trillion total, 95 billion active parameters), is a different product from the hosted Qwen3.8-Max. Its model card says Max offers “more features, such as vision input & non-thinking support, 1M context length by default, official built-in tools”. The download is text-only, with a shorter context, under a custom licence whose thresholds appear in Table 4.

Kimi: the largest model, the highest price

Kimi K3 is the biggest model in the group: 2.8 trillion total parameters, 104 billion active, a 1M-token context, and text, image and video input. Moonshot trained it with quantisation-aware training, so the released weights are natively MXFP4, which is why a 2.8T model downloads at about 1.56 TB rather than 5.6 TB.

On Moonshot’s platform, K3 costs $3.00 per million input tokens ($0.30 cached) and $15.00 per million output tokens, 20 to 25 times DeepSeek’s off-peak rates. Moonshot reports strong agentic and multimodal results at maximum reasoning effort, including 91.2 on BrowseComp and 84.8 on OSWorld-Verified. Agent and computer-use workloads are where the premium could pay off; only your own tests can confirm it. The MIT-derived licence adds conditions for large resellers and very large products (Table 4).

MiniMax: two models, two different deals

MiniMax currently offers two different things under one brand. MiniMax-M3, released in June, is downloadable: about 428B total parameters, about 23B active, a 1M-token context through MiniMax’s sparse attention design, published in BF16. Its model card reports 80.5 on SWE-bench Verified and 78.1 on MMMU-Pro.

The M3 licence is the most restrictive here. The base grant covers non-commercial use; commercial use needs a one-time notice to MiniMax below $20 million in annual revenue, written authorisation above it, and a “Built with MiniMax M3” label. Serving the model through an API counts as commercial use, and military use is prohibited.

M3.1 Flash, launched on 19 October at $0.15 input, $0.60 output and $0.015 cached input per million tokens, ties DeepSeek on price. MiniMax says “model weights will be released later”. Until then it is an API product with no published licence, parameter count or context window.

GLM: MIT one release, custom the next

Zhipu’s GLM-5.2, released in June, was MIT-licensed and text-only. GLM-5.3 reached the API in August and open weights on 28 August, after a two-week safety hold: roughly 744–753B total parameters, about 40B active, a 1M-token context. Its new licence requires hosts with more than $10 billion in annual revenue to pass Z.ai’s security review. Few buyers will hit that threshold, but it shows a family’s licence can change between point releases.

At $1.40 input and $4.40 output, GLM-5.3 is mid-priced. Artificial Analysis measured 85.5 output tokens per second and a 2.44-second time to first token on Z.ai’s API. Z.ai’s vLLM recipe starts at one node of eight H200 or H20 accelerators for the FP8 checkpoint, which makes it the most practical large model to self-host.

Licences: what “open-weight” permits

A downloadable checkpoint only means you can obtain the file. The table summarises the clauses that decide what you can do with it; it is not legal advice.

Table 4 — Licence conditions that change a deployment decision

ModelRevenue or user triggerWhat it triggersExempt
DeepSeek V4.1 Flash (MIT)NoneKeep the copyright notice—
Qwen3.8-27B (Apache 2.0, reported)NoneNotice and licence retention—
Qwen3.8-2.4T-A95B>100M MAU or >$20M monthly revenue; MaaS/AI work-assistant revenue >$50M per yearDisplay model name; separate licenceInternal use
Kimi K3MaaS revenue >$20M per 12 months; >100M MAU or >$20M monthly revenueSeparate agreement; display “Kimi K3”Internal use; use via Moonshot or certified partners
GLM-5.3Hosting company revenue >$10B per 12 monthsZ.ai security reviewRouting, embedding
MiniMax-M3Any commercial use; >$20M annual revenueNotice; prior authorisation; “Built with MiniMax M3”None stated

The practical split is between internal use and resale. If you run a model only inside your company, the thresholds in the Kimi and large-Qwen licences generally do not apply. If you sell API access to a model, MiniMax-M3 requires permission at any scale, and Kimi K3 and Qwen’s flagship require agreements above their revenue thresholds.

Hosted API versus self-hosting economics

Headline prices are per token. A bill depends on how many of your tokens are input versus output, how many hit the cache, and, for DeepSeek, what time of day they arrive.

Example workload: 1 billion input tokens a month, of which 70% are cache hits, plus 200 million output tokens. This is roughly the shape of an agent or retrieval system that reuses long system prompts.

Table 5 — Monthly API bill for the example workload (prices checked 20 October 2026)

ModelCached inputUncached inputOutputMonthly total
DeepSeek V4.1 Flash (off-peak)700M × $0.003 = $2.10300M × $0.15 = $45.00200M × $0.60 = $120.00$167.10
DeepSeek V4.1 Flash (all at peak)$4.20$90.00$240.00$334.20
MiniMax M3.1 Flash700M × $0.015 = $10.50300M × $0.15 = $45.00200M × $0.60 = $120.00$175.50
Qwen3.8-Max (reported rates)700M × $0.20 = $140.00300M × $2.00 = $600.00200M × $6.00 = $1,200.00$1,940.00
GLM-5.3 (no cache discount assumed)—1,000M × $1.40 = $1,400.00200M × $4.40 = $880.00$2,280.00
Kimi K3700M × $0.30 = $210.00300M × $3.00 = $900.00200M × $15.00 = $3,000.00$4,110.00

The Qwen cache rate assumes explicit cache hits at 10% of the input price, as CloudZero reports for Alibaba Model Studio. GLM-5.3 is shown without a cache discount because Z.ai’s cached rate was not verified; its real bill would be lower. The DeepSeek cache-hit rate is taken from third-party transcriptions of DeepSeek’s pricing page.

Output is the largest line in every bill except GLM’s, where the missing cache discount inflates input. For Kimi K3 it is 73% of the total, so long answers or long reasoning traces widen the gap between vendors well beyond what input prices suggest.

Chinese open-weight models

Self-hosting changes what you pay for. In a mixture-of-experts model, only the active parameters are computed per token, but every parameter must sit in accelerator memory.

Table 6 — Weight memory before KV cache and activations

ModelReleased precisionWeights in memory (approx.)
Qwen3.8-27BBF16~54 GB (~27 GB at FP8)
MiniMax-M3BF16~856 GB (~428 GB if quantised to FP8)
DeepSeek V4.1-Flash-1017FP8~0.55–0.76 TB
GLM-5.3FP8~0.75 TB
Kimi K3MXFP4~1.56 TB (repository size)
Qwen3.8-2.4T-A95BFP8 / BF16~2.4 TB / ~4.9 TB

An eight-GPU H200 server holds 8 × 141 GB = 1,128 GB. GLM-5.3 and DeepSeek V4.1 Flash fit on one such node at FP8 with headroom for a modest KV cache, which is why Z.ai’s recipe starts there. Kimi K3 and Qwen’s 2.4T model do not fit and need either newer accelerators with more memory per device or a multi-node deployment.

To compare a node with an API bill, use monthly node cost = GPUs × hourly rate × 730 hours. At an assumed $3.00 per GPU-hour (a placeholder, not a quote), one eight-GPU node costs 8 × $3.00 × 730 = $17,520 a month, before redundancy, engineers or monitoring. That is about 4.3 times the Kimi K3 API bill for the example workload and more than 100 times DeepSeek’s off-peak bill. Self-hosting pays at much higher volume, under data-residency rules that exclude vendor APIs, or when the licence blocks the hosted route you need.

Utilisation decides the outcome: a node billed for 730 hours that is busy for 200 of them has more than tripled its effective cost per token. Measure throughput on your own prompts before trusting any published tokens-per-second figure.

Government restrictions and procurement

United States, federal civilian agencies. On 20 October 2026, DHS published a notice in the Federal Register (FR Doc. 2026-21447) of an exclusion and removal order under the Federal Acquisition Supply Chain Security Act of 2018. It covers information technology products and services from Hangzhou DeepSeek Artificial Intelligence Co. and its affiliates, plus “third-party products and services developed using DeepSeek implementations”, across federal civilian agency systems and procurement. The public notice gives no compliance deadline, does not say whether self-hosted copies of the MIT-licensed weights count as covered products, and makes no statement about private companies. The signed order sits in the docket (DHS-2026-0628), and federal suppliers should read it directly.

Earlier restrictions. Texas, New York, Virginia and Iowa barred DeepSeek on state government devices or networks in January and February 2025, and the US Navy told personnel not to use it in January 2025. The No Adversarial AI Act, introduced on 25 June 2025, would bar executive agencies from using AI from adversary-country companies; we found no evidence it has been enacted. None of these measures restricts private companies in general, but federal civilian contractors are exposed because the order reaches third-party products built on DeepSeek.

DeepSeek and Huawei. Reports of a hardware partnership are partly substantiated and partly not. In June 2026, a Huawei-led research group said, via the Shenzhen government, that it had post-trained DeepSeek V4-Pro on more than 1,000 Ascend 910C chips; DeepSeek did not comment. Bloomberg reported on 4 September that DeepSeek planned to buy at least 160,000 Ascend 950DT chips for inference while continuing to train on Nvidia hardware. The Information reported design collaboration on future chips. Neither company has confirmed a commercial arrangement.

Table 7 — Procurement checklist by family

CheckDeepSeekQwenKimiMiniMaxGLM
Weights downloadable nowYesYesYesM3 yes; M3.1 Flash noYes
Licence fits intended useMITDepends on model and scaleDepends on scaleNotice or authorisation for any commercial useYes, below $10B revenue
US federal civilian restrictionYes, FASCSA order (Oct 2026)None foundNone foundNone foundNone found
US state device bansTX, NY, VA, IA (2025)None foundNone foundNone foundNone found
Vendor API data locationVendor-operated; confirm in termsAlibaba Cloud regions incl. Singapore, Frankfurt, Virginia (reported)Vendor-operated; confirm in termsVendor-operated; confirm in termsVendor-operated; confirm in terms
Continuity signalFive-hour outage, 15 Oct 2026Hosted flagship ahead of open releaseSingle large modelWeights for newest model pendingLicence changed between point releases

Self-hosting puts data handling under your controls; a vendor API puts it under the vendor’s terms and jurisdiction. Where a model was trained does not by itself make either route safe or unsafe. Confirm the current position with your legal and procurement teams.

Which model fits which workload

  • Startup on hosted APIs: DeepSeek V4.1 Flash and MiniMax M3.1 Flash, the cheapest endpoints by far. Run DeepSeek batch jobs off-peak, and check M3.1 Flash’s factual reliability before using it for question-answering.
  • Self-hosting on one GPU: Qwen3.8-27B, the only current-generation option.
  • Self-hosting on one eight-GPU node: GLM-5.3 or DeepSeek V4.1 Flash; DeepSeek’s MIT licence is simpler unless you sell to US federal civilian agencies.
  • Software engineering: shortlist DeepSeek V4.1 Flash, MiniMax-M3 and GLM-5.3 and test them on your own repository, since vendor coding tables use different harnesses.
  • Chinese-language or multilingual products: none of the cards consulted reports a comparable multilingual benchmark, so test on your own corpus.
  • Reasoning and mathematics: HLE scores sit within 1.5 points, and Kimi K3’s 93.5 on GPQA Diamond has no like-for-like counterpart in the other cards. Run your own evaluation.
  • Multimodal: Kimi K3, Qwen3.8-27B or MiniMax-M3. Qwen’s 2.4T download is text-only, and GLM-5.3’s multimodal support was not confirmed.
  • Regulated or government-facing: exclude DeepSeek from US federal civilian supply chains; for the rest, check licence terms, data residency and vendor support.

A short evaluation protocol. Pick 50–200 tasks that represent real traffic. Run every candidate with the same prompts, tools, context limits and sampling settings. Record pass rate, time to first token, output speed, cost per successful task and failure types, and repeat after each model update.

Conclusion

The five families now differ more on rights and costs than on capability. DeepSeek has the cheapest API and the cleanest licence but sits under a federal civilian procurement order. Qwen has the only single-GPU option, and a flagship download that is narrower than what Alibaba hosts. Kimi posts strong agentic numbers on its own tables at the highest price. MiniMax’s licence requires notice for commercial use. GLM is mid-priced and the easiest large model to self-host. Choosing among Chinese open-weight models in 2026 means choosing which of those trade-offs your workload, budget and customers can accept, and then testing.

Frequently asked questions

What are Chinese open-weight models?

Large language models from Chinese developers whose trained weights can be downloaded and run on your own hardware, chiefly DeepSeek, Qwen, Kimi, MiniMax and GLM. The training data and code are not public, and commercial rights vary by licence.

Are DeepSeek and Qwen models open source?

Partly. DeepSeek V4.1 Flash (MIT) and Qwen3.8-27B (Apache 2.0) use standard permissive licences, but neither family publishes training data, and Qwen’s largest open model uses a custom licence.

Can Chinese open-weight models be used commercially?

Yes for most buyers, with model-specific conditions. MiniMax-M3 needs a notice to MiniMax for any commercial use; Kimi K3 and Qwen’s 2.4T model add conditions only above high revenue or user thresholds.

Which of these models can run on a single GPU?

Only Qwen3.8-27B. The other current flagships need hundreds of gigabytes to several terabytes of accelerator memory.

Is DeepSeek banned in the United States?

Not in general. DHS has ordered federal civilian agencies to remove DeepSeek products, and several states ban it on government devices; the published notice does not extend to private companies or individuals.

Is self-hosting cheaper than using an API?

Usually not at moderate volume. Self-hosting tends to win on data control, licence fit or very high, steady utilisation.

How should I compare benchmark results across vendors?

Only when benchmark version, harness and settings match, which is rare. Use scores to build a shortlist, then test it on your own tasks.

Keep reading

Here are the latest posts from the blog.

Types of AI Models

Types of AI Models in 2026: A Field Guide From LLMs to World Models

Model details checked against official announcements, model cards and documentation on 1 November 2026. Line up four AI models that were current at the end …

Read more

AI Coding Agent

AI Coding Agents Are Getting Expensive. Who’s Paying for the Tokens?

Pricing checked on official vendor pages on 1 November 2026. All USD. Cost scenarios are illustrative and labelled as such. On 28 October, TechCrunch published …

Read more

generative AI architecture

Generative AI Architecture: The 7 Layers That Are Hard to Undo

Published 30 October 2026. API prices checked 28–30 October 2026 on each provider’s official pricing page. Cost figures below are labelled as illustrative where they …

Read more

Indirect Prompt Injection

Gemini 4 Carbon Hit 0% on Indirect Prompt Injection. Is That Enough?

When Google announced Gemini 4 Carbon on 28 October, one line in the safety section stood out. Carbon scored 0.0% on Gray Swan’s Indirect Prompt …

Read more

Agents Are Reshaping Server CPU Design. The Evidence Is Narrower.

by
CPU for AI agents

Modal ran 50 coding tasks one at a time on NVIDIA’s new Vera CPU and on Intel, AMD and AWS chips. Vera was often faster, “but not always, and not by a particularly large margin,” the company wrote. Then Modal packed the same machines with concurrent containers. On that test, it says, Vera finished up to twice as many tasks as an Intel Granite Rapids server.

That gap between the serial and concurrent results is the most useful fact in this month’s argument about the CPU for AI agents. NVIDIA says agents are bringing single-threaded performance back. AMD has announced an agent-focused rack that can hold 32,768 CPU cores. Yet the clearest public data point so far points less at faster cores than at how a processor behaves when hundreds of small jobs compete for memory.

Key Takeaways
  • Confirmed: NVIDIA has shipped its first Vera systems and markets them for agent work. AMD has announced Helion, a rack blueprint planned for November 2026. Both vendors now pitch CPUs specifically for agentic workloads.
  • Reported: Modal measured up to 2x the task throughput on Vera against an Intel Granite Rapids machine. The test used one node and a synthetic benchmark with network calls removed, and nobody has independently checked it.
  • Analysis: Modal attributes its result to memory bandwidth under concurrency, not single-threaded speed. That supports a narrower claim than “single-core performance is back.”
  • Analysis: A faster core only speeds up the CPU-bound share of an agent’s task. When most time goes to GPU inference, external APIs or rate limits, the end-to-end gain is small.
  • Practical: Before buying hardware, trace where one task’s time goes, then load-test at your real concurrency and track p99 latency and cost per successful task.

Quick Navigation

What the Vera, Modal and Helion announcements establish

NVIDIA’s position is the most developed. Vera has 88 custom Olympus cores and 176 threads through what NVIDIA calls spatial multithreading. It supports up to 1.5 TB of LPDDR5X memory at up to 1.2 TB/s, and all of this sits on a single compute die. NVIDIA markets it explicitly for “the CPU work behind agentic AI”: code execution, tool use, sandboxing and orchestration. It says it has hand-delivered the first systems to Anthropic, OpenAI, Oracle Cloud Infrastructure and SpaceXAI. It hasn’t given a date for general availability.

The design argument came in a Wccftech interview with NVIDIA’s Vera team, published on 18 October; the interviewees aren’t named. Their claim is that chiplet-based server CPUs gave up per-core performance to reach high core counts. “What suffered over that time in terms of CPU design was really single-threaded perf and IPC,” the team said. They put Vera at up to 2x faster than leading x86 chips on agentic workloads, and the product page claims “up to 1.8x” on sandbox tasks. Both are vendor figures, with no configuration disclosed.

The first customer data

Modal, which runs sandboxed code for AI agents, published the first customer-side data a day later. It has received its first Vera server and plans to move its Sandboxes product onto the platform. That first requires porting its infrastructure and image builds to Arm, and Modal hasn’t given an availability date.

AMD’s rack-scale answer

AMD’s Helion, announced on 12 October, is an open rack blueprint based on an Open Compute Project design. The CPU-only version holds up to 32,768 Zen 6 cores. Its product page lists three blades: dense 256-core servers, balanced 256-core servers and a 96-core high-frequency “performance” option. A separate configuration adds Instinct MI350P GPUs. AMD says configurations are planned for November and has published no benchmarks.

Together, these establish that NVIDIA and AMD now both sell server CPUs configured specifically for agents. They don’t yet establish that agent workloads, in general, reward per-core speed over core count. AMD hedges by offering both. Its 96-core high-frequency blade is a quiet concession that some agent work wants fast cores, while its 256-core blades bet that most of it wants density.

Why agent execution reaches the CPU

A conventional inference request is short and GPU-heavy. Text arrives, the model generates a response, and the CPU’s job is mostly moving data and scheduling batches. An agent loop is different, because it alternates between model calls and work the model asked for:

Agent request → orchestration → model inference → tool execution → tool response → next decision → … → final response

Each tool step may compile code, run tests, search files, query a database or call an external API. Several of those run on the CPU. Each step also depends on the previous one: the model can’t choose the next action until the tool returns. That serial dependency is what makes per-step latency matter.

Real systems aren’t this tidy. Many agents issue several tool calls in parallel, and some frameworks overlap model inference with tool execution. Many tool calls leave the machine entirely. The diagram is a skeleton, not a description of every agent.

Independent research supports the CPU’s role, with caveats. A study from Georgia Tech and Intel profiled several agent frameworks on an Intel Emerald Rapids host with an NVIDIA B200 GPU. It found that “tool processing on CPUs can take up to 90.6% of the total latency.” The top figure came from retrieval in a RAG pipeline. In the SWE-agent coding workloads, bash and Python execution took between 43.8% and 78.7% of latency, depending on the benchmark. The authors also found throughput saturating on CPU factors, such as core over-subscription, cache coherence and synchronisation, at larger batch sizes.

CPU for AI agents

Where a faster CPU for AI agents helps, and where it doesn’t

The useful question is which resource each step is waiting on. A faster core only shortens steps that are bound by that core.

Step is bound byTypical examplesWhat helpsWhat a faster CPU core does
CPUCompiling, running tests, parsing, local search, orchestration logicFaster cores, more memory bandwidth, less contentionShortens the step directly
GPUModel inference, time to first token, long-context prefillGPU capacity, batching, caching, smaller modelsLittle, beyond feeding the GPU
Network or external serviceWeb search, third-party APIs, remote databasesCaching, parallel calls, closer endpointsAlmost nothing
Rate limitProvider request or token capsBackoff, queueing, higher tiers, fewer callsNothing; it may hit the limit sooner
An illustrative coding agent

Amdahl’s law sets the ceiling. Take an illustrative coding agent, with assumed numbers rather than measured ones. Each of 12 steps spends 3 seconds waiting for remote model inference and 1.5 seconds compiling and testing on the local CPU, for a 54-second task. A core that is 20% faster cuts each CPU step to 1.25 seconds and the task to 51 seconds. That’s a 6% end-to-end gain from a 20% faster chip.

Now change one assumption. Under load, contention for memory bandwidth stretches each 1.5-second CPU step to 3 seconds. The task now takes 72 seconds, a third longer than on an idle machine. In this case, the hardware’s behaviour under concurrency matters far more than its single-thread speed. That is roughly the situation Modal describes.

Tail latency adds a second reason to care. In “The Tail at Scale”, Jeffrey Dean and Luiz André Barroso showed how rare slow responses multiply when a request depends on many components. Their example was a server that answers in 10 ms but takes a second at the 99th percentile. If a request has to wait on 100 such servers, 63% of requests take more than a second. An agent that runs a dozen dependent steps has the same exposure in sequence: a slow p99 on any step becomes the user’s wait.

Reading Modal’s “up to 2x” result

Modal deserves credit for publishing the method and the code. Its Agentless Agent Bench replays oracle solutions from the Terminal-Bench dataset in prewarmed Docker containers, with network calls stubbed out. Throughput is defined as verified task completions per 60 seconds, and the test raises the number of concurrent containers until the machine saturates. That isolates host CPU behaviour cleanly. It also means the benchmark contains no model inference and no external I/O, which a real agent always has.

What the published material doesn’t settle matters for anyone thinking of buying:

  • Concurrency: Modal doesn’t say at which container count the 2x occurs. “Up to” means the gap is smaller elsewhere.
  • Equivalence: Modal calls the Intel Granite Rapids system “equivalent”, but it had 128 physical cores to Vera’s 88. Memory configuration and SKUs aren’t given.
  • Cost and power: nothing suggests the systems were matched on price, power or rack space. Throughput per dollar or per watt could differ in either direction.
  • Latency: Modal says task latency under load was “equal to, or sometimes even lower than” on x86, but publishes no p95 or p99 figures.
  • Sample size: Modal ran a single node, early in testing. No third party has reproduced the result.

None of that makes the result wrong. It makes it a strong hint about one kind of workload: many concurrent, CPU-heavy coding sandboxes on a single server. A purchasing decision would need the same benchmark run at your concurrency, on cost-matched systems, with tail latency reported and with your real tools, including their network waits.

CPU for AI agents

Memory bandwidth, scheduling and the cost of concurrent agents

Modal’s explanation is that Vera’s memory bandwidth “allows for significantly more container workloads to pack per core, without CPU throttling.” That fits the physics. Compiling and test runs are often memory-intensive. When hundreds run at once, cores spend time stalled on memory rather than executing instructions, and adding cores past that point adds waiting, not work. NVIDIA claims Vera has twice the memory bandwidth of x86 CPUs with DDR5, and three times the bandwidth per core.

Scheduling compounds the problem. Hundreds of mostly idle agent sessions, each waking briefly to run a tool, generate context switches and run-queue pressure. The operating system must decide which waiting process gets a core. The Georgia Tech and Intel study found throughput saturating from core over-subscription in two frameworks at a batch size of 128. Its proposed batching and scheduling changes, which need no new hardware, cut P50 latency by up to 1.7x in some configurations. Software fixes come before silicon.

For the business, the unit that matters is cost per successfully completed task at a given latency target. A platform hosting agents sells sessions, not cores. If a server can carry more concurrent sandboxes before p99 latency breaks the service-level objective, each task costs less, whatever the per-core score. That’s the economic case for agent-tuned CPUs, and it depends on concurrency behaviour as much as on clock speed. Our analysis of when adding another agent stops paying for itself applies the same per-task arithmetic to model tokens.

Deciding whether a faster CPU for AI agents is worth paying for

The decision gets easier if you work through four questions in order.

  1. What share of task time is CPU-bound? If tool execution and orchestration on your hosts take less than a fifth of end-to-end time, even a much faster CPU will move task latency by a few per cent. Spend on inference capacity, caching or parallel tool calls first.
  2. Does CPU time grow with concurrency? Compare a step’s duration on an idle host and at production load. If it stretches sharply, the constraint is contention, which points to memory bandwidth, cache size, core count or scheduling, not raw single-thread speed.
  3. Is something else capping throughput? Provider rate limits and slow external APIs put a ceiling on completed tasks that no host hardware can lift. Our reference on AI API rate limits in 2026 sets out where those ceilings sit.
  4. What does a successful task cost at your SLO? Run candidate platforms at target concurrency and divide total cost by tasks completed within the latency target. A chip that wins on throughput but loses on tail latency may not win here.

For a hosted platform running thousands of untrusted coding sandboxes, the answers may well point to a high-bandwidth, agent-tuned CPU. For an enterprise agent that mostly calls SaaS APIs and waits on a remote model, they usually won’t.

What to measure before choosing a CPU for AI agents

Start small. Three measurements answer most of the question:

  • A per-task time breakdown: orchestration, model inference (including time to first token), each tool call, and waits on external services.
  • End-to-end latency at p50, p95 and p99, both idle and at target concurrency.
  • Completed successful tasks per second, along with failure, retry and rate-limit counts.

If the breakdown shows meaningful CPU time, add:

  • CPU utilisation, run-queue length and time spent waiting, to separate busy cores from contended ones;
  • memory bandwidth and cache misses, where the platform exposes them;
  • concurrent sessions and active work per core, to find the packing limit;
  • power and rack constraints, if density is the reason for changing platform.

Then test candidates with your own workload, not a vendor benchmark, using the same metric: cost per successful task at the latency you’ve promised users.

Conclusion

The evidence supports a narrower claim than the headlines. Agent platforms do put real, serial, latency-sensitive work on CPUs, and vendors are now designing and marketing processors for it. But the one public customer benchmark points mainly at memory bandwidth and behaviour under concurrency. Single-thread speed played a smaller part. And it comes from a synthetic test on a single node, with network calls removed.

The defensible takeaway is that the right CPU for AI agents depends on where your agents spend their time. Measure that first. If the answer is CPU-bound tool work under heavy concurrency, agent-tuned chips like Vera and the high-frequency Helion blades are worth testing. If the answer is model inference or someone else’s API, they aren’t where your next dollar should go.

FAQ

Why does CPU performance matter for AI agents?

Agents alternate between model calls and tool work, such as compiling code, running tests or searching files. Much of that tool work runs on the CPU, and each step waits for the previous one. CPU-bound steps add directly to task latency.

Are AI agents more CPU-intensive than traditional inference?

Often, yes. A single chat request is mostly GPU work. An agent task adds tool execution, orchestration and sandbox overhead between model calls. How much depends on the tools: local code execution is CPU-heavy, while calls to external APIs mostly wait on the network.

Does a faster CPU reduce LLM inference latency?

Not much. Model inference runs on GPUs or other accelerators. A faster CPU can help with tokenisation, batching and data movement around inference, but the main gains for agents come from shortening the tool and orchestration steps between model calls.

What does Modal’s reported 2x throughput figure establish?

That on one Vera server, Modal’s synthetic coding benchmark completed up to twice as many tasks as on an Intel Granite Rapids server under heavy concurrency. It doesn’t establish cost or power efficiency, tail latency, or results for agents that spend time on network calls. It hasn’t been independently reproduced.

When should a team prioritise CPU upgrades over GPUs?

When tracing shows that CPU-side tool execution or orchestration takes a large share of task time, and that share grows with concurrency. If most time is spent on model inference, external APIs or rate-limit waits, GPU capacity or software changes will do more.

How should engineers benchmark CPUs for agent workloads?

Replay real agent tasks, including their tool calls, at production concurrency on cost-matched systems. Record p50, p95 and p99 latency, completed tasks per second, failures, and CPU and memory contention. Compare platforms on cost per successful task at a fixed latency target.

Keep reading

Here are the latest posts from the blog.

Types of AI Models

Types of AI Models in 2026: A Field Guide From LLMs to World Models

Model details checked against official announcements, model cards and documentation on 1 November 2026. Line up four AI models that were current at the end …

Read more

AI Coding Agent

AI Coding Agents Are Getting Expensive. Who’s Paying for the Tokens?

Pricing checked on official vendor pages on 1 November 2026. All USD. Cost scenarios are illustrative and labelled as such. On 28 October, TechCrunch published …

Read more

generative AI architecture

Generative AI Architecture: The 7 Layers That Are Hard to Undo

Published 30 October 2026. API prices checked 28–30 October 2026 on each provider’s official pricing page. Cost figures below are labelled as illustrative where they …

Read more

Indirect Prompt Injection

Gemini 4 Carbon Hit 0% on Indirect Prompt Injection. Is That Enough?

When Google announced Gemini 4 Carbon on 28 October, one line in the safety section stood out. Carbon scored 0.0% on Gray Swan’s Indirect Prompt …

Read more

Announced vs Delivered Megawatts: Data Centre Capacity Delivery

by
data centre capacity delivery

Bloomberg reported on 15 October that Nscale’s flagship data centre at Glomfjord, just north of the Arctic Circle in Norway, was running about six months behind schedule. Only 6.4 MW of a planned 30 MW were in operation, according to accounts of the report. The timing matters. A month earlier, the British neocloud had filed to list on the New York Stock Exchange, reportedly targeting a valuation of up to $35 billion. Its filing showed $103.4 billion of contracted value, of which only $2.6 billion was active. Data centre capacity delivery, the slow conversion of announced megawatts into compute that customers use and pay for, is the variable underneath that gap. The industry routinely reports it as if it were a single number.

Key Takeaways
  • The reported numbers give 21.3%, not a completion rate. 6.4 MW running against 30 MW planned is 21.3% of the plan, or a 23.6 MW difference. Neither figure’s definition has been published, so that’s arithmetic, not a progress score.
  • The site’s customer is ByteDance, not Microsoft. Reports citing Bloomberg say Glomfjord is being built for ByteDance. Microsoft’s contracted Norway capacity is at a separate Narvik site due in 2027.
  • Nscale’s own figures disagree. The company’s website lists Glomfjord as “30 MW operational.” Bloomberg reports 6.4 MW running. Both can be true only if they measure different things.
  • Contract value runs well ahead of active revenue. Nscale’s filing shows $2.6 billion active out of $103.4 billion contracted, and about 25,000 active GPUs out of 461,000 active or contracted.
  • Delay costs money before it costs revenue. Bloomberg reports penalties of millions of dollars, settled with better terms on new contracts. The filing lets Anthropic cancel tranches delivered too late.
  • Capacity should be reported by milestone. Announced, energised, commissioned, accepted and serving workloads are different facts, and investors need each one stated separately.

Quick Navigation

Two numbers that don’t measure the same thing

Start with the arithmetic, because it is the only part that can be stated with certainty. 6.4 divided by 30 is 21.3%. The difference is 23.6 MW. What those numbers mean is harder to say.

Bloomberg’s report, as relayed by Quartz and DCD, describes 30 MW as planned and 6.4 MW as running. It doesn’t say whether either figure is IT load (power available to servers), total facility power including cooling, or grid connection capacity. These can differ substantially. A site with a 30 MW grid connection may support noticeably less IT load once cooling and electrical losses are counted.

The company’s own description adds to the confusion. Nscale’s Glomfjord page lists the site as “30 MW operational” and “expandable to 60 MW.” In March 2025, DCD reported that Nscale and InfraPartners would add a 30 MW liquid-cooled building next to the existing 30 MW, set to “go live in Q2 2025.” One plausible reading is that Bloomberg’s planned 30 MW refers to that new building rather than the whole campus. If so, an older facility is operating alongside it. That would make the 21.3% a measure of the expansion only. We couldn’t confirm it, and Nscale declined to comment to Bloomberg, citing its pre-IPO quiet period.

The customer is also different from what’s often assumed. Microsoft is Nscale’s biggest contracted customer overall: its agreements provide for payments of up to about $43.8 billion through December 2033, according to the filing as reported by TokenPost. But reports citing Bloomberg say Glomfjord is being built for ByteDance. Tom’s Hardware found that a ByteDance subsidiary, Spring (SG), accounted for 73% of Nscale’s 2025 revenue through 2,304 Nvidia B200 GPUs at the site. Microsoft’s Norway capacity is at Narvik, a 230 MW joint-venture site with Aker that is due to deliver in 2027.

A seven-step audit of data centre capacity delivery

A megawatt passes through several states before it earns money, and announcements rarely say which one they mean. The seven below aren’t a universal sequence. Contracts can define delivery differently from engineering milestones, and large sites are commissioned in phases. Still, each state answers a different question.

StageWhat it meansThe question it answers
AnnouncedA public statement of planned capacityWhat does the company intend to build?
ContractedA customer has committed to take capacity on agreed termsWho has agreed to pay, and when?
ConstructedBuilding shell, halls and plant are physically completeDoes the facility exist?
EnergisedThe grid connection is live and power reaches the site’s distributionCan the site draw its rated power?
Installed and commissionedServers, accelerators, networking and cooling are installed and tested togetherCan the equipment run at load?
Customer-acceptedThe customer has signed off against contractual testsHas billing, or the contractual clock, started?
Serving workloadsCustomer jobs are runningIs the capacity doing useful work?

The gap between stages 1 and 7 is where pipelines start to look different from reality. Nscale’s filing illustrates the scale: about 25,000 GPUs active out of 461,000 classified as active or contracted. That’s not evidence of anything going wrong. Most of those contracts are for capacity that was never meant to exist yet. But a headline figure that blends all seven stages tells an investor very little about near-term revenue.

data centre capacity delivery

The physical chain behind data centre capacity delivery

Behind every operational megawatt is a chain of dependencies, and a hold-up in one link stops everything after it. The chain runs roughly like this. A grid connection has to be agreed and the network reinforced. High-voltage transformers and switchgear have to be delivered, installed and protected. The building and its cooling and power distribution have to be finished and commissioned. Only then can racks, accelerators and network fabric be installed, tested and handed to a customer.

At Glomfjord, Bloomberg’s reporting points to problems in the middle of that chain, not at the grid connection. According to Quartz’s account, Bloomberg cited slow construction, local regulatory hurdles and coolant leaks. It also cited a fire that began in December when a transformer burned, which Bloomberg drew from public records. A Norwegian safety agency identified 10 violations at the site, and regulators closed the case in September after Nscale made fixes. DCD adds that Nscale towed in a floating hotel, the Kalmar, to house construction workers.

That list establishes what went wrong at this site. It doesn’t establish how much each problem contributed to the six months. A transformer fire is a specific event. That doesn’t make it a sign of the industry-wide transformer shortage that often features in reporting on data centre delays, and nothing in the public record links Glomfjord’s delay to equipment lead times or grid capacity. Likewise, coolant leaks in a liquid-cooled hall are a commissioning failure. They aren’t evidence that liquid cooling is unworkable.

The last stage of the chain is the least visible. A hall can be energised and cooled with its GPUs on site and still not serve a customer. Racks need integrating, the network fabric has to be built and tested, firmware and drivers have to be aligned, and the customer’s acceptance tests have to pass. Even then, the hardware has to be kept busy. As our analysis of why accelerator utilisation decides the cost of every token argued, idle accelerators still have to be paid for. A commissioned megawatt with low utilisation costs much the same and earns far less.

data centre capacity delivery

The Arctic trade-off at Glomfjord

The case for building near the Arctic Circle is real but specific. Nscale says the site runs on “100% renewable power generated by the region’s hydropower dams,” uses adiabatic cooling that “leverages the local climate,” and benefits from a “stable oversupply” of renewable power. These are the company’s claims. The low ambient temperatures are a matter of geography. The power claims haven’t been independently tested in public, and the grid-oversupply claim comes with no figures.

The costs are specific too. Nscale’s own site page describes Glomfjord as “a small industrial town of 1000 people.” A town that size has limited local workforce, accommodation and contractor capacity for a fast build, which is consistent with the reported need to bring in a floating hotel for workers. Specialist equipment and replacement parts have to travel further, and replacing a failed transformer can take longer far from major suppliers. None of this proves that the location caused the delay. It does mean that the conditions that make Glomfjord efficient to run are different from the ones that make it fast to build.

The general lesson is narrower than “Nordic sites are slow.” Cheap, clean power and cold air reduce operating costs once a site is running. They do little to speed up construction, and they can’t make a remote site easier to staff. A buyer comparing sites needs both numbers: the operating cost per megawatt and the date by which that megawatt will be delivered.

Stress-testing the revenue timeline

A delay doesn’t just postpone revenue. It changes when each cash flow happens, and those changes don’t move in step.

Four events are often treated as one: accounting revenue recognition, contractual payment obligations, cash collection, and actual workload consumption. Under many capacity contracts, the customer’s obligation starts at acceptance, not at energisation. Revenue is then recognised as capacity is made available, cash arrives on the invoice terms, and the customer’s workloads may ramp up more slowly than either. Nscale’s Glomfjord contract terms aren’t public, so it’s unknown which of these the delay pushed back, or by how much.

What the filing does disclose is that the risk runs both ways. Nscale says its contracts include delivery milestones, service-level requirements and penalties for delay. Anthropic may terminate a tranche without liability if Nscale fails to deliver it within a specified period after the expected delivery date. Bloomberg reported, through people familiar with the matter, that Glomfjord’s delays left Nscale liable for penalties totalling millions of dollars. Customers were reportedly offered better terms on new contracts instead of collecting them. That’s a quieter cost, but a real one. It shows up as lower future margins, not as a one-off charge.

Here’s an illustrative calculation. It isn’t Nscale’s data. Suppose a provider spends $1 billion building and equipping a hall that is expected to bill $20 million a month from acceptance. A six-month delay defers $120 million of billings, but the capital is already spent and financing costs keep running. At an 8% annual cost of debt on the full $1 billion, six months of interest is about $40 million. That’s money paid with nothing coming in, before any penalties or concessions. The figures are hypothetical. The mechanism isn’t. Capex goes out on the construction timeline, and income arrives on the acceptance timeline.

That mechanism is why delivery risk reaches the balance sheet. Our analysis of how lenders repriced CoreWeave’s AI infrastructure debt found the same gap at a much larger scale. CoreWeave’s $103.7 billion of remaining performance obligations depended on capacity that hadn’t yet been built, while its debt payments were already scheduled. Nscale’s $103.4 billion contract figure is a different measure from CoreWeave’s RPO, but the same question applies to both: how much of it can be delivered before the bills come due?

Reading a capacity pipeline before an IPO

Pipeline figures are hard to compare because they measure different things, and press releases rarely say which. Three recent examples show the range.

Google’s 6 October agreements with Constellation, reported as 3.59 GW, are electricity supply: an 890 MW, 20-year nuclear-uprate power purchase agreement and a 2,700 MW, 15-year supply deal. That’s power secured, not data-centre capacity built. Alibaba’s 20 GW is a stated target for Alibaba Cloud’s data-centre capacity by 2032. It’s stage 1 in the audit above. Nscale’s $103.4 billion is contracted value, measured in dollars rather than megawatts. Adding or ranking these figures against each other would produce a meaningless number.

The same caution applies to compute commitments. As we found in our analysis of the obligation behind Anthropic’s $518 billion compute commitment, a commitment from the buyer’s side depends on capacity that providers like Nscale still have to deliver. A delay at the supplier becomes a timing problem for the buyer.

For anyone reading a prospectus, a short set of questions separates pipeline from production:

  • Which measure? IT load, facility power or grid connection, and is it stated as nameplate or delivered?
  • Which stage? Contracted, energised, commissioned, accepted or serving workloads. How much is at each stage today?
  • When does the money start? At delivery, acceptance or consumption, and what penalties or termination rights apply if the date slips?
  • What’s the conversion record? How have previous sites performed against their announced dates and capacities?
  • Who carries the gap? Is construction financing committed, or does it depend on a capital raise that a delay could make harder?

The last point applies here. Nscale’s filing says it had no binding financing commitments for the Anthropic agreements, which are its largest. That doesn’t mean it can’t get financing. It does mean the delivery timeline and the financing timeline depend on each other.

What data centre capacity delivery reporting should look like

The Glomfjord reports don’t show that Nscale’s business model is broken. A delay of six months at one site, with fires, leaks and inspections along the way, is the kind of thing that happens on complex industrial projects, and a delay isn’t a failure. What the episode shows is how little a single headline megawatt figure tells you. The same campus is “30 MW operational” on the company’s website and 6.4 MW running in Bloomberg’s reporting, and readers can’t tell which figure describes what.

A better standard would be simple: report capacity at each milestone, define the unit used, and give the date. Announced, energised, commissioned, accepted and serving workloads are five different facts. A company preparing to list at a reported valuation of up to $35 billion is better served by publishing all five than by asking investors to trust one. Until the industry reports data centre capacity delivery that way, the gap between announced and delivered megawatts will keep turning up in places like Glomfjord, and readers will find out about it from news reports rather than from the companies.

FAQ

What is the difference between announced and operational data centre capacity?

Announced capacity is what a company plans or has publicly committed to build. Operational capacity is what is energised, commissioned and running workloads. Between the two are construction, grid connection, equipment installation and customer acceptance, and any of them can slip.

Why are data centre projects delayed after they’re announced?

Common causes include grid connection and network reinforcement, delivery and installation of transformers and switchgear, construction and permitting, cooling and power commissioning, and integrating servers and networks. At Glomfjord, Bloomberg reported slow construction, regulation, coolant leaks and a transformer fire. It didn’t say how much each contributed.

How does grid energisation differ from commissioning?

Energisation means the grid connection is live and power can reach the site. Commissioning means the site’s cooling, electrical systems and IT equipment have been tested together at load. A site can be energised long before it is commissioned.

What does a delayed data centre mean for revenue?

It usually defers billing, which often starts at customer acceptance, while construction and financing costs continue. It can also trigger penalties or contract concessions. The exact effect depends on contract terms, which are rarely public.

How should investors read a contracted capacity pipeline?

Ask what unit each figure uses, which milestone it has reached, when payments start, what happens if delivery slips, and how the company’s past sites performed against their announced schedules. Nscale’s filing shows $2.6 billion active out of $103.4 billion contracted, a ratio worth asking about.

Keep reading

Here are the latest posts from the blog.

Types of AI Models

Types of AI Models in 2026: A Field Guide From LLMs to World Models

Model details checked against official announcements, model cards and documentation on 1 November 2026. Line up four AI models that were current at the end …

Read more

AI Coding Agent

AI Coding Agents Are Getting Expensive. Who’s Paying for the Tokens?

Pricing checked on official vendor pages on 1 November 2026. All USD. Cost scenarios are illustrative and labelled as such. On 28 October, TechCrunch published …

Read more

generative AI architecture

Generative AI Architecture: The 7 Layers That Are Hard to Undo

Published 30 October 2026. API prices checked 28–30 October 2026 on each provider’s official pricing page. Cost figures below are labelled as illustrative where they …

Read more

Indirect Prompt Injection

Gemini 4 Carbon Hit 0% on Indirect Prompt Injection. Is That Enough?

When Google announced Gemini 4 Carbon on 28 October, one line in the safety section stood out. Carbon scored 0.0% on Gray Swan’s Indirect Prompt …

Read more

AI Infrastructure Debt: CoreWeave Repricing

by
AI infrastructure debt

In April, CoreWeave sold $1.75 billion of 9.75% senior unsecured notes due October 2031 at par. A week later it added another $1 billion of the same notes, and investors paid 102 cents on the dollar for them. On Thursday, 15 October, CNBC reported the yield to maturity on those notes at 15.78% (as cited by Benzinga). On our arithmetic, that yield corresponds to a price of roughly 80 cents. In six months, the buyers of the add-on have lost about a fifth of their money on paper. CoreWeave hasn’t missed a payment, cut its revenue guidance or defaulted on anything.

That gap is the story. The coupon CoreWeave pays hasn’t changed, and neither has a single dollar of what it owes. What changed is the price at which other people are willing to take on that obligation. This article asks what that price says about AI infrastructure debt: whether lenders are punishing one company after a bad week for its shares, or repricing a business model in which expensive, fast-ageing GPUs must earn enough to cover both the technology and the loan that bought it.

The short answer is that the evidence supports a repricing of financing risk, not a forecast of default. The longer answer depends on three things that are easy to blur: what a GPU is worth to a lender, how quickly contracted demand turns into cash, and what happens when the cost of borrowing rises for a company that must keep borrowing.

Key Takeaways
  • A 15% yield is a price, not a verdict. CoreWeave’s 9.75% 2031 notes yielding 15.78% means they trade near 80 cents. It doesn’t mean a missed payment or a promised 15% return.
  • CDS spreads overstate default odds if read naively. Converting about 1,088 basis points into a default probability depends on recovery assumptions, and the spread also includes a premium for fear and illiquidity.
  • GPU collateral is weaker than its purchase price suggests. Lenders anchored CoreWeave’s secured loans on customer contracts first, a sign of how little they rely on resale value alone.
  • $103.7 billion of RPO is not $103.7 billion of cash. Only 41% is due within two years, and most of it requires capacity that hasn’t been built yet.
  • Refinancing is the pressure point. About $10.6 billion of principal falls due by the end of 2027, so the price of new money matters more than the price of old bonds.
  • The $18 billion junior tranche is Anthropic’s, not CoreWeave’s. It belongs to the Broadcom TPU financing, and its pricing is still unknown.

Quick Navigation

Reading the AI infrastructure debt signal in CoreWeave’s bonds

A fixed coupon, a moving yield

A bond’s coupon is the interest written into the contract. CoreWeave’s June 2026 dollar notes pay 9.625% a year on $1.25 billion until July 2032, whatever happens to the share price. The yield to maturity is different. It is the annual return a buyer would earn by paying today’s market price, collecting every coupon and getting 100 cents back at maturity. When the price falls, the same fixed coupons plus a larger gain at maturity produce a higher yield. Price and yield move in opposite directions.

The numbers reported around 14 and 15 October fit together once that relationship is applied. On Wednesday, CNBC put the 2032 notes at about 14.8%, after touching 15% briefly, and the 2031 notes at about 14.9% (via investingLive). A 9.625% bond maturing in mid-2032 yields about 14.9% at a price of 80, by our calculation using semi-annual compounding and settlement on 15 October. That matches separate reporting, based on Forbes data, that the 2032 notes had fallen below 80 cents. On Thursday the 2031 notes were reported at 15.78% by CNBC and “past 16%” by Reuters. Different dealers, quote times and bid or offer sides easily explain a gap of a few tenths. Corporate bonds trade over the counter, and a quoted yield is often an indicative level rather than a trade.

Two cautions follow. A 15% yield isn’t a promised 15% return. It’s what a buyer earns only if CoreWeave pays every coupon and repays principal in full. If the company restructured, the realised return would depend on what creditors recovered. Suppose a buyer paid 79.8 for the 2031 notes, collected two years of coupons (19.5 points), then recovered 40 cents in a restructuring. That buyer would end up with 59.5 against 79.8 invested, a loss of about a quarter. The 40% recovery figure is purely illustrative. Nor does crossing 15% prove anything on its own. It’s a round number, and round numbers attract headlines more than they mark legal or economic thresholds.

Bonds, CDS and shares measure different things

A credit default swap (CDS) is insurance-like protection on a borrower’s debt. The buyer pays an annual premium, quoted in basis points, and is compensated if a defined credit event occurs. CNBC reported CoreWeave’s five-year CDS briefly above 1,140 basis points on 15 October before settling at 1,088. In other words, protecting $10 million of debt cost about $1.09 million a year. Reuters put the cost at about 1,000 basis points and translated it into a “nearly 63%” chance of default within five years.

That kind of conversion needs its assumptions shown. Assume 40% recovery and a constant default rate, and a 1,088bp spread implies roughly 60% cumulative default probability over five years. Change the recovery assumption and the answer moves a lot. More importantly, CDS spreads also price fear, illiquidity and the cost of hedging, which is why spread-implied probabilities usually run well above realised default rates. The figure is a market price, not an actuarial estimate.

The share price fell 15.46% to $65.78 on Wednesday, then recovered on Thursday after Nvidia’s chief executive, Jensen Huang, said publicly that CoreWeave’s cost of capital was falling. Equity holders get the upside and absorb losses first. Bondholders get at most par. When shares, bond yields and CDS spreads all move the same way, they’re related, but they aren’t measuring the same thing. In this case they did move together, and the bond and CDS markets kept pricing stress while the shares bounced.

AI infrastructure debt

How GPU-backed AI infrastructure debt is secured

Not all AI infrastructure debt is the same, even inside one company. The 2031 and 2032 notes under stress are senior unsecured: they rank ahead of shareholders but have no specific assets pledged to them. Much of the rest is delayed-draw term loans (DDTLs) held in special-purpose subsidiaries. CoreWeave’s Q2 10-Q says these are “collateralized with the assets underlying the contributed contracts and the pledged contractual cash flows, generally from investment grade counterparties.” StockTitan’s reading of the debt note puts DDTL debt at $13.6 billion and unsecured senior notes at $10 billion, out of $35.6 billion of principal at 30 June.

The DDTL lender, in other words, is mainly lending against a customer contract, with the GPUs as a second line of defence. That’s a sensible structure. It also means the unsecured noteholders sit behind lenders with claims on many of the company’s most valuable assets.

One accelerator, five different values

A GPU has several values at once, and they aren’t interchangeable:

  • Purchase price, what CoreWeave paid, often under supplier financing.
  • Carrying value, the purchase price less accumulated depreciation. CoreWeave’s FY2025 10-K depreciates technology equipment over six years.
  • Resale value, what a buyer in the secondary market would pay for that specific card, today, in that condition.
  • Value to a customer, which can be much higher than resale value inside a powered, networked cluster with a contract attached.
  • Collateral recovery value, what a lender actually collects after removal, transport, refurbishment, legal costs and a forced-sale discount.

The last figure is the one that matters in a default, and it’s the one with the least public evidence. As we argued in our piece on why accelerator useful-life assumptions deserve more scrutiny, a six-year accounting life is a management estimate. It isn’t a measurement of how long a chip will earn its keep. An accelerator can keep computing long after newer models outgrow its memory and its rental price collapses.

When AI infrastructure debt outlives its collateral

The risk for a lender isn’t that GPUs stop working. It’s that the loan amortises more slowly than the market value of the hardware falls. If a new chip generation arrives every year or two, the cards backing a five-year loan may be two generations old before half the principal is repaid. Their value also depends on things that don’t travel with them. A rack of liquid-cooled accelerators is worth far more in a data centre with contracted power than on a loading dock. Our analysis of the residual-value problem in the $60 billion AI chip financing made the same point about TPUs: secondary markets for accelerators are thin and untested under stress, and the conditions that push a borrower towards default are likely to depress resale prices at the same time.

We found no public disclosure of the loan-to-value ratios or advance rates on CoreWeave’s DDTLs, so we won’t guess at them. What the filings do show is that lenders chose to anchor these loans on contracts rather than on hardware alone. That choice says something about how much confidence they place in GPU collateral by itself.

AI infrastructure debt

Backlog is not cash

CoreWeave’s chief executive, Mike Intrator, answered the sell-off in a memo to staff. According to Reuters, he argued that the company’s backlog “more than covers” its debt obligations. He also wrote, as Yahoo Finance reported, that “there will be periods of time when the market misunderstands” and “acts irrationally”. The argument deserves to be taken seriously, because on the company’s own numbers it is arithmetically true. It just doesn’t answer the question lenders are asking.

What $103.7 billion of RPO actually promises

The term CoreWeave uses in its filings is remaining performance obligations, or RPO: contracted revenue it has not yet earned because it hasn’t yet delivered the compute. At 30 June the 10-Q reported “$103.7 billion of unsatisfied RPO”. It expected 41% of that to be recognised within 24 months, 39% in months 25 to 48, and the rest by month 78. Set against $35.6 billion of debt principal, that is a ratio of almost three to one.

Here is what sits between that RPO figure and a bond coupon:

  • Revenue must be earned first. Roughly $42.5 billion is scheduled for the next two years, and most of it depends on capacity that hasn’t been built yet. CoreWeave spent $14.1 billion on property and equipment in the first half of 2026 alone, and the 10-Q lists $35.5 billion of future lease payments on data-centre leases that haven’t started.
  • Revenue isn’t free cash. Every dollar recognised carries power, staffing, lease and interest costs. The Q2 net loss was $626 million on $2.58 billion of revenue.
  • Timing doesn’t line up. About $10.6 billion of principal falls due by the end of 2027. The 41% tranche of RPO arrives over the same window, but only as new data halls are energised.
  • Contracts aren’t immune to change. The 10-Q says the transaction price can include “potential reductions” in future. The termination terms in customer contracts weren’t disclosed in the sections we could read.
  • Customer concentration is falling but still high. CoreWeave’s largest customer was 36% of Q2 revenue, down from 71% a year earlier. Reuters names Microsoft and OpenAI among the biggest customers.

There’s also a point in the company’s favour. CoreWeave held $9.7 billion of deferred revenue at 30 June, which is cash customers have paid ahead of delivery. Deutsche Bank told clients on 15 October that roughly two-thirds of a backlog above $100 billion sits with investment-grade counterparties (World Today News, republishing CNBC). A big customer’s appetite for AI spending doesn’t make its contract unreliable. The question isn’t whether the customers will pay. It’s whether CoreWeave can deliver the capacity on time and on budget, while servicing debt raised before that capacity earns anything.

The refinancing loop in AI infrastructure debt

Refinancing means repaying maturing debt with new debt. In the early years, AI infrastructure debt is usually rolled over rather than repaid from profits. A company that is investing faster than it generates cash has to keep doing that, so the market price of its existing bonds becomes an operating variable rather than a scorecard.

The pieces are all in the filings. Q2 interest expense was $640 million, about a quarter of revenue for the quarter, and cash interest paid in the first half was $806 million. Reuters reports that CoreWeave lifted its 2026 capital-expenditure forecast in August to as much as $39 billion. About $4.4 billion of principal is due in the rest of 2026 and $6.2 billion in 2027, according to StockTitan’s reading of the maturity table. Cash at 30 June was $5.5 billion. Since then CoreWeave has added a $2.6 billion DDTL facility and $4.2 billion of 2.875% convertible notes, which shows that at least one door is still open.

Here is an illustrative calculation, not a forecast. If CoreWeave refinanced the $6.2 billion due in 2027 at 15% rather than at the 9.75% it paid in April, the extra interest would be about $325 million a year. That’s roughly half of one quarter’s current interest bill. It’s survivable on its own, but it sets the mechanism in motion. Higher financing costs must be recovered through higher prices or higher utilisation. As our analysis of why accelerator utilisation decides the cost of every token showed, idle capacity is paid for whether or not anyone uses it. A GPU cloud that charges more to cover its borrowing costs loses ground to hyperscalers that fund capex from operating cash flow. Lower prices, in turn, make the next loan harder to justify.

The loop also explains why commercial success doesn’t settle the matter. CoreWeave can win contracts, grow revenue and deliver well, and still find each new dollar of capital more expensive. It runs the other way too. A wide credit spread doesn’t stop the business from executing. A spread at 1,088 basis points mostly means that the next unsecured bond, if CoreWeave chose to issue one, would be expensive. Convertibles at 2.875% are cheaper in cash terms, but they shift some of the cost to shareholders through dilution.

Nvidia’s role cuts both ways. Its 2025 order form commits it to buy qualifying unused capacity through 13 April 2032, from an initial value of $6.3 billion. That supports utilisation. As TokenPost notes, though, it is not a guarantee to CoreWeave’s lenders, and either side can end it under defined conditions. Huang’s public reassurance on 15 October helped the shares, but it doesn’t change any creditor’s claim.

Where CoreWeave differs from the $60 billion Anthropic chip deal

The sell-off didn’t happen in isolation. On 14 October Oracle shares closed down nearly 9% and Nebius fell about 10%, according to CNBC figures carried by investingLive. The same CNBC report put Oracle’s five-year CDS at 288 basis points, up more than 12% on the week. Intrator’s memo itself blamed “weakness and delays surrounding the attempted IPOs and debt financings” of rivals. These moves were correlated, but correlation doesn’t prove a common cause. Oracle is an investment-grade company with a large legacy software business. Nebius has a different balance sheet and depreciates its GPUs over four years rather than six. What they share is sensitivity to one variable: how quickly AI demand turns into cash for whoever owns the hardware.

The comparison most often drawn is with Broadcom’s financing for Anthropic. As reported by Bloomberg and summarised by Quartz on 2 October, banks were assembling about $60 billion of debt. According to Tech Times, the borrower is a special-purpose vehicle that buys Google TPUs from Broadcom and leases them to Anthropic under a five-year, $125.2 billion lease. The structure has a $42 billion Class A senior secured tranche with a partial residual-value guarantee from Broadcom, and an $18 billion Class B junior tranche.

The two have shared risks. Both rest on the value of specialised AI accelerators and on whether an AI customer keeps paying. But the structures are different in ways that matter. The SPV has one lessee and one revenue stream, and the chip vendor stands behind part of the hardware’s value. CoreWeave is an operating company with many customers, data-centre leases, construction risk and a stack of debt in which the stressed notes are unsecured claims on the parent. A lender to the Anthropic SPV is mainly underwriting Anthropic’s credit. A CoreWeave bondholder is underwriting CoreWeave’s ability to build, operate and refinance. The two are not equivalent, and price moves in one say little directly about the other. We covered the obligation side of that deal in Anthropic’s $518 billion compute commitment.

The $18 billion junior tranche belongs to that deal

Some commentary this week has linked an “$18 billion unguaranteed junior tranche” to CoreWeave. The reporting we could verify places it in the Anthropic financing as Class B, with Blackstone committing about $9 billion and syndicating the rest. Junior means it is repaid after the senior tranche. The absence of Broadcom’s guarantee on Class B isn’t stated outright in the reports, and even if it applies, that doesn’t mean the tranche has no collateral or covenants. Those terms haven’t been published. Neither has the pricing. Tech Times reports that Class B lenders may wait for Anthropic’s planned IPO before committing in full.

That makes the tranche a useful gauge of how much appetite there is for subordinated AI infrastructure credit, once it prices. It can’t tell us anything about CoreWeave’s own bonds, and we won’t guess at its clearing level.

What would change the case on AI infrastructure debt

The market’s message about AI infrastructure debt is narrower than the headlines suggest. CoreWeave’s unsecured lenders now want about 15% to hold paper that was issued at under 10% this year. That repricing applies to the model, a highly leveraged GPU cloud funding years of construction ahead of revenue, as much as to the company. It isn’t evidence of an imminent default. CoreWeave’s ratings, as last published in April, were B+ with a positive outlook at S&P and Ba3 at Moody’s. Since June it has raised fresh money, and its largest contracts sit with investment-grade customers.

What the evidence can’t settle yet is whether the gap between contracted demand and cash in hand closes before the 2027 maturities do. Several observable facts would move the assessment:

  • The Q3 10-Q. Watch whether RPO conversion keeps pace with the 24-month schedule, whether deferred revenue grows, and how interest expense compares with revenue.
  • Rating-agency action. Any change from S&P or Moody’s, and the rationale behind it, would be the first independent credit judgement since the sell-off.
  • The next financing. Its form (secured DDTL, unsecured note or convertible) and its price will show which doors remain open, and at what cost.
  • Delivery milestones. Data-centre energisation dates and lease commencements govern when RPO becomes revenue.
  • Secondary GPU prices. Prices for previous-generation accelerators are the only market test of collateral value, and they’re poorly documented today.
  • Class B pricing in the Anthropic deal. This is the clearest gauge of appetite for subordinated AI credit, even though it isn’t CoreWeave’s debt.

If the next quarter shows RPO converting on schedule, with financing still available at falling spreads, Huang’s claim that capital is getting cheaper would start to look right. If deliveries slip while 2027 maturities approach, the bond market’s 15% will look like the more accurate forecast.

Frequently Asked Questions

What is AI infrastructure debt?

It’s borrowing used to buy AI accelerators and build or lease the data centres that run them. It includes corporate bonds, loans secured on equipment and customer contracts, supplier financing, and lease obligations.

Why can a bond’s yield rise above its coupon?

The coupon is fixed in cash terms. If the bond’s price falls below 100, a buyer collects the same coupons plus a gain at maturity, so the yield rises. CoreWeave’s 9.625% 2032 notes yield about 15% at a price near 80.

Does a 15% bond yield mean CoreWeave will default?

No. It means investors want a high return to hold the risk. Default depends on whether CoreWeave can meet its payments and refinance. Its last published ratings, from April 2026, were B+ (S&P) and Ba3 (Moody’s).

How does GPU-backed financing work?

Typically a subsidiary borrows against a pool of GPUs and the customer contracts they serve. Lenders take security over both, and the contract cash flows repay the loan. CoreWeave’s DDTLs follow this pattern.

Why is GPU collateral hard to value?

Accounting value follows a fixed schedule (six years at CoreWeave), but resale value depends on new chip generations, condition, power and cooling, and whether there’s a buyer in a downturn. Public data on forced-sale prices is thin.

Is contracted AI backlog the same as available cash?

No. RPO is revenue still to be earned by delivering compute. It turns into cash only as capacity comes online and customers are billed, and the costs of delivering it come first.

What should investors watch next?

Quarterly RPO conversion, any rating action, the cost and form of CoreWeave’s next financing, data-centre delivery dates, and secondary-market GPU prices.

Keep reading

Here are the latest posts from the blog.

Types of AI Models

Types of AI Models in 2026: A Field Guide From LLMs to World Models

Model details checked against official announcements, model cards and documentation on 1 November 2026. Line up four AI models that were current at the end …

Read more

AI Coding Agent

AI Coding Agents Are Getting Expensive. Who’s Paying for the Tokens?

Pricing checked on official vendor pages on 1 November 2026. All USD. Cost scenarios are illustrative and labelled as such. On 28 October, TechCrunch published …

Read more

generative AI architecture

Generative AI Architecture: The 7 Layers That Are Hard to Undo

Published 30 October 2026. API prices checked 28–30 October 2026 on each provider’s official pricing page. Cost figures below are labelled as illustrative where they …

Read more

Indirect Prompt Injection

Gemini 4 Carbon Hit 0% on Indirect Prompt Injection. Is That Enough?

When Google announced Gemini 4 Carbon on 28 October, one line in the safety section stood out. Carbon scored 0.0% on Gray Swan’s Indirect Prompt …

Read more