Inkling 975B: What the Open Weights Now Really Change

Inkling 975B: What the Open Weights Now Really Change

Thinking Machines Lab shipped Inkling on July 15, 2026. It runs 975 billion parameters. It ships under Apache 2.0. And it is the best open weights any US lab has put out.

That last sentence about open weights is doing a lot of quiet work. Inkling debuted at 41 on the Artificial Analysis Intelligence Index. Kimi K3 sits at roughly 57. GLM-5.2 sits at 51.

So the leading American open weights release lands third or fourth in its own category. The interesting question is not whether Inkling wins. It is what a 975B model with a permissive license actually changes for anyone downstream.

The honest answer: less than the launch suggests, and not what most coverage claims.

Key Takeaways on Frontier-Scale Open Weights

  • Inkling is a 975B-parameter MoE with 41B active per token, a 1M-token context window, and Apache 2.0 terms.
  • Running it at BF16 needs roughly 2TB of aggregated VRAM. Downloadable does not mean runnable.
  • Six labs shipped open weights above 100B in 2026. Five of them are Chinese.
  • licenses diverged sharply this year. Kimi K3 dropped Modified MIT for a bespoke document with a $20M revenue gate.
  • The real shift from open weights is control over deployment, not access to capability.

Quick Navigation

What Inkling 975B Actually Is

Start with the specification, because the shape explains why these open weights exist.

Inkling is a Mixture-of-Experts transformer: 975B total parameters, 41B active per token. It runs 66 decoder layers with 256 routed experts plus 2 shared experts, and routes each token to 6 of the routed set. If those terms are unfamiliar, our glossary covers MoE and context windows.

Pretraining used 45 trillion tokens of text, images, audio, and video. Training ran on NVIDIA GB300 NVL72 systems, using Muon for large matrix parameters and Adam for the rest.

Two design choices stand out in the architecture notes. Short convolutions inside the attention block give an explicit path for mixing nearby tokens. A separate RMSNorm sits directly after the embedding lookup.

Neither ships with an ablation, so their contribution is unmeasured.

The third choice is the practical one. Most post-training compute went to asynchronous reinforcement learning past 30 million rollouts, and that run produced a controllable effort dial. You set reasoning_effort and the model spends a different token budget.

The launch post is unusually candid. The company states plainly that Inkling is not the strongest model available today, open or closed.

That framing matters. The pitch is breadth and fine-tunability, not a leaderboard position. Open weights here are a distribution strategy, not a capability claim.

Every Open-Weight Release Above 100B in 2026

Here is the full open weights field, ordered by ship date. Parameter shape and license are the two facts that do not go stale in a week.

ModelLabReleasedTotal / ActiveContextLicence
Kimi K2.6Moonshot AI (CN)20 Apr 20261T / 32B256KModified MIT
DeepSeek V4DeepSeek (CN)24 Apr 2026 (preview)1.6T and 284B variants1MMIT
Mistral Medium 3.5Mistral (FR)29 Apr 2026128B dense—Modified MIT
MiniMax M3MiniMax (CN)1 Jun 2026Not disclosed1MNot verified
Kimi K2.7 CodeMoonshot AI (CN)13 Jun 2026~1T / 32B—Disputed (see below)
GLM-5.2Z.ai / Zhipu (CN)mid-Jun 2026744B / ~40B1MMIT
InklingThinking Machines (US)15 Jul 2026975B / 41B1MApache 2.0
Kimi K3Moonshot AI (CN)weights 26–27 Jul 20262.8T / 104B1MCustom “Kimi K3 License”

Caveats on the table

Three entries need flags, and no other roundup I found carries them.

MiniMax M3. Parameter count is not publicly specified in the sources I could verify. It is included on the strength of frontier positioning, not a confirmed figure.

Kimi K2.7 Code. Sources disagree on license. Some list Apache 2.0, others list the Modified MIT that governs the K2 line. Check the model card before building on it.

Kimi K3 dates. Hosted launch was 16 July. Weights landed 26 July, one day ahead of the stated 27 July target. Both dates appear in coverage.

Two exclusions from the open weights table

Gemma 4 is excluded from the open weights table because its largest variant is 31B dense. Llama 4 Scout is excluded as a 2025 release.

What the Licence Column Tells You About Open Weights

Read that table by license rather than parameter count and a different picture appears.

Apache 2.0 and MIT are permissive licences, not unconditional ones. They permit commercial use without a revenue or user-count gate, but Apache 2.0 and MIT have notice and redistribution conditions. Check the actual licence shipped with each model rather than treating “open weights” as a single legal category.

That is the strongest thing about the Inkling release. Among frontier-scale open weights, Apache 2.0 with no attached usage policy is the cleanest set of terms on offer.

Modified MIT sounds permissive, and mostly is. Still, open weights with a clause are not weights without one. The K2-family clause requires prominent “Kimi K2” attribution once a product passes 100 million monthly active users or $20 million in monthly revenue.

Most teams will never hit that. But it is a term, and terms compound across a stack.

Kimi K3 broke the open weights pattern. Moonshot replaced Modified MIT with a bespoke license, and the change went largely unexamined in launch coverage.

Simon Willison flagged the K2 lineage, and the K3 document goes further. Reports say firms above $20 million in annual revenue must sign a contract with Moonshot before offering K3 to outside customers as a service. Attribution rules apply on top.

So the largest open weights model in the world is not open by the Open Source Initiative definition. Neither is it uniquely restrictive. It is a commercial license wearing an open label. So read it with a lawyer, not a skim.

What Open Weights Actually Change

Three things genuinely shift when open weights ship publicly. None of them is “everyone can now run frontier AI.”

Open weights give you deployment control

Open weights let you choose where the model runs. That is the whole thing, and for regulated buyers it is enormous.

A hospital, a bank, or a defense firm can keep the model inside its own walls. No data leaves. The vendor never sees a prompt, and API terms cannot change under you mid-contract.

Price discipline

So open weights cap what closed vendors can charge for the same capability. When GLM-5.2 delivers similar coding performance at a fraction of frontier pricing, that becomes the reference point in every procurement conversation.

Still, the effect reaches teams who never self-host. They simply negotiate better, because open weights set the floor.

Modification rights

Fine-tuning on your own data is the pitch behind Inkling. Tinker exists to make that path short, and a broad base model adapts to more workflows than a narrow one.

Distillation matters here too. Permissive weights can support developing a smaller student model, subject to the applicable licence conditions and any other rights involved. Apache 2.0 should not be treated as a blanket promise that every resulting asset is owned free of third-party obligations.

What Open Weights Do Not Change

Now the correction, because open weights coverage overclaims here.

Open weights access is still gated by hardware

Inkling open weights need roughly 2TB of aggregated VRAM at BF16. NVFP4 quantization cuts that substantially but requires SM100-class hardware.

Kimi K3’s checkpoint runs 1.56TB across 96 shards. Self-hosting it realistically means eight to sixteen nodes of eight H100 or B200 accelerators.

So “open” in open weights describes the license, not the barrier. Memory and interconnect economics still set the ceiling, and those have not moved because a download link appeared.

Community quantization helps at the edges. One 1-bit GGUF cut Kimi K3 from 1.56TB to 594GB, keeping about 79% accuracy. Still a serious machine, though.

Reproducibility is not included

Thinking Machines says open weights rather than open source, and the distinction is precise. Training data and the training pipeline stay private.

Every release in that table does the same. So you get the artifact, never the recipe. That is the hard limit on what open weights can prove. So you cannot audit what went in, verify contamination claims, or rebuild the model from scratch.

Benchmarks still need care

Inkling’s Terminal Bench 2.1 figures come from an internal harness, while competitor scores are self-reported. Those are not directly comparable.

It also trails GLM-5.2 and Kimi K2.6 on HLE, Terminal Bench, and SWE-bench Verified, and posts 43.9% on SimpleQA Verified against DeepSeek V4 Pro’s 57.0%. We covered why leaderboard gaps at this level are hard to interpret.

One number does stand out. Inkling posts the highest FORTRESS adversarial score among compared open-weights models at 78.0%, which matters more for regulated deployment than another point of coding accuracy.

What Inkling-Small Changes

The open weights nobody can download yet may matter more than the ones that shipped.

Thinking Machines Lab released Inkling-Small and its full weights on 30 July 2026. It has 276B total parameters and 12B active. The released model’s model card should be used for specifications and evaluation settings, rather than carrying forward preview benchmark numbers.

Read that twice. So the small model wins on some benchmarks, while the big one carries the headline.

Why size beats score here

Active parameters describe the computation used per token, not the complete weight-memory requirement. Inkling-Small still has 276B total parameters. At BF16, weights alone imply roughly 552 GB before runtime overhead; quantization and offloading change that estimate. Hardware suitability therefore depends on precision, memory, context and serving configuration, not the 12B-active label alone.

The full Inkling-Small weights are now published. The practical access question has moved from release timing to the memory, throughput and operating-cost tradeoffs of a real deployment.

The Fine-Tuning Economics Behind Open Weights

Thinking Machines is not really selling open weights. It is selling a customization pipeline.

Tinker exists to make fine-tuning short. So Inkling was trained broadly rather than narrowly, because open weights only pay off if people adapt them. Breadth adapts to more workflows than a specialist base does.

Fine-tuning a frontier-scale model on your own data is expensive, and the result is yours. Calling a closed API is cheap per token, and the result is rented.

Open weights change which side of that trade is available. But switching costs rise once you adapt a model. So the lock-in moves rather than vanishing.

Why Five of Six Frontier Open Weights Are Chinese

The geography of frontier open weights is the most under discussed fact in that table.

Moonshot, DeepSeek, Z.ai, and MiniMax all ship at this scale routinely. American labs mostly do not, and Inkling is notable partly because it breaks a pattern.

Open weights are a share-capture move when you are behind on distribution. A downloadable model gets into stacks that would never sign an API contract with a Chinese vendor.

Export controls push the same way. If you cannot match a rival’s compute budget, giving the weights away buys reach instead. Reach compounds differently than revenue does.

But adoption of open weights is not purely technical. Moonshot has faced accusations, including from the White House OSTP director, that K3 was distilled from a competitor’s model. Those claims are unresolved.

Regulated US buyers weigh provenance alongside benchmarks. That is the gap Inkling aims at, even with a lower index score.

The Safety Argument Around Open Weights

The Safety Argument Around Open Weights

Publishing open weights is irreversible, and that drives most of the disagreement.

Once a checkpoint is out and mirrored, no vendor can pull it back or patch it. And anyone with modest compute can fine-tune the safety training away.

Critics argue frontier open weights hand capability to actors who could not build it. That concern centres on cyber and biological uplift, and it does not depend on the license at all.

Supporters point out that inspection requires access. Outside researchers cannot audit a model they can only query through a filtered API.

The MarkTechPost breakdown notes Thinking Machines flags role-play and indirect prompts as residual risks in its own project page. That kind of published limitation is only possible when someone can test for it.

Evidence has not settled either side, so this post will not settle it either. But the FORTRESS score in the Inkling release suggests labs are starting to compete on adversarial robustness, which is a healthier signal than benchmark parity.

How to Choose Among 2026 Open Weights

Skip the leaderboard for a moment. Four questions decide most open weights selections.

What can you actually run? Start with available VRAM, then filter the open weights list. A 744B model you can serve beats a 2.8T model you cannot.

What does the license require at your scale? Check revenue and user thresholds against your projections, not your current numbers. MIT and Apache 2.0 have neither.

Do you need the weights, or just the price? If you will call an API anyway, open weights matter to you only as negotiating leverage.

How much does provenance matter? For some buyers it decides everything, and no benchmark will move them. Open weights from a US lab answer a question a score cannot.

Conclusion: Open Weights Changed the Contract, Not the Compute

Inkling is a good model with excellent terms. Yet it will not top a leaderboard, and its makers said so first. Thinking Machines said so themselves, which is more than most launches manage.

The significance of these open weights sits elsewhere. The 2026 field now offers genuine frontier-scale capability under Apache 2.0 and MIT, which was not true two years ago.

But the constraint moved rather than disappearing. Access to weights is now free. Access to the two terabytes of memory needed to serve them is not, and that gap decides who actually benefits.

Watch deployment results for Inkling-Small now that its weights are available: memory use, useful throughput and cost at the required quality. Separately, compare the actual commercial and redistribution terms of each model licence rather than treating different open-weight releases as legally interchangeable.

What This Table Will Look Like in Six Months

Two forces will reshape it, and they pull opposite ways.

Scale keeps climbing. Kimi K3 crossed the 3-trillion class, so the next tier arrives before year end. Yet each jump narrows the pool of buyers who can serve the result.

Smaller active-parameter counts can reduce compute per token. Adoption still depends on total weight storage, usable throughput and the workload, so both active and total parameters belong in the comparison.

So expect this open weights table to split in two. One row set for labs proving capability, another for models people genuinely run.

FAQ About Inkling and 2026 Open Weights

What licence does Inkling use?

Apache 2.0, with no attached usage policy. That permits commercial deployment, fine-tuning, and redistribution without royalty or revenue thresholds. Thinking Machines describes the release as open-weights rather than open source, because training data and the training pipeline are not published.

What hardware do you need to run Inkling 975B?

Running the open weights takes roughly 2TB of aggregated VRAM at BF16. NVFP4 W4A4 quantization reduces that considerably but requires SM100-class hardware or newer. In practice this means a multi-node GPU cluster, not a workstation.

Is Inkling better than Kimi K3 or GLM-5.2?

Not on aggregate benchmarks, though license terms differ. Inkling debuted at 41 on the Artificial Analysis Intelligence Index against roughly 57 for Kimi K3 and 51 for GLM-5.2. It leads on FORTRESS adversarial robustness at 78.0% and carries the cleanest license of the three.

Which 2026 open weights have the most permissive license?

Inkling uses Apache 2.0. Apache 2.0 and MIT permit commercial use without revenue/user gates, but include notice and redistribution conditions. They should not be described as having no attribution requirements. Verify the exact current licence for each other model before relying on the comparison.

Does open weights mean open source?

No. Open weights means the trained parameters are downloadable. Open source, under the Open Source Initiative definition, additionally requires the training data and code, and a license without discriminatory conditions. Every frontier-scale release in 2026 publishes weights only.

Why do Chinese labs release open weights more often?

Distribution strategy under constraint drives most open weights releases. Open weights get a model into stacks that would not sign a vendor contract, and they convert compute-limited capability into ecosystem position. Export controls make that trade more attractive than competing on API revenue alone.

Keep reading

LLM context window

LLM Context Window Comparison 2026: Limits and Costs

Specifications and prices checked October 11, 2026 · Prices in USD per million tokens, standard API tier Short answer: In an LLM context window comparison …
LLM inference latency

LLM Inference Latency by Model and Provider: Real Numbers

Published October 2026 · Data read October 11, 2026 · Source: Artificial Analysis provider benchmarks, secondary research The same open-weight model, Llama 3.3 70B, starts …
GPU cloud pricing

GPU Cloud Pricing 2026: What an Hour Actually Costs

Published: October 2026 · Pricing last verified: October 10, 2026. Provider pricing and availability may change; check the linked source before committing to a workload. …
AI API rate limits

AI API Rate Limits in 2026: The Numbers That Bind

Two teams call the same model on the same tier. One runs a support chatbot and never sees a 429. The other runs a five-step …

The AI Glossary: 10 Terms You Now Meet Everywhere

AI glossary

Most AI writing assumes you already know the vocabulary. This AI glossary fixes that.

Below are ten terms from the AI glossary that show up constantly in chip news, model launches, and filings. Each entry gives a plain definition first, then the number or fact that makes it matter.

This is batch one. The AI glossary will grow, and every term here links from its first mention across the site.

Why This AI Glossary Exists

Technical vocabulary moves faster than the explainers do, which is the whole case for an AI glossary.

Take KV cache. It went from research jargon to procurement conversation in about eighteen months. Nobody wrote the bridging definition, so readers either already knew or quietly skipped the paragraph.

This AI glossary is the bridge. Each definition is written for someone competent who simply has not met the term yet, which is a very different audience from a beginner.

A glossary also gives readers a stable reference when an article uses unfamiliar terminology. A clear definition and a direct link help readers check the meaning in context.

How to Read This AI Glossary

Every AI glossary entry follows the same shape, so you can skim or read in order.

The first line is the definition. Read only that if you are in a hurry, then move on. The second paragraph in each AI glossary entry gives context: a figure, a date, or a trade-off. That is where the actual understanding lives.

This AI glossary groups terms by layer, from silicon upward. So the AI glossary reads as a stack, not an alphabet.

Key Takeaways From This AI Glossary

  • This AI glossary starts with memory. HBM and LPDDR solve opposite problems: bandwidth versus cost per gigabyte.
  • MoE, distillation, and quantization all shrink the cost of a model, each in a different way.
  • KV cache, not model size, is usually why long context gets expensive.
  • RAG and MCP sit above the model. One supplies documents, the other supplies tools.
  • Inference is where most AI money goes across a model’s life.

Quick Navigation

AI Glossary: Memory and Hardware

Memory decides what a chip can hold and how fast it feeds the math. So this AI glossary starts there.

HBM (High Bandwidth Memory)

HBM is DRAM stacked in vertical layers and wired close to the processor, trading capacity for very high bandwidth.

The current generation matters if you read chip news for buying signals. HBM4 entered mass production in February 2026, and the JEDEC JESD270-4 standard doubles the interface from 1,024 bits to 2,048 and lifts channels from 16 to 32. Nvidia’s Rubin platform is expected to pair eight stacks for 288GB and over 22 TB/s. Samsung and SK Hynix together supply roughly 90% of it, and each vendor roadmap now stretches to HBM4E.

LPDDR (Low-Power Double Data Rate)

LPDDR is mobile-class DRAM tuned for power efficiency and capacity rather than peak bandwidth.

Think phones, laptops, and edge boxes rather than data centre racks. LPDDR delivers far less bandwidth than HBM, but costs a fraction per gigabyte and draws much less power. That is why small serving appliances use it while data centre racks do not. We covered how memory choice shapes accelerator margins here.

AI Glossary: Model Architecture

These three AI glossary terms all answer one question. How do you make a capable model cheaper to run?

MoE (Mixture of Experts)

MoE splits a model into many expert sub-networks and routes each token to only a few of them, so total parameters far exceed the parameters used per token.

DeepSeek-V3 shows the gap plainly: about 671 billion total parameters, roughly 37 billion active per token. Compute per token drops sharply. Memory does not, because every expert must stay loaded and ready.

Distillation

Distillation trains a small student model to copy the behavior of a larger teacher model.

The idea dates to a 2015 paper from Hinton and colleagues. It is now routine. DeepSeek shipped R1-distilled versions built on Qwen and Llama bases. Students typically land a few points below the teacher while costing far less to serve.

Quantization

Quantization stores weights and activations at lower numeric precision, such as 8-bit or 4-bit instead of 16-bit.

Cutting from 16-bit to 4-bit roughly quarters the memory a model takes up. Formats like GPTQ, AWQ, and GGUF made this routine, and FP8 and FP4 now run natively on recent accelerators. You lose a little accuracy and gain a lot of bandwidth headroom.

AI Glossary: Runtime and Serving

Now the AI glossary terms that describe what happens when a model actually answers something.

Inference

Inference is running a trained model to produce an output, as opposed to training it.

It splits into two phases with different bottlenecks. Prefill processes the prompt and is compute-bound. Decode generates tokens one at a time and is memory-bandwidth-bound. Serving dominates a model’s lifetime cost, which is why so much hardware design now targets this half of the AI glossary rather than training.

KV cache

The KV cache stores key and value tensors from tokens already processed, so attention does not recompute them at every step.

Skipping that work is what makes generation fast. The cost is memory, and it grows linearly with sequence length and batch size. At long context the KV cache often consumes more memory than the model weights themselves, which is why techniques like grouped-query attention and paged attention exist.

Context window

The context window is the maximum number of tokens a model can consider at once, counting both the prompt and the output.

Windows now run from a few thousand tokens to over a million. But a large window is a ceiling, not a promise. Retrieval accuracy often degrades well before the stated limit, and benchmark figures rarely capture that. Every extra token also enlarges the KV cache.

AI Glossary: Retrieval and Tooling

These last two AI glossary terms sit above the model. Neither changes the weights.

RAG (Retrieval-Augmented Generation)

RAG fetches relevant documents from an external store and places them in the prompt, so the model answers from supplied evidence rather than memory alone.

The approach comes from a 2020 paper by Lewis and colleagues at Facebook AI. It remains the cheapest way to give a model fresh or proprietary information without retraining. Quality depends far more on the retrieval step than on the model, which teams consistently underestimate.

MCP (Model Context Protocol)

MCP is an open standard that lets AI applications connect to external tools and data through a common client-server interface.

Anthropic released it in November 2024. OpenAI, Google, and Microsoft have since adopted it. The 2026-07-28 specification made the protocol stateless, which lets servers scale on ordinary HTTP infrastructure. Security is still maturing: the NSA published design considerations in May 2026 flagging gaps around prompt injection and tool poisoning.

AI Glossary: Terms People Mix Up

Four pairs cause most of the confusion in any AI glossary. Sorting them beats adding ten more definitions.

Inference versus training in this AI glossary

Training builds the model once, over weeks, on a cluster. Inference runs it billions of times afterwards. The AI glossary treats them separately because the hardware, the bottleneck, and the cost curve all differ.

Both terms shrink cost, but not the same way. Quantization keeps the same model and stores its numbers less precisely. Distillation builds a genuinely smaller model that imitates a bigger one. You can do both to the same system.

The context window is a limit set by the model, not by your hardware. The KV cache is the memory actually consumed while operating inside that limit. A vendor advertises the first. Your infrastructure bill reflects the second.

RAG versus MCP in the AI glossary

RAG brings documents to the model. MCP lets the model reach out to tools and systems. One is read-only context; the other is an action interface. Many production stacks run both, which is why this AI glossary lists them side by side.

AI Glossary: How the Ten Terms Fit Together

AI glossary

Read the AI glossary as a stack and the relationships get obvious.

At the bottom of the AI glossary sits memory. HBM and LPDDR decide how fast weights can reach the math units, and everything above inherits that ceiling.

Above memory sits architecture, the middle band of the AI glossary. MoE, quantization, and distillation are three different strategies for fitting more capability under the same memory ceiling.

Above architecture sits runtime, where most questions actually arise. Inference, KV cache, and context window describe what happens while a request is being served, and where the memory actually goes.

At the top of the AI glossary sits the application layer. RAG and MCP never touch the weights. They shape what the model sees and what it can act on.

So a change at the bottom of this AI glossary propagates upward. Wider HBM interfaces make longer context affordable, which makes larger retrieval payloads practical, which changes what RAG systems can attempt.

How the AI Glossary Connects Across the Site

Use this glossary alongside the related articles to move between a term’s definition and its practical consequences.

When an article links a term to this page, the fragment link opens the relevant definition.

A definition is a starting point. Follow the related technical discussion when the implementation details matter.

The related articles explain how memory, model architecture and serving choices affect practical systems.

A reader arriving at a chip-economics article can use the glossary to check a term and then return to the original discussion.

How This AI Glossary Is Marked Up

Structure matters as much as wording when machines read a page.

Each entry uses schema.org DefinedTerm, and all ten sit inside a single DefinedTermSet. That tells crawlers and language models that this is a controlled vocabulary, not a listicle.

The pairing matters. A lone DefinedTerm is a fragment. Wrapped in a DefinedTermSet with a stable URL, the AI glossary becomes a citable reference object that can be extended without breaking anything.

Each AI glossary term also carries a termCode and its own anchor, so external pages can link straight to one definition.

How to Check and Cite a Definition

When citing a definition, include the term’s direct link and check that the wording fits the context in which you are using it.

Read the short definition first, then the accompanying explanation and any linked source. A concise definition still needs the qualifications that follow it.

For a hardware specification, follow the source to check the product generation, units and operating conditions.

A numerical specification is more useful when its source and scope are clear. Do not treat a component specification as a guarantee of whole-system performance.

“HBM4 doubles the interface to 2,048 bits” is checkable. “HBM is very fast” is not. The AI glossary aims for the first kind throughout.

Every AI glossary entry has a permanent fragment link. Anything that cites this page can point at the exact definition rather than the whole document.

Hardware terms age fastest. HBM moved through three generations in four years, and the numbers quoted above will shift again.

Before quoting a changing specification, check the date and version in the linked source. A page’s modification date alone does not prove that every entry has been revalidated.

Software terms age differently. RAG has meant roughly the same thing since 2020, while MCP changed its transport layer twice in eighteen months.

What Batch 2 of the AI Glossary Adds

Ten terms is a start, not a reference work. The AI glossary is built to extend. The next batch covers the gaps this one leaves.

Planned entries include speculative decoding, FlashAttention, LoRA, tokenizer, embedding, vector database, agentic loop, guardrails, eval, and TCO. Each will follow the same two-part shape.

The AI glossary grows in batches rather than singly, since a set update is one schema change instead of ten.

Who This AI Glossary Is For

Three readers, roughly, and the entries serve all three. The first is an engineer who knows the stack but not this corner of it. A backend developer meeting KV cache for the first time needs one paragraph, not a tutorial.

The second is an investor or analyst reading chip filings. For them the number attached to each term matters more than the mechanism.

The third reader of this AI glossary is a language model answering somebody else’s question. That reader is new, and it changes how definitions should be written: state the thing plainly, attach a checkable fact, and skip the throat-clearing.

Conclusion: Use the AI Glossary as a Reference, Not a Read

Nobody reads an AI glossary front to back, and this one is not written for that.

Bookmark the AI glossary. Follow a link into it when a term stops you mid-article. Then go back to what you were reading.

The terms cluster around one theme worth noticing. Eight of the ten exist because memory and bandwidth, not raw compute, now set the limits on what AI systems can do affordably. HBM, LPDDR, KV cache, quantization, MoE, distillation, and context window are all answers to that same constraint.

Understand that pattern and most infrastructure news stops feeling like jargon.

FAQ About This AI Glossary

What is the difference between HBM and LPDDR?

Both are DRAM, but they optimize differently. HBM stacks memory dies vertically beside the processor for extremely high bandwidth, at high cost and power. LPDDR targets low power and cheaper capacity, with much lower bandwidth. Data centre accelerators use HBM; phones, laptops, and edge devices use LPDDR.

Why does the KV cache matter more than model size?

Model weights are a fixed cost, loaded once at startup. This AI glossary flags the difference deliberately. The KV cache grows with every token in the conversation and with every concurrent request. At long context lengths it frequently exceeds the weights in memory use, which makes it the practical limit on how many users a server can handle at once.

Is RAG better than fine-tuning?

They solve different problems, which is why the AI glossary lists them apart. RAG supplies facts the model did not memorize and updates instantly when documents change. Fine-tuning changes behavior, format, and tone. Most production systems use RAG for knowledge and light fine-tuning for style, rather than choosing one.

What does MCP actually do?

MCP standardizes how an AI application talks to external tools and data sources. Instead of writing custom integration code for every service, a developer runs or connects to an MCP server that exposes tools, resources, and prompts through one interface. The July 2026 revision made it stateless so it scales on ordinary web infrastructure.

Does quantization hurt model quality?

Some, but less than most people expect. Dropping from 16-bit to 8-bit is usually near-lossless for large models. Four-bit shows measurable degradation on reasoning-heavy tasks, though modern methods narrow the gap considerably. The right question is whether the accuracy you lose costs more than the throughput you gain.

Why do MoE models need so much memory?

Because every expert must be loaded even though only a few run per token. A model with 671 billion total parameters and 37 billion active still needs all 671 billion resident somewhere. MoE saves compute, not memory, which is a distinction this AI glossary flags deliberately.

How often is this AI glossary updated?

Check the date and version of a linked source before relying on a changing specification. A general page update does not establish a separate review date for every definition.

Can I cite or link to a single AI glossary entry?

Yes. Every AI term has a permanent anchor, so you can point at one definition rather than the whole page. The markup uses schema.org DefinedTerm inside a DefinedTermSet, which lets other tools reference entries individually.

Keep reading

LLM context window

LLM Context Window Comparison 2026: Limits and Costs

Specifications and prices checked October 11, 2026 · Prices in USD per million tokens, standard API tier Short answer: In an LLM context window comparison …
LLM inference latency

LLM Inference Latency by Model and Provider: Real Numbers

Published October 2026 · Data read October 11, 2026 · Source: Artificial Analysis provider benchmarks, secondary research The same open-weight model, Llama 3.3 70B, starts …
GPU cloud pricing

GPU Cloud Pricing 2026: What an Hour Actually Costs

Published: October 2026 · Pricing last verified: October 10, 2026. Provider pricing and availability may change; check the linked source before committing to a workload. …
AI API rate limits

AI API Rate Limits in 2026: The Numbers That Bind

Two teams call the same model on the same tier. One runs a support chatbot and never sees a 429. The other runs a five-step …