Two labs. Two completely different models. Nearly identical scores on MMLU, ARC, and HellaSwag. That happened this month, and composite benchmarks are the reason nobody can explain it cleanly.

Kimi K3 and Qwen 3.7 landed within weeks of each other. Both claimed frontier-class performance. Both posted numbers so tightly clustered that the gaps disappear into statistical noise.

Yet anyone who has spent a working week with both models can tell you they behave nothing alike. That gap — between what composite benchmarks report and what the model actually does — is the story here.

Composite benchmarks are no longer failing quietly. They are failing in public, with two of the year’s biggest releases standing on top of them.

Key Takeaways: Composite Benchmarks at a Glance

• Composite benchmarks have saturated at the top of the curve, where every frontier model now clusters between 87% and 95%.
• Kimi K3 and Qwen 3.7 differ by 0.5% on MMLU, 0.4% on ARC-Challenge, and 0.6% on HellaSwag — all inside the noise floor.
• Meaningful separation appears only on harder, less-saturated tests: 3.2% on HumanEval and 4.7% on MATH.
• Training data contamination means gains on composite benchmarks may reflect memorization rather than reasoning.
• Enterprises are replacing composite benchmarks with domain-specific eval suites, live A/B tests, and regression testing on production prompts.

Why Composite Benchmarks Stopped Working Before Kimi K3 Launched

The failure did not start this month. Composite benchmarks began losing resolution the moment every serious model cleared 85% on MMLU.

Once GPT-4, Claude 3.5, and Gemini 1.5 all landed within a few points of one another, composite benchmarks quietly stopped doing their job. They still produced numbers. Those numbers just stopped meaning anything at the top of the distribution.

Three ways composite benchmarks break at the ceiling

Ceiling compression. As models approach maximum scores, the questions left standing are the ambiguous ones, the badly written ones, and the absurdly niche ones. Improvement on that residue is not capability. It is noise wearing a percentage sign.

Training contamination. Benchmark datasets leak into training corpora. The model does not learn the skill — it recalls the answer. Stanford’s HELM project has documented this pattern across composite benchmarks at length, and the findings make uncomfortable reading.

Aggregate masking. A single score averages dozens of subtasks. A model can be genuinely poor at multi-step logic and excellent at trivia recall, and composite benchmarks hide both facts with equal efficiency.

Contamination deserves a concrete example. MMLU questions have circulated in public GitHub repositories, Reddit threads, and fine-tuning datasets for years. Any model trained on a large web crawl has almost certainly seen many of them verbatim.

That is not cheating in any deliberate sense. It is an unavoidable side effect of scraping the open internet. But it means a jump from 86% to 88% on composite benchmarks might reflect better recall of previously seen items rather than sharper reasoning.

There is no clean way to tell from the outside. That opacity is the part that should worry buyers most.

By the time Kimi K3 arrived with strong MMLU numbers, composite benchmarks were already a broken thermometer. They could confirm the room was hot. They could not tell you whether it was 95°F or 115°F.

Qwen 3.7 from Alibaba’s research team posted similar figures, and the overlap was not coincidence. Both teams optimized against the same saturated composite benchmarks and got the same saturated results.

None of this is really about Kimi K3. It is about what happens when an entire industry leans on composite benchmarks that can no longer separate winners from losers.

The Kimi K3 and Qwen 3.7 Scores That Broke Composite Benchmarks

Numbers carry this argument better than adjectives do.

On MMLU (5-shot), Kimi K3 claims 88.4% against Qwen 3.7’s 87.9% — a delta of 0.5%. On ARC-Challenge, it is 94.1% versus 93.7%, a gap of 0.4%. On HellaSwag, 95.8% versus 95.2%, or 0.6%.

Every one of those deltas would flip with a different random seed or a reformatted prompt. They are not results. They are rounding, and composite benchmarks present them with the same confidence they once reserved for real progress.

The picture changes as the tests get harder. On HumanEval, Kimi K3 posts 82.3% against Qwen 3.7’s 79.1% — a 3.2% gap that might matter depending on your task.

On MATH, the competition-level set, the spread widens to 76.5% versus 71.8%. That 4.7% difference is real signal you can build a decision around, and it appears precisely where composite benchmarks have not yet saturated.

What composite benchmarks miss inside a 4.7% gap

Put that MATH gap in practical terms. If you are shipping a tutoring product that walks students through competition-style problems, a 4.7% accuracy difference compounds across thousands of sessions.

A student stuck on a hard combinatorics problem does not care about MMLU scores. They care whether the explanation is correct. That is exactly the kind of domain-specific signal composite benchmarks bury under an average.

Both model teams appear to understand this. Their technical reports increasingly foreground domain-specific results and treat composite benchmarks as table stakes — which tells you how much confidence the builders themselves place in the headline numbers.

Chatbot Arena from LMSYS remains one of the few evaluation platforms capturing genuine preference data. It is messy, subjective, and slow to converge. It also works, because unlike composite benchmarks it measures what happens when real people ask real questions.

What Composite Benchmarks Hide About Real-World Capability

So if the scores are effectively tied, are the models effectively the same?

No. And the distance between those two statements is where most procurement mistakes live. Here are five differences composite benchmarks cannot see.

1. Instruction following slips past composite benchmarks

Kimi K3 and Qwen 3.7 handle multi-step instructions differently. One might execute a five-part request cleanly while the other silently drops step three every single time.

A useful test: write a prompt with six numbered constraints, including one that conflicts with a common default behavior. Watch which model flags the conflict and asks, and which one quietly picks a side. No composite benchmarks test this reliably.

2. Tone and style control that composite benchmarks ignore

Enterprise buyers care whether a model can hold a brand voice across ten thousand outputs. Composite benchmarks do not measure this at all.

A financial services firm needing formal, hedged language will get wildly different mileage from two models with identical HellaSwag scores.

3. Latency and throughput sit outside composite benchmarks

A model scoring 88% at 50 tokens per second is a different product from one scoring 87% at 200 tokens per second. Composite benchmarks ignore operational reality completely.

For a customer-facing chat application, the faster model may be the only viable option regardless of which one looks better on a leaderboard.

4. Failure modes that composite benchmarks score as equal

How a model fails matters as much as how often. Some hallucinate with total confidence. Others hedge appropriately.

Composite benchmarks treat every error as equivalent, and errors are not equivalent. A model that says “I’m not certain, but here’s my best understanding” is far safer in a clinical or legal setting than one delivering the same wrong answer as settled fact.

5. Context window use that composite benchmarks never test

Kimi K3 and Qwen 3.7 both advertise long context windows. How well each one actually uses that window varies enormously, and composite benchmarks say nothing about it.

The Needle in a Haystack test exposed this months ago across the field. Some models claiming 128k windows start dropping details reliably past 30k tokens — a failure that only surfaces when you test for it directly.

There is a second-order problem here. Because composite benchmarks cannot separate models, marketing fills the vacuum: cherry-picked demos, curated examples, hype cycles that collapse on contact with production traffic.

The Kimi K3 launch generated real excitement. Excitement is not evaluation. I have learned to wait a few days and form my own view before repeating anyone’s numbers.

Why Enterprises Are Abandoning Composite Benchmarks for Custom Evals

The sharpest teams stopped waiting. They are not hoping composite benchmarks get fixed — they are building replacements in-house, and the recent release cycle accelerated that shift noticeably.

Domain-specific suites that outperform composite benchmarks

A legal tech company builds 500 questions from real case law. A healthcare startup writes tests from actual clinical notes. These suites measure what matters for one use case instead of gesturing at general intelligence.

One mid-sized e-commerce team built a 200-prompt eval entirely from their own support tickets — partial refund disputes, multi-item order changes, the messy edges. That suite caught a critical failure in a candidate model that composite benchmarks gave no hint of.

A/B testing with real users

Rather than trusting published scores, companies run two models in parallel and measure satisfaction, task completion, and error frequency. It is messier than a leaderboard and considerably more honest.

The tradeoff is traffic volume. A low-volume internal tool might need weeks before results converge, which is a real cost worth planning for before you abandon composite benchmarks entirely.

Red-teaming protocols

Organizations like NIST are building adversarial testing frameworks that reach well past anything composite benchmarks capture. The rigor in their documentation is genuinely impressive once you dig in.

Regression testing instead of composite benchmarks

When a model updates, mature teams do not check whether the MMLU score moved. They check whether the 47 specific prompts their product depends on still behave correctly.

Specificity beats generality in production every time. Ship a version bump on the strength of composite benchmarks alone and you will learn this lesson the expensive way.

The economics are straightforward. A company choosing between Kimi K3 and Qwen 3.7 cannot justify that decision on a 0.5% MMLU delta. They need to know which model handles their data, their edge cases, and their users.

Evaluation-as-a-service is growing fast in response. Braintrust lets teams build custom pipelines without starting from zero, and it shines when you want to track eval results as prompts evolve.

Eleuther AI’s lm-evaluation-harness is the better pick when you want full control over evaluation logic and do not mind the configuration work. Both tools exist because composite benchmarks stopped answering the questions engineers actually have.

This migration is not theoretical. It is happening now, in real engineering teams, on real deadlines.

What Replaces Composite Benchmarks: The Emerging Evaluation Stack

If MMLU, ARC, and HellaSwag can no longer separate frontier models, what does? Not one thing. A stack of four layers that together do the job composite benchmarks once did alone.

Layer 1: Capability probes that replace composite benchmarks

Narrow, deep tests for specific skills. Can the model chain three inference steps? Stay consistent across 100,000 tokens? Apply a complex API spec correctly?

Each probe tests one thing properly instead of averaging everything badly, which is the structural flaw in composite benchmarks. A strong instruction-following probe might present a 15-step workflow with deliberate ambiguity at step 8 and measure whether the model flags it.

Layer 2: Human preference data

Chatbot Arena proved crowdsourced head-to-head comparison produces reliable signal. It captures the qualities composite benchmarks miss — helpfulness, clarity, appropriate confidence.

Those qualities are not soft. They are just hard to quantify. The catch is timing: Arena rankings take weeks to stabilize after a launch, which is painful when a procurement decision is due Friday.

Layer 3: Task-specific leaderboards

Instead of one ranking to rule them all, we are getting specialized boards: best for code, best for creative writing, best for structured extraction.

The fragmentation is a feature. It forces you to decide what you are optimizing for before you start comparing numbers, which composite benchmarks never required anyone to do.

Layer 4: Continuous monitoring

Production evaluation is not an event. Models drift, prompts change, user expectations move.

A workable implementation: log a random 1–2% sample of production outputs, run them through a lightweight automated scorer, and alert when quality drops beyond a set threshold week over week. Unglamorous, and it catches regressions no pre-launch test would have predicted.

None of this means composite benchmarks belong in the bin.

They remain a serviceable first filter. A model scoring 60% on MMLU is not frontier-class, and you learn that in ten seconds instead of ten days. Composite benchmarks also give the research community a shared vocabulary, which carries real coordination value.

The failure is specific and located at the top of the distribution, where scores compress and genuine differences vanish. Below that band, composite benchmarks still sort effectively.

Kimi K3 and Qwen 3.7 exposed the compression, but blaming the models gets it backwards. They are doing precisely what they should: getting better. The measuring tools need to catch up.

Conclusion: Composite Benchmarks Have Outgrown Their Job

Composite benchmarks worked beautifully while models climbed from 50% to 80%. They fail completely now that every serious contender sits between 87% and 95%.

When two models built on different continents by different teams score within fractions of a percent across three separate suites, those suites have stopped measuring progress. Composite benchmarks have become participation trophies for frontier models.

Five things worth doing with this:

• Stop choosing frontier models on MMLU deltas. A 0.5% difference is not actionable information.
• Build custom evals for your actual use case. Test what matters to your users, not what mattered to a paper from three years ago.
• Watch human preference rankings. Chatbot Arena is slow, but it is honest in ways composite benchmarks no longer are.
• Ask vendors for domain-specific results. If they cannot show performance on tasks resembling yours, their composite benchmarks are marketing collateral.
• Revisit your evaluation process quarterly. The measurement field is moving as fast as the models, arguably faster.

The story here is ultimately one of growth. The field outgrew its first set of rulers, and what replaces composite benchmarks will be messier, more fragmented, and considerably more useful.

FAQ About Composite Benchmarks and Kimi K3

The race to build commercially viable humanoid robots is burning through billions in venture capital, and the numbers have moved fast enough to catch most coverage flat-footed. Comparing Agility, Figure, Apptronik funding as of mid-2026 tells a genuinely different story than it would have told even a year ago.

As recently as early 2024, Figure AI was the funding leader at a $2.6 billion valuation, Agility was the steady operator, and Apptronik was the disciplined underdog running on a comparatively tiny budget. The humanoid robot funding picture no longer looks like that at all. Figure AI is now valued at $39 billion. Apptronik has raised nearly a billion dollars with Google as a new backer. Agility just announced it’s going public.

This piece breaks down where the funding actually stands today, what the investor rosters signal, and which company is best positioned to survive the capital-intensive slog toward real commercial scale.

Key Takeaways on Agility Figure Apptronik Funding

• Figure AI’s September 2025 Series C valued the company at $39 billion, roughly 15 times its 2024 valuation, on close to $1.9 billion raised total.
• Apptronik closed a roughly $935 million round in early 2026 at a $5–5.5 billion valuation, with Google joining as a backer alongside a new Google DeepMind partnership.
• Agility Robotics announced on June 24, 2026 that it’s going public via a SPAC merger with Churchill Capital Corp XI, at a valuation near $2.5 billion.
• Agility remains furthest along commercially, with roughly 100 Digit units deployed and low tens of millions in annual run-rate revenue.
• Apptronik is no longer the capital-constrained underdog in this story — it’s now the second-best-funded of the three.

How the Humanoid Robot Funding Numbers Stack Up in 2026

Each of these three companies has taken a genuinely distinct path through the funding, and the gaps between them have only widened.

Figure AI moved fastest and furthest in the Agility, Figure, Apptronik funding race. Founded in 2022, Figure raised a $675 million Series B in February 2024 at a $2.6 billion valuation, with Microsoft, OpenAI’s Startup Fund, Nvidia, and Jeff Bezos among the backers. That was just the beginning. In September 2025, Figure closed a Series C of over $1 billion, led by Parkway Venture Capital with Brookfield, Nvidia, Macquarie, and others participating, pushing its valuation to $39 billion. Total funding now sits around $1.9 billion.

Agility Robotics has been part of the funding story the longest. Founded in 2015, the company built its Digit robot over nearly a decade of iteration before raising a $150 million Series B in 2024. It followed that with a $400 million Series C in March 2025, bringing total funding to roughly $641 million. Then, on June 24, 2026, Agility announced a definitive agreement to go public through a SPAC merger with Churchill Capital Corp XI, at a valuation of roughly $2.5 billion.

Where Apptronik Fits Into Agility, Figure, Apptronik Funding Now

Apptronik’s part of the funding story has changed the most. Founded in 2016, the Austin-based company spent years on a comparatively modest budget after its original $14 million Series A. That changed in early 2026, when Apptronik closed a round of roughly $935 million at a $5 to $5.5 billion valuation, with Google joining as a backer alongside Mercedes-Benz, GXO Logistics, and Jabil. The round came alongside a new strategic partnership with Google DeepMind to build Apptronik’s next-generation robot on Gemini Robotics.

Here’s how the humanoid robot race funding compares as of mid-2026: Figure has raised roughly $1.9 billion total, most recently at a $39 billion valuation. Agility has raised roughly $641 million total and is now proceeding toward a public listing at around a $2.5 billion valuation. Apptronik has raised roughly $935 million total, most recently at a $5 to $5.5 billion valuation.

Figure AI still commands the highest valuation in the Agility, Figure, Apptronik funding race by a wide margin, which says something about how AI hype continues to reshape hardware investment logic. But the real shift is Apptronik’s leap from also-ran to genuine second-place contender on both funding and valuation.

Investor Composition Behind Agility, Figure, Apptronik Funding

Who writes the checks matters as much as how large they are, and the cap tables behind the funding tell three distinct stories.

Figure AI’s investor list reads like a tech all-star roster, and it’s the clearest signal in the Agility, Figure, Apptronik funding story of where AI-native capital is flowing. Microsoft, Nvidia, and OpenAI’s Startup Fund backed its 2024 round, and its 2025 Series C added Brookfield, Macquarie, Intel Capital, LG, Salesforce, and Qualcomm Ventures. That composition signals Figure is positioned at the intersection of AI infrastructure and robotics, not just another hardware play.

How Google’s Entry Changes Agility, Figure, Apptronik Funding

Apptronik’s investor base transformed with its 2026 round, reshaping the funding hierarchy. Google’s participation, paired with the Google DeepMind partnership for Gemini Robotics, gives Apptronik a strategic AI relationship that rivals Figure’s Microsoft and OpenAI ties. Mercedes-Benz, GXO Logistics, and Jabil round out a roster focused on manufacturing and logistics deployment rather than pure AI research.

Agility Robotics brings a different kind of backing to the Agility, Figure, Apptronik funding comparison. Amazon participated in Agility’s earlier rounds and has been testing Digit robots in its fulfillment centers, while Nvidia’s venture arm, NVentures, joined more recently. The SPAC route to going public also brings in an entirely new category of investor: public markets. That’s a structurally different path than either Figure or Apptronik has taken, and it comes with its own disclosure requirements and quarterly scrutiny that private companies don’t face.

Each company is essentially courting a different version of the future in the Agility, Figure, Apptronik funding story: Figure toward general-purpose AI-native robotics, Apptronik toward Google-backed industrial deployment, and Agility toward public-market validation of its commercial traction.

Capital Efficiency and Runway Across Agility, Figure, Apptronik Funding

Raising money is one thing. Spending it wisely is another, and this is where the Agility, Figure, Apptronik funding story runs into the harder question of capital efficiency.

Hardware companies burn cash faster than software startups, a fact every part of the humanoid robot funding equation has to account for. Building prototypes, standing up pilot production lines, and recruiting specialized robotics engineers all cost far more than spinning up cloud instances. A single experienced robotics engineer with a relevant PhD commands $250,000 to $350,000 in total annual compensation in the current market — multiply that across a 200-person technical team and the burn adds up before a single actuator gets machined.

Runway Looks Very Different Across the Funding Now

With roughly $1.9 billion raised, Figure carries the most substantial runway in the Agility, Figure, Apptronik funding comparison, even accounting for the sharp cost increases that come with scaling from pilot batches to full production runs. Agility’s $641 million in total funding, combined with its RoboFab factory in Salem, Oregon, gives it real manufacturing capacity already on the books, though going public will add new reporting obligations and investor scrutiny of burn rate specifically.

Apptronik’s roughly $935 million round fundamentally resets its runway picture from where it stood even a year ago. The company that once had to be extremely capital-efficient or risk falling behind now has a war chest closer to Agility’s than the thin budget the earlier funding numbers suggested.

That said, all three companies will likely need additional capital before reaching profitability. Humanoid robots remain among the most capital-intensive hardware categories in tech, and manufacturing scale-up costs tend to spike sharply once volume production begins.

Deployment Timelines: Agility, Figure, Apptronik Funding vs. Revenue

Billions in the humanoid robot race funding mean nothing without a path to revenue, and this is where the gaps between the three companies are still widest, even after the 2025 and 2026 funding shifts.

Agility remains furthest along commercially. Digit robots have been piloted inside Amazon warehouses since 2023, and as of early 2026, roughly 100 units were deployed generating low tens of millions in annual run-rate revenue at an estimated $30-per-hour robots-as-a-service rate. That’s still a small fraction of Agility’s roughly $641 million in cumulative capital raised, but it’s real, named-customer revenue that neither competitor has matched at the same scale.

Figure AI has strong partnerships — including with BMW for manufacturing deployment — and has replaced its earlier OpenAI collaboration with its own in-house model, called Helix, powering the robot’s reasoning and manipulation. On the deployment side of Agility, Figure, Apptronik funding, though, Figure’s commercial scale still trails Agility’s.

Apptronik’s Apollo robot has real partnerships with Mercedes-Benz, GXO Logistics, and Jabil, and its new Google DeepMind collaboration on Gemini Robotics is aimed at a next-generation robot debuting in 2026. Apptronik’s dramatically larger war chest, the newest shift in the Agility, Figure, Apptronik funding picture, now gives it room to scale deployment faster than its previous funding level would have allowed, though it’s still working from an earlier commercial base than Agility.

The pattern across the funding and revenue is consistent with the broader market: the global humanoid robotics market is projected to grow from roughly $3 billion in 2025 to as much as $38 billion by 2035, according to industry estimates. That’s the scale of bet investors across all three companies are making.

How Big Tech Capex Connects to Agility, Figure, Apptronik Funding

The humanoid robotics funding surge behind the funding doesn’t exist in a vacuum. It’s directly connected to broader Big Tech capital spending, and understanding that connection changes how you read these rounds.

Microsoft’s and Nvidia’s continued involvement in Figure AI signals a bet that humanoid robots become physical endpoints for AI models and simulation platforms — a thesis that now runs through most of the Agility, Figure, Apptronik funding story. Nvidia’s robotics logic is straightforward: every humanoid robot that trains in Isaac Sim and runs inference on Jetson hardware is a recurring chip customer, and that logic now extends to Agility through NVentures’ investment too.

Google’s entry into Apptronik’s round follows a similar logic to Microsoft’s Figure bet, just arriving later and larger in the Agility, Figure, Apptronik funding timeline. Pairing capital with the Gemini Robotics partnership means Google isn’t just writing a check — it’s betting that humanoid robots become a genuine deployment surface for its foundation models, the same way OpenAI positioned itself with Figure back in 2024.

Amazon’s relationship with Agility follows a different logic entirely within the Agility, Figure, Apptronik funding picture. Amazon already runs hundreds of thousands of robots across its warehouses, and Digit represents the next step: machines that can work in spaces built for humans without costly facility changes. Even capturing a small fraction of Amazon’s 1,000-plus global fulfillment facilities would represent thousands of units and meaningful recurring service revenue.

These connections show why Agility, Figure, Apptronik funding matters well beyond the robotics industry itself — each round is also a proxy for how seriously a different corner of Big Tech takes embodied AI as the next deployment layer.

Conclusion: What Agility, Figure, Apptronik Funding Reveals

The Agility, Figure, Apptronik funding picture in mid-2026 looks almost nothing like it did two years ago. Figure AI leads decisively on valuation at $39 billion. Apptronik has closed the funding gap dramatically with its roughly $935 million round and new Google partnership. Agility is taking the most structurally different path of the three by going public.

A few things worth watching next in the Agility, Figure, Apptronik funding story. Track Figure’s Helix model and BMW deployment numbers, since that’s the clearest signal of whether its valuation is backed by real commercial traction. Watch Apptronik’s Gemini Robotics-powered robot when it debuts, since that will show whether the Google partnership translates into a genuine product advantage. Follow Agility’s SPAC process closely, since a successful public listing would make it the first pure-play humanoid company with real public-market price discovery.

No single company has this locked up. Agility, Figure, Apptronik funding numbers will keep moving, and the capital allocation patterns forming right now will likely determine which company, if any, builds the first truly commercial humanoid robot at real scale.

FAQ About Agility, Figure, Apptronik Funding