GLM-5.2 beat Claude Fable 5 on one leaderboard. Here's what the rest of the data says.
Open weights now sit within 1-3 points of the frontier on individual coding benchmarks at 6-30x lower cost. On composites, private benchmarks, and long-horizon agentic work, the frontier lead is intact.
In June, GLM-5.2 became the first open-weight model to take #1 on Design Arena's coding leaderboard, passing Claude Fable 5 by about 10 Elo points with a score of 1360. The posts wrote themselves: open source has caught the frontier.
It's a real result. It's also one leaderboard. Look across the full benchmark landscape and the picture is messier. Open models have closed most of the gap on some measurements, none of it on others, and the measurements themselves are part of the story.
This piece walks through the benchmarks one by one: where open weights match the frontier, where they don't, and which numbers you shouldn't trust from either side.
What GLM-5.2 actually did
The numbers are strong, and they're not cherry-picked from one eval.
Design Arena is head-to-head human preference on coding output, not synthetic scoring. GLM-5.2 sits at 1360 Elo, first overall. It's also the first open-weight model past 80% on Terminal-Bench 2.1, with a reported 81.0 on the Terminus-2 harness and a best run of 82.7. On SWE-bench Pro it scores 62.1 against GPT-5.5's 58.6. On FrontierSWE, 74.4 against 72.6. On MCP-Atlas, 77.0 against 75.3. It leads every model in the comparison on AIME 2026 and IMOAnswerBench. All at roughly a sixth of GPT-5.5's per-token price.
The model itself is a 744B-parameter mixture-of-experts with 40B active parameters, a 1M-token context window, and an MIT license. You can download the weights and run it in your own VPC, assuming you have cluster-class hardware or a managed inference provider.
One caveat before the applause: most of these scores are vendor-reported. That doesn't make them wrong. It makes them a starting point.
The generational jump matters as much as the absolute scores. On DeepSWE, one of the hardest software engineering evals, GLM-5.2 scores 46.2 against GLM-5.1's 18. That's a better-than-150% improvement in a single release cycle. GLM-5.1 shipped in April. GLM-5.2 shipped in June. The open labs aren't just closing gaps, they're iterating on two-month cycles while frontier labs ship twice a year.
The field is five deep
GLM isn't a lone outlier. Five independent open-weight families reached near-frontier scores within months of each other in 2026, and each occupies a different point on the price-capability curve.
DeepSeek V4 ships in two variants: Pro (1.6T total parameters, 49B active) and Flash (284B total, 13B active), both MIT-licensed with 1M-token context. V4-Pro hits 80.6% on SWE-bench Verified and leads all open models on LiveCodeBench at 93.5%. Flash is the price shock: $0.28 per million output tokens on the official API.
Kimi K2.6 from Moonshot AI is a 1T-parameter MoE (32B active) with 256K context under a modified MIT license. It scores 80.2% on SWE-bench Verified and 54.0% on Humanity's Last Exam with tools, which leads every model in DeepInfra's comparison, closed models included. Moonshot's model card claims it scales to 300 sub-agents and has run 4,000 coordinated tool calls in a single 13-hour session. Treat autonomous-execution claims as marketing until you've reproduced them.
Qwen3-Coder-Next is the practical local option: 80B total parameters, 3B active, Apache 2.0. It scores 70.6% on SWE-bench Verified, well behind the giants. That 10-point gap is the honest price of a model you can actually self-host without a GPU cluster.
Mistral's open lineup covers Small 4 (119B/6.5B active), Large 3 (675B/41B active, Apache 2.0), and Medium 3.5. Mistral says an NVFP4 checkpoint of Large 3 runs on a single 8xH100 node, which makes it the largest open model with a realistic single-node deployment story.
Four of the five families come from Chinese labs. Whatever you make of that geopolitically, it means the open frontier no longer depends on any single company's release schedule.
Benchmark by benchmark
Here's the current state across the four most-cited coding and reasoning benchmarks, compiled from DeepInfra, Kingy AI, and vendor model cards. Scores marked with an asterisk are vendor-reported rather than independently run.
MODEL SWE-B VERIFIED SWE-B PRO TERMINAL-BENCH GPQA DIAMOND
Claude Opus 4.7 82.0 64.3 - 90.2
GPT-5.5 ~82 58.6 - 93.2
GLM-5.2 - 62.1* 81.0* (TB 2.1) 91.2*
DeepSeek V4 Pro 80.6 55.4 67.9 (TB 2.0) 90.1
Kimi K2.6 80.2 58.6 66.7 (TB 2.0) 90.5
Qwen3-Coder-Next 70.6 42.7 36.2 -
SWE-bench Verified is where the gap has nearly vanished. DeepSeek V4-Pro sits 1.4 points behind Claude Opus 4.7 on the standard agentic coding benchmark. A year ago the best open model was more than 10 points back.
SWE-bench Pro tells a different story. It covers 1,865 problems across 41 production-grade, multi-language repositories, and Opus 4.7 leads at 64.3%. GLM-5.2's vendor-reported 62.1 is the only open score within 3 points. Kimi K2.6 ties GPT-5.5 at 58.6, which is its own kind of headline, but the frontier lead on the hardest real-world coding work is intact.
Terminal-Bench is a mess of versions and harnesses, which we'll get to. Read the version tags before comparing anything in that column.
GPQA Diamond, graduate-level science reasoning, shows open models within 1-3 points of GPT-5.5. At this level of difficulty a 2-3 point gap is real capability, but it's no longer a class difference.
Then there are the composites. On Artificial Analysis' Intelligence Index v4.1, which aggregates ten independently run evals, GLM-5.2 is the top open-weight model at 51, about 5 points behind Claude Fable 5. On the May snapshot of v4.0, GPT-5.5 scored ~60 and Claude Opus 4.7 scored 57 against Kimi K2.6's 54. A broad index is harder to game than a single leaderboard because you'd have to optimize across ten evaluation types at once. The frontier lead survives every composite measure.
Agentic benchmarks split the same way. GLM-5.2 nearly ties Claude on MCP-Atlas (77.0 vs 75.3 over GPT-5.5). On Tool-Decathlon, Claude holds a clear lead. One head-to-head on a workflow orchestration task scored Opus 91/100 against Kimi K2.6's 68/100, with the gap concentrated in lease handling, cross-run scheduling, and live streaming. None of that was on a public benchmark, which is the point.
Scaffolding cuts both ways, though. On Humanity's Last Exam without tools, GLM-5.2 trails the strongest reasoning models. Enable tool calling and it jumps past GPT-5.5. The same weights, a different harness, a different ranking. Which model 'wins' depends on how you let it work, and that's as true in your production stack as it is on a leaderboard.
The pattern is consistent: the longer the horizon and the higher the stakes, the more the frontier lead reappears.
Private benchmarks say the gap is bigger than you think
The cleanest data on this question comes from a May 2026 analysis by Havard Tveit Ihle covering 17 benchmarks and about 110 datapoints, built largely on Epoch AI's benchmarking hub. The finding: on public benchmarks, open models trail the closed frontier by 4-6 months. On private benchmarks, where the test data isn't available to train on, the gap is 8-10 months. Nearly double.
Two things follow from that. First, open-weight developers are training closer to the public tests, whether deliberately or through contaminated training data. Second, the gap was smallest around DeepSeek R1 in January 2025 and has grown since. The headlines say the gap is closing. The private-benchmark trend says it's been widening for over a year.
Ihle also speculates the real-world gap is larger still. Closed labs have more enterprise usage data and less incentive to hill-climb on public leaderboards, so their models are likely better on tasks nobody benchmarks.
He flags his own caveat too: private benchmarks often test open models through third-party inference providers, and subtle serving bugs can degrade scores, which would inflate the measured gap. The data isn't clean in either direction. That's exactly the point.
The scores depend on the harness
Terminal-Bench results swing by up to 10 points depending on which agent harness runs the eval. Moonshot's published 66.7% for Kimi K2.6 uses the Terminus-2 harness; other configurations report scores 10 points higher for comparable models. GLM-5.2's Terminal-Bench result flips the same way: on one harness it trails the best frontier models by a few points, on another it beats Claude Opus 4.8.
Verbosity distorts cost claims. DeepSeek V4-Pro generated 190 million output tokens running the Artificial Analysis index, four times the median for comparable models. At list price, that pushes its effective per-task saving against Opus from the headline 7x down to roughly 1.75x. Still a saving. Not the headline.
Hallucination behavior differs too. On AA-Omniscience, which measures whether a model admits it doesn't know, V4-Pro scores poorly: when it doesn't know, it answers anyway. That never shows up in a SWE-bench score and it will absolutely show up in your product.
If a comparison doesn't state the benchmark version, the harness, and the token counts, treat it as marketing.
The part that isn't contested: price
Official list prices per million tokens, as of June 2026:
MODEL INPUT $/M OUTPUT $/M
GPT-5.5 $5.00 $30.00
Claude Opus 4.7 $5.00 $25.00
GLM-5.2 (Z.ai) $1.40 $4.40
Kimi K2.6 (Moonshot) $0.95 $4.00
Mistral Large 3 $0.50 $1.50
DeepSeek V4 Pro $0.44 $0.87
Qwen3-Coder-Next (OpenRouter) $0.11 $0.80
DeepSeek V4 Flash $0.14 $0.28
Sources: Z.ai, Moonshot, DeepSeek, and provider model cards compiled by Kingy AI. Prices move often; check before you model anything on them.
At 50 million output tokens a month, a realistic volume for a team running coding agents daily, that's $1,250 on Opus 4.7 or $1,500 on GPT-5.5, against $220 on GLM-5.2, $200 on Kimi K2.6, and $14 on DeepSeek V4 Flash. At 100 million tokens the annual difference against Opus alone exceeds $12,000, before the verbosity caveat above.
Cached input pricing widens the spread further. DeepSeek charges $0.0028 per million tokens on cache hits, Moonshot $0.16, Z.ai $0.26. For agent loops that replay the same repository context hundreds of times, cache pricing can matter more than the headline rate.
Open weights also buy things no price sheet captures: fine-tuning on your own data, deployment inside your VPC or air-gapped, provider portability, and no vendor deprecating a model out from under your production pipeline.
Self-hosting isn't free, though. It moves cost from token bills to GPUs, engineering time, and utilization risk. A poorly utilized cluster costs more than an API. GLM-5.2 at 744B parameters and DeepSeek V4 Pro at 1.6T are cluster-class deployments; of this group, only Qwen3-Coder-Next is realistic on hardware a normal team owns.
So who's right?
Both stories, at the same time.
On individual coding benchmarks, open models now sit within 1-3 points of the frontier at 6-30x lower output cost. That's real, and a year ago it would have sounded like fiction.
On composite indexes, private benchmarks, and long-horizon agentic work, frontier models hold a measurable lead. By the cleanest available data, that lead has been growing since early 2025, not shrinking.
The practical answer isn't picking a side. It's routing: open weights for the bulk of your workload - draft patches, test generation, summarization, retries - and frontier models for the fraction of requests that need the extra margin: architecture decisions, ambiguous debugging, high-stakes refactors. Teams that route by task type report the biggest cost improvements without a quality cliff.
Where's the frontier premium still defensible? Three places, on current data. Reasoning breadth: no open model matches GPT-5.5 or Fable 5 across a ten-eval composite. Hard repository work: when a wrong patch breaks 40 downstream callers, Opus 4.7's 6-point SWE-bench Pro margin over most open models is cheap insurance. And agentic reliability on the tasks nobody benchmarks: concurrent tool calls, state management across long sessions, error recovery. Benchmarks measure what labs choose to test. Production breaks in the parts that weren't on the test.
And before you trust any number in this article, including ours: pick 20 real tasks from your own repos, run them through two or three candidates, and measure pass rate, diff quality, token usage, and review burden. Public benchmarks are a signal. Your workload is the test.
Full disclosure: Kimchi.dev runs managed inference for open-weight models, so we have a position here. The numbers don't care about our position. Run your own eval before you believe anyone's leaderboard, including ours.