Same model, different harness: why Kimchi scores 72.6 on Terminal-Bench 2.1

If you compare AI coding tools like most of the people out there, you're usually comparing the wrong thing.

You pick Claude Code with Opus 5 and put it up against Cursor with GPT-5.5. Different model, different harness, different everything. Then you draw conclusions about which "tool" is better. The model did most of the work (or most of the failing), and the harness gets credit it didn't earn.

We wanted to know what happens when you hold the model constant and swap the harness. So we ran GLM-5.2 through four coding agents on Terminal-Bench 2.1. Same model. Same 89 tasks. Same benchmark. The only variable was the harness sitting between the model and the terminal.

The spread was 13 points. That's the harness tax.

The results

Same model in every row. GLM-5.2, single-model, no multi-model routing. Inference ran through Moonshot's API. The harness is the only thing that changed.

The gap tells a clear story. Against Claude Code and Opencode, the difference is large: 7.2 and 12.8 points respectively. Against Cursor, it's narrower - 3.6 points. Cursor's composer models are built to extract more from the model they're paired with, and it shows. But Kimchi still edges it out with the same model, on the same tasks, with the same scoring.

The 13-point spread between Kimchi and Opencode isn't GLM-5.2 being smarter in Kimchi. It's the harness doing its job.

What Terminal-Bench 2.1 actually tests

Terminal-Bench 2.1 is the verified refresh of the original Terminal-Bench. Same 89 curated tasks. Each task drops the agent into a fresh terminal environment - a Docker container with its own dependencies, file structure, and tooling. The agent has to explore the environment, figure out what's there, complete the task specification, and validate its own work.

The 89 tasks span five domains: software engineering (build systems, package management, code fixes), system administration (server config, network setup), data processing (ETL, transformation pipelines), model training (ML pipeline construction), and security (vulnerability analysis, hardening).

These aren't toy problems. Each task has a human-written solution and a comprehensive test suite. The agent doesn't get partial credit for trying. It either passes the tests or it doesn't.

The methodology is straightforward: run each task 5 times, score the average pass rate. That's it. No subjective grading, no human judges. Pass or fail, averaged across trials to account for variance in model behavior.

Why the harness matters more than you think

Here's what most people miss about AI coding agents. The model is the engine, but the harness is the transmission, the steering, and the brakes. A great model in a sloppy harness will crash into the same wall repeatedly. A decent model in a well-tuned harness will find its way around it.

This is why our results matter. We didn't use a better model than the other harnesses. We used the same one.

What Kimchi's harness does differently

Kimchi's harness is built and tuned against the behavior of today's top-tier open-source models - Kimi, MiniMax, GLM, and their peers - not against a single frontier API.

Runtime sanitation of the message stream. Open-weight models have known output quirks. They emit phantom tool calls. They leave dangling tool results after a history rewrite. A naive harness feeds those errors back to the model until they pollute its context and derail the run. Kimchi scrubs them before the model ever sees them, so switching between models - even mid-session - just works.

Behavioral guardrails. Instead of only counting tokens, the harness watches what the model is actually doing. Specific guards cover exploration, the bash tool, and the review-write phase. When the behavior itself is wrong - stale edits, aimless exploration, repeated command patterns - the harness intervenes before those errors compound.

Loop guard. When a model drifts into repetition - the classic open-weight failure mode of edit, run, fail, edit the same thing again - a dedicated loop detector recognizes the pattern by comparing what the model keeps attempting, not just what it says. It interrupts with a directive to stop, name what's failing, consider alternatives it hasn't tried, and change approach. If the model keeps looping anyway, the harness ends the turn instead of letting it burn tokens going in circles.

Circuit breaker. If the provider itself is degraded, a process-wide circuit breaker recognizes retried calls are failing on infrastructure, not on the model, and terminates the run cleanly rather than retrying forever.

Defensive context management. Kimchi compacts with a generous token reserve below the model's window. Compaction happens early, before the model starts degrading near its limits - which matters more for open-weight models whose effective working context is shorter than the advertised number. Just as important is what survives compaction: the harness continuously prunes the history of artifacts that serve no purpose for the model - stale nudges, UI-only messages, malformed tool-call pairs - so what the model actually pays attention to stays clean.

Any model runs through the same harness. We don't optimize for any single vendor. Every part of the harness is tuned against the best open-weight models on the market - GLM, Kimi, DeepSeek, and whatever comes next.

For GLM-5.2 specifically, we resolved an issue with unbound thinking that customers would encounter when using this model with other harnesses. Nothing else beyond that - the score comes from the harness, not from GLM-specific tricks.

What this doesn't show

Let me be clear about what this benchmark doesn't prove.

It's not a model comparison. We ran GLM-5.2 across all harnesses. We didn't test whether GLM-5.2 is better or worse than Claude Opus 5 or GPT-5.6 on Terminal-Bench. That's a different article for a different purpose.

No multi-model routing. Kimchi supports multi-model orchestration - an orchestrator model delegates to specialized subagents. We didn't use it here. Every run was single-model, straight GLM-5.2. Multi-model routing would likely change the score, but we wanted to isolate the harness variable. That comes in a future benchmark once our new inference router ships.

Opencode results are from different runs. The Opencode numbers are real results, but from different time periods. The version and inference stability differ from the other harnesses. Take that 59.8% as directional, not as a controlled comparison.

The takeaway

If you're evaluating AI coding tools and you're looking at open-weight models, the harness matters more than the marketing tells you. The same model can score 72.6% or 59.8% depending on what sits between it and the terminal.

Kimchi's harness was built for open-weight models. That's not a side project. It's the core product. The 72.6% on Terminal-Bench 2.1, with the same model that scores 59.8% in another harness, is what that focus buys you.

If you do consider open source models, go for Kimchi.