We ran DeepSeek Harness against Kimchi on the same model. It lost by 22 points
Same model, same benchmark, same conditions. Kimchi passed 69.4% of Terminal-Bench tasks. DeepSeek Harness's best preset passed 47.1%.
DeepSeek Harness (dsh) came out on August 13, 2026. Seven weeks later it has 239,003 GitHub stars and 28.7k forks.
So naturally we had to run it against Kimchi.
What we ran
dsh is DeepSeek's open-source agent runtime. The model adapter, tool registry, session log, and the agent loop itself are all plugins. It ships four presets - named compositions of tools, prompt, and persona, picked per session. We tested three: minimal (two tools and a one-sentence system prompt), standard (the full agent with all tools, planning, delegation, and web access), and creator (standard plus tools that can modify the runtime it runs on).
The setup:
- Same model in both harnesses: GLM 5.3 flash
- Terminal-Bench, official methodology: five trials per preset, average pass rate
- All runs at maximum reasoning
Why GLM 5.3 Flash and not a DeepSeek model? That's a great question, and there are two reasons. First, this is a harness comparison. The model has to be identical on both sides, or you're measuring two things at once.
Second, there are public claims that DeepSeek models post inflated Terminal-Bench scores, and we didn't want the model choice settling the fight before it started.
The numbers
| Run | Score | Cost | Cost per point |
|---|---|---|---|
| Kimchi | 69.4% | $36.03 | $0.52 |
| dsh creator preset | 47.1% | $27.50 | $0.58 |
| dsh minimal preset | 46.9% | $26.60 | $0.57 |
| dsh standard preset | 44.0% | $25.80 | $0.59 |
Kimchi passed 69.4% of tasks at $36.03. The best dsh preset passed 47.1%. That's a 22.3-point gap on the same model.
Then there's the spread inside dsh itself. Same model, same benchmark, three presets: 44.0% to 47.1%. The composition alone moved the score by three points. A benchmark number without the harness named is half a number.
The interesting one is minimal. Two tools, no compaction, and a system prompt that is one sentence: "You are a helpful software engineer assistant." It scored 46.9% - three points higher than standard, the full agent with every tool enabled.
We ran a similar comparison in August - Cursor, Claude Code, Opencode, and Kimchi on the same model - and the spread between harnesses was 13 points. The pattern holds: the harness is part of what's being measured.
Methodology
Terminal-Bench, official methodology, five trials per dsh preset (minimal, standard, creator), average pass rate, all runs with GLM 5.3 flash at maximum reasoning, in October 2026. Same conditions for the Kimchi run. Model choice based on inference resilience in our prior testing.
Try Kimchi free at app.kimchi.dev, or read our previous harness comparison: Same Model, Different Results: What Actually Separates AI Coding Agents.