# Kimi K3 vs Fable 5 Benchmark

> Canonical: https://www.yalc.ai/gtm-ai-agents/kimi-k3-vs-fable-5-benchmark/

One command runs both models on the Claude Code skills you actually use, blind judges every output, and tells you when the cheap model is good enough.

## Categories

- intelligence

**Repo:** https://github.com/Othmane-Khadri/kimi-k3-vs-fable-5-benchmark

**Github Skill Url:** https://github.com/Othmane-Khadri/kimi-k3-vs-fable-5-benchmark

**Booker Url:** https://app.iclosed.io/e/acquisition-and-consulting-llp/yalc-implementation-call

**Yalc Fit Score:** 8

**Yalc Verdict:** The only model comparison that runs on your own work instead of a leaderboard. Fail loud runs, a blind randomized judge, and an always labeled cost estimate keep it honest.

**Hook:** Kimi K3 vs Fable 5. Scored on your agent, not a leaderboard.

**Before State:** Everyone argues Kimi vs Claude off a leaderboard. A leaderboard never runs your agent, your skills, your output formats, or your reliability bar. So the model choice ends up a guess, and the spend ends up a habit nobody audits.

**After State:** One command runs Fable 5 and Kimi K3 on the Claude Code skills you already use, same prompt, each model through its own CLI. A blind Opus judge scores every pair, cost and latency get logged on every task, and you get a one page verdict on where the cheap model is genuinely good enough.

## Gates

- **Wire Kimi into Claude Code**
  - N: 1
  - Body: SETUP.md gives you the three ways in. Run the kimi CLI next to Claude Code, point Claude Code itself at Kimi with the endpoint and model swap, or add one note to your CLAUDE.md so Claude Code can delegate a subtask with kimi -p. The kit uses the delegation path, so your existing logins are enough.
  - Tools: Kimi CLI,Claude Code,Cursor
- **It reads the skills you already run**
  - N: 2
  - Body: The runner scans ./.claude/skills, turns each skill into a self contained benchmark task, and asks you which ones to test. Empty folder? A four task starter suite covering coding, extraction, tool use, and long context runs instead.
  - Tools: Claude Code
- **Both models, same task, headless**
  - N: 3
  - Body: Each task runs through claude -p on claude-fable-5 and kimi -p on Kimi K3. Latency is measured on both sides, Fable cost is read from the CLI, and Kimi cost is estimated and always labeled as an estimate. A call that returns nothing fails loud instead of being scored as a loss.
  - Tools: Fable 5,Kimi K3
- **A blind judge scores every pair**
  - N: 4
  - Body: Opus judges both answers with the order randomized so it cannot favor a side, 1 to 10 with a written reason. Errored tasks are excluded from the win and cost math, never counted as losses.
  - Tools: Opus 4.8
- **One page verdict**
  - N: 5
  - Body: results.html shows the winner per task, quality scores, latency, cost with the Kimi estimate labeled, and a headline verdict. A sample report from a real run ships in the repo.
  - Tools: 

**Result Stat:** 11 tasks in the first real run. Correctness parity on every objective check. Kimi won the lean structured tasks at about a third of the price. Fable won all writing and research. Kimi ran 2 to 5x slower.

## Closer

The verdict from the first run. Kimi K3 for high volume correct answer work you clean up after. Fable 5 for the polished stuff you cannot babysit.

One trap inside. kimi -p quietly prepends a bullet that breaks strict JSON. The kit strips it and the setup guide explains it.

**Reviewer:** Othmane Khadri