Kimi K3 vs Fable 5 Benchmark | Yalc GTM AI agent

Free trial

Plug in the platform

Create your the platform account

Spin up the platform this agent runs on, then clone the agent and point it at your account.

Claim your trial

Watch it run

Walkthrough coming soon

A full video walkthrough of Kimi K3 vs Fable 5 Benchmark, start to finish, is on the way.

Coming soon

Clone the agent

The agent recipe on GitHub

The Claude Code agent, wired and documented. Read the recipe, fork it, and run it in your own session today. No black box, every gate is in the open.

View the agent on GitHub

One command runs both models on the Claude Code skills you actually use, blind judges every output, and tells you when the cheap model is good enough.

See Yalc live in action Clone the agent Every gate in markdown. Runs in your own Claude Code session.

Before and after

What changed

Before

Everyone argues Kimi vs Claude off a leaderboard. A leaderboard never runs your agent, your skills, your output formats, or your reliability bar. So the model choice ends up a guess, and the spend ends up a habit nobody audits.

After

One command runs Fable 5 and Kimi K3 on the Claude Code skills you already use, same prompt, each model through its own CLI. A blind judge panel scores every pair, cost and latency get logged on every task, and you get a one page verdict on where the cheap model is genuinely good enough.

How it works

The breakdown, gate by gate

1

Wire Kimi into Claude Code

SETUP.md gives you the three ways in. Run the kimi CLI next to Claude Code, point Claude Code itself at Kimi with the endpoint and model swap, or add one note to your CLAUDE.md so Claude Code can delegate a subtask with kimi -p. The kit uses the delegation path, so your existing logins are enough.

Kimi CLIClaude CodeCursor

2

It reads the skills you already run

The runner scans ./.claude/skills, turns each skill into a self contained benchmark task, and asks you which ones to test. Empty folder? A four task starter suite covering coding, extraction, tool use, and long context runs instead.

Claude Code

3

Both models, same task, headless

Each task runs through claude -p on claude-fable-5 and kimi -p on Kimi K3. Latency is measured on both sides, Fable cost is read from the CLI, and Kimi cost is estimated and always labeled as an estimate. A call that returns nothing fails loud instead of being scored as a loss.

Fable 5Kimi K3

4

Two blind judges score every pair

One judge from each model family, Opus and Kimi, and every pair is judged in both orders so position bias cannot decide it. A win only counts when the judges agree; disagreements become ties. Errored trials count as failures, with a separate error rate so flakiness stays visible. Run with trials and reliability is reported as pass^k.

Opus 4.8Kimi K3

5

One page verdict, plus a routing policy

results.html shows the winner per task, quality scores, latency, cost with the Kimi estimate labeled, and a headline that prints a tie when the difference is not statistically meaningful. The run also writes model-routing.json plus a paste ready CLAUDE.md snippet, so your agents route each task type to the right model. A sample report ships in the repo.

Results

What goes in, what comes out

11 tasks, 3 trials each, judged by both Opus and Kimi in both orders. Correctness dead even on every objective check. The two judges agreed only 48 percent of the time. Fable a small real quality edge, no single task a clean winner.

The honest verdict. On the work that has a right answer, the cheap model is as good, at about a third of the price. Fable holds a small edge on customer facing polish. And the run's own routing policy routed nothing to Kimi, because the evidence was not strong enough to justify a switch.

The real lesson is about benchmarks. Judge once, with one model, in one order, and half your verdicts are noise. This one judges twice, with two families, across both orders, and admits a tie when it cannot call a winner.

Kimi K3 vs Fable 5. Scored on your agent, not a leaderboard.