Kimchi is Cast AI’s coding agent, in early access, and the name is a small joke that keeps paying off. Kimchi is the Korean staple of fermented cabbage — cabbage, salt, chili, garlic, and time; you pack it in a jar and let the transformation do the work. Cast AI named the tool after it, and named its autonomous mode after the process itself: Ferment. And I keep a kombucha brewing app — another jar doing the same slow work. The tool, its mode, and my app all circling one idea: too good to skip.

This is not a benchmark. It is one real feature, built twice on the same codebase — once in Kimchi’s chat mode, once in Ferment, its autonomous project mode. The same model wrote the code both times; what changed was the architecture around it, and that turned out to be the whole story. (Honest context: harness engineering is the work I most want to be doing, and the best way I know to understand a tool is to make it do something real.)

One note on method: while Kimchi worked, a second agent watched its session logs from the outside, pulling the numbers and quotes below. One agent building, another auditing — not by accident, the same reason Kimchi separates building work from grading it.

(I ran all of this against a throwaway staging copy, not the instance my family uses.)

The task

Kombucha needs starter liquid. Starter is brewed from its own recipe, seeded from starter you already have — so the thing that refills your stock also consumes it. I asked for a feature that tracks stock, plans brewing when it runs low, and warns in time: a recursive requirement, a nice trap. And my app has a second trap I left in place — its build command does not typecheck.

Run A: chat mode

Fast, obedient, a little shallow. It built a working starter module for about 2.8M tokens — but the module stood on its own, never connected to the actual brewing flow, so nothing was deducted when you brewed. Then the expensive part: the build passed with a missing import still in it (that second trap), the deployed page went blank, and the agent spent roughly 2M more tokens hunting a one-line fix. It never ran a typecheck. It trusted my repo’s own checks, and my repo was not honest with it.

Run B: Ferment

Ferment scopes before it codes. Its second question to me was whether finishing a starter batch should feed the stock back — the exact integration the single pass never modeled. It missed one thing (that a starter batch consumes starter too), and I fixed that with a single sentence at a prompt. That is what the checkpoints are for: the miss cost a line of text, not a shipped feature.

Then it did something the chat run never did. It wrote itself a small script to validate its database migration before running it. It ran the standalone typechecker on both the server and the frontend, unasked, on a repo whose own build skips that step. It walked straight past the trap that broke the first run. The first deploy worked. And the feature is deep: stock goes down when a batch starts, back up when a starter batch finishes, and the dashboard warns me a full brew cycle ahead.

The personas I did not believe in

I will admit upfront. The “cast of roles” idea — an agent with a Planner, a Builder, a Reviewer, an Explorer — always looked like theater to me: one model putting on different hats, talking to itself.

Watching Ferment work changed my mind — not all the way to the hype, but partway. The role separation did something real: something in it ran the typecheck the single pass skipped, and asked the integration question up front.

I still could not tell from the outside whether that was real separation, so I read how they built it. It is real. Kimchi runs each role on a different model: Kimi K2.7 plans, reviews, and judges; MiniMax M3 does the bulk building; a cheaper model explores — routed by cost and difficulty. Their own line is blunt: “you’re not grading a milestone with the same model that built it — the builder is MiniMax M3, the judge is Kimi K2.7.”

And that reframed both of my runs. Chat mode is the builder alone — one model, one pass. Ferment is the same builder wrapped in the orchestra: a planner and reviewer with different weights, an explorer, checkpoints between them. The reviewer that caught the typecheck was never in the room during Run A. So it is not one model wearing a different label — it is different weights with a different job, the reviewer-independence argument I had been making for myself (an evaluator with no stake in the work being done), shipped as a product. What keeps a reasoning-model reviewer honestly skeptical of a generation-model builder is the machinery that excites me most.

A small slip, and the bigger question behind it

One minor thing went wrong. Ferment writes its scope down first — goal, success criteria, constraints — and constraint number one was, in its own words, “work only on branch kimchi/starter-stock-ferment; never commit to main.” Then it worked on main instead. No harm came of it: it made no commits at all, and would most likely have moved to the branch before committing. A technical slip, not a real incident.

But it made me curious, so I asked: you wrote this rule for yourself, then worked around it — why? The answer was more interesting than the slip. It has an isolation step built for exactly this and never reached for it, and proposed four ways to prevent a repeat:

  • Invoke that isolation step at session start — a thing to remember.
  • Add the rule to its AGENTS.md — a note to read.
  • Print the branch each turn — a habit to keep.
  • Refuse to write on main, checked before every edit — a guard that does not depend on the model choosing to comply.

Read that list by layer. The first three are good intentions — things the agent has to remember. Only the fourth actually holds, because it does not ask the model to agree. Asked in good faith how to stop itself from breaking a rule, it reached for three intentions and one real guard, and set them down as equals.

That is not a criticism of Kimchi. It is the clearest example I have of how slippery this is. The urge to fix a broken rule by writing more words is so strong that even a system explaining its own mistake reaches for more words first.

Where “never again” actually lives

So the tempting fix is to write a firmer rule. But a rule in a file is the same kind of thing that just failed. I sort each failure into one of three layers instead:

  1. A deterministic check — a hook, a gate, a step in the pipeline. This is the only place “never again” is real, because it does not ask the model to agree. Branch protection. A typecheck that blocks “done” until it passes, no matter what the build script skips. Something that returns an error, not a paragraph that asks nicely.
  2. Notes the model genuinely lacks — the AGENTS.md layer. This helps, but only for knowledge, not for discipline. “This repo’s build does not typecheck.” “Starter is consumed and produced by the same flow — model both.” The agent honors these because it has no reason not to. They raise the floor. They do not hold a ceiling.
  3. Things no file fixes — timing, terminals, the shape of a UI. Those are engineering work, not a line of markdown. Pretending a rule fixes them is how a rules file rots.

The discipline is small: before writing a rule, ask if it can be a check instead. Only when it truly cannot does it earn a line in the file. A good rules file is short, made of the things you cannot enforce any other way.

What I came away with

Ferment used more models than chat mode did — but not a smarter one. MiniMax M3 built the feature both times. Everything else that decided the outcomes — the scoping that found the real requirement, the typecheck that saved a debugging spiral, the reviewer with no stake in the build — was the architecture around that model. Cast AI says it in one line: the difference is “not the model. It’s the architecture.” I did not read that until after the trial. It is just the thing the trial kept showing me — and the layer I have spent this past year building small tools around.

Watching someone else’s harness from the operator seat, with a real task on the line, is the most fun I have had with an agent in a while. Ferment got the shape of the work more right than I expected, and got a boring part wrong in a way that taught me more than the wins did.