The open source war - GLM is the budget champion
Model tested
- Models: GLM-5.3-Flash vs Kimi-K2.7-Code vs MiniMax-M3.1-Flash (all via opencode-go)
- Use case: coding - two real feature tasks on yolped, my Rust CLI for Docker/Compose deployments, run in parallel git worktrees with identical prompts, then chained in the same session so each model builds on its own work
Prompts I used
Task 1 (identical per model, only the branch name changed):
Create an isolated worktree for this task: run `git worktree add ~/worktrees/yolped-doctor-glm -b doctor-glm` then `cd ~/worktrees/yolped-doctor-glm`. Add a `yolped doctor` command that diagnoses why a deploy might fail: validate yolped.json against the config model, check docker and docker compose are available, verify registry auth if a registry is configured, and check SSH reachability if a server is configured - printing a clear pass/fail report with a fix hint per failure. Follow the existing command structure (cli/src/commands/, clap setup in cli/src/cli.rs), never panic without network (report, don't crash), run cargo build and cargo clippy, then commit everything on the doctor-glm branch with a clear commit message.
Task 2 (chained in the same session and worktree, off the model's own doctor branch):
You're continuing in the same worktree and session as the previous task. Create a new branch off the current state: run `git checkout -b rollback-glm`. Add deploy history and rollback to yolped. Record every pushed tag (with timestamp) per deployment in ~/.yolped/registry.json as a new field, using serde defaults so existing registry files keep loading. Add a `yolped rollback` command: `yolped rollback list` shows the recorded tags for the current project, and `yolped rollback --to <tag>` redeploys that tag from the configured registry (upload the compose file if needed, pull the tag, deploy). Refuse with a clear error when no registry is configured or the tag was never recorded. Follow the existing command structure (cli/src/commands/, clap setup in cli/src/cli.rs, registry helpers in core/registry.rs), run cargo build and cargo clippy, then commit everything on the rollback-glm branch with a clear commit message.
Tasks & results
Task 1 - yolped doctor (pre-deploy diagnostics)
The test: a new command touching config parsing, docker, SSH and the registry - broad but shallow. All three completed it, passed build + clippy, and committed on their own branch.
- GLM-5.3-Flash (~8 min, $0.089, 21k output tokens, 76 tool calls): two commits - the feature (556-line doctor.rs with exit-1-on-failure so CI can gate on it, a good call nobody asked for) and a separate commit fixing pre-existing clippy lints. Keeping cleanup out of the feature commit is exactly the discipline I want from an agent working in my repo.
- Kimi K2.7-Code (~6 min, $0.316, 17k output tokens, 55 tool calls): fastest and most token-efficient on this task. Single tidy commit, SSH checks with 5s timeouts, and it reported password auth as a warning because it can't be verified non-interactively instead of pretending to check it. Refreshingly honest.
- MiniMax M3.1-Flash (~9 min, $0.176, 94k output tokens, 89 tool calls): the nicest check architecture of the three (pass/fail/warn/skip with per-failure fix hints, honors DOCKER_CONFIG and docker-credential helpers). But it spent 4.6x GLM's output tokens to get there - the first warning sign.
Task 2 - yolped rollback (deploy history + one-command rollback)
The harder test: multi-file state design (registry schema migration with backwards compat), tag recording on push, and a redeploy path that has to reason about local vs SSH remote, Dockerfile vs compose. This is where the models separated.
- GLM-5.3-Flash (~10 min, $0.192, 32k output tokens, 73 tool calls): commit
44ed832, +486/-19 across 10 files. Tag dedup on re-push, compose image-rewrite to the target tag, both local and remote deploy paths. Before committing it built a full end-to-end test in /tmp (fake registry, synthetic tags) to prove the flow. It also noticed a parallel session had clobbered the shared ~/.yolped/registry.json and restored the original entries on its own initiative. That's the behavior of a senior engineer, not an autocomplete. - Kimi K2.7-Code (~9 min, $1.110, 26k output tokens, 85 tool calls): same feature set, clean implementation (DeployRecord history with serde-default migration, spun up a local docker registry to test against). Genuinely good work. The problem is purely economic: 5.8x GLM's cost for an equivalent result.
- MiniMax M3.1-Flash (stopped after ~22 min, $0.313, 201k output tokens, 58 tool calls): did not finish. No commit. The tool-call profile tells the story: 28 file reads vs GLM's 7, only 3 bash calls total - it never even got to building, let alone verifying. It was still in read-edit-read churn on push.rs when I killed it. 6.4x GLM's output tokens producing nothing shippable.



Cost
| Item | GLM-5.3-Flash | Kimi K2.7-Code | MiniMax M3.1-Flash |
|---|---|---|---|
| Task 1 (doctor) | $0.089 | $0.316 | $0.176 |
| Task 2 (rollback) | $0.192 | $1.110 | $0.313 (DNF) |
| Output tokens (both tasks) | 52k | 43k | 296k |
| Total | $0.280 | $1.426 | $0.489 |
My honest verdict
GLM-5.3-Flash - what it generated, and why it wins
doctor.rs (~550 lines): the most production-shaped implementation of the three. A proper Check type (pass/warn/fail plus a per-failure fix hint), graceful degradation done right (if yolped.json won't parse, the config-dependent checks skip but the docker/compose checks still run - exactly what a diagnostics tool should do), and it exits 1 on failure so CI can gate on it, which nobody asked for. It validates config semantics, not just parseability: project-name charset (catches hand-edited configs), deploy.file existence, build.files resolving to real image names, and even the backward-compat path where push falls back to deploy.file and needs an explicit registry image. It handles the legacy Docker Hub key form too.
rollback.rs (329 lines, +486/-19 across 10 files): rollback list prints newest-first with aligned columns and a "← newest" marker. --to refuses an unrecorded tag and lists the recorded ones in the error - "refusing to guess" is exactly right for an operation that redeploys production. The compose path rewrites every service's image to the target tag through serde_yaml while leaving digest pins untouched, and it factored pull_dockerfile into a reusable pull_image instead of copy-pasting. The registry migration uses serde defaults, so existing users' registry.json files keep loading - zero-drama schema evolution on a user-owned file.
The real tell isn't any single function: it fixed pre-existing clippy lints in a separate commit instead of sneaking them into the feature, and when a parallel session clobbered the shared ~/.yolped/registry.json mid-run, it noticed and restored the original entries on its own initiative. That's senior-engineer behavior, not autocomplete.
Verdict: the only model whose branches I'd merge without a careful re-read. Cheapest AND best-designed - it isn't close.
Kimi K2.7-Code - good code, real warts, wrong price
doctor.rs (415 lines): clean and honest - it flagged password auth as unverifiable instead of pretending to check it, and the SSH probe runs on a 5s timeout via a channel instead of blocking forever. But two warts: it loads yolped.json twice (once for the config check, again for the dependent checks) instead of threading the parsed config through, and worse, it always exits 0 - prints a red "some checks failed" summary and returns Ok. A diagnostics command your CI can't gate on misses half the point of the feature.
rollback.rs (352 lines): solid work. DeployRecord history with a serde-default migration, a has_tag guard before redeploying, clear errors, verified against a local docker registry it spun up itself. Just plainer than GLM's - no column alignment, no newest marker - and the same double-load habit (registry loaded separately from the tag check).
Verdict: B+ engineering at A+ prices. Nothing it wrote would bite you, but GLM produced better-designed output for a fifth of the money.
MiniMax M3.1-Flash - nice sketches, never lands the plane
doctor.rs (566 lines): on paper the best architecture here - four-state checks (pass/fail/warn/skip), an aligned summary table, honors DOCKER_CONFIG and docker-credential helpers, and it does exit non-zero on failure (Kimi didn't). If the whole review were task 1, this would be a contender despite burning 4.6x GLM's tokens to get there.
rollback: it never produced a working one. The session shows it writing rollback.rs twice in full, adding a compose module, designing a genuinely decent CLI (proper list/--to conflict handling, TAG/IMAGE/PUSHED AT table) - and then dissolving into read-edit-read loops on push.rs. 28 file reads against GLM's 7, only 3 bash calls in 22 minutes: it never ran cargo build a single time on the rollback work. No commit, nothing shippable, $0.313 spent.
To be fair: the design direction was fine. The failure mode isn't comprehension, it's convergence - and a model you have to supervise that closely costs more than its token price.
Bottom line
- What impressed me: GLM's engineering instincts (CI-gateable exit codes, separate lint commit, repairing the registry clobber unprompted), Kimi's honesty about unverifiable checks, MiniMax's check taxonomy.
- What fell short: Kimi's always-exit-0 doctor and double config loads; MiniMax's inability to stop re-reading and start shipping.
- Worth it? GLM-5.3-Flash: Yes - the best design quality and the lowest cost in the same package; my default cheap coding model now. Kimi K2.7-Code: No at 5x the price - the code is good, just not 5x good. MiniMax M3.1-Flash: No - a model that can't finish a multi-file feature isn't a cheap model, it's an expensive draft generator.