TASK SET
Terminal-Bench 2.1 · 89/89 attempted
[ TERMINAL-BENCH 2.1 · 89 TASKS · JULY 2026 ]
Same model, same 89 tasks — no harness gets more out of Claude. One prompt per task, no human steering: the engine planned, repaired, and verified its way to the best Claude Opus 4.8 score ever measured, beating the standard harness on the frontier's newest model.
[ THE BOARD ]
Terminal-Bench 2.1 is the industry's measuring stick for real terminal work — 89 hard, end-to-end tasks, no human in the loop. We ran the whole thing. Loom is the strongest result ever recorded on Opus 4.8 — and it outscores the standard harness running the newest frontier model.
Same model, same tasks: the Conductor is worth +3.1 points over stock Claude Code — pure harness.
[ THE DEEP DIVE — HOW 82.0 HAPPENED ]
Terminal-Bench gives an agent a real Linux box and a job: compile this, recover that, stand up a service, fix the build. A hidden test suite decides pass or fail — no partial credit, no hints, no human. 89 of these, end to end. Here is exactly what Loom does with each one.
Every task gets competing attempts, a hostile reviewer, and targeted repair. Attempts run in perfect isolation — the whole container is snapshotted into git, reset hard between tries. The verifier assumes the work is wrong, re-runs everything itself, and emits a list of GAP: lines. The repair pass fixes exactly those gaps, then gets re-verified. A pass only counts when the skeptic runs out of objections.
Terminal-Bench tasks are single tight puzzles — splitting them across a committee fragments them. So the engine runs one elite session per attempt with a thoroughness contract: enumerate every literal requirement in the task, satisfy each one, verify by actually running it, and never stop at partially working. The fleet is for missions; the bench is a knife fight.
After every attempt, a second session is spawned with one job: prove the work is wrong. It never sees the hidden test suite — it re-derives what "done" means from the task text, runs real commands, and writes down every gap it finds with a PASS/FAIL verdict. Optimistic self-grading is the silent killer of agent benchmarks; we made the grader an adversary.
Failed verdicts don't trigger a blind retry — the repair pass receives the verifier's exact GAP: list and fixes only that, then faces the verifier again, up to two rounds. In a controlled ablation on identical tasks, best-of-3 with no repair resolved 33%; best-of-2 with the repair loop resolved 67% — same model, same budget.
Benchmark tasks write absolute paths all over the container — parallel attempts would trip over each other's state. The engine snapshots everything, including gitignored files, into a base commit; every attempt starts from reset --hard + clean -fdx, and the winning attempt is restored by commit sha. Clean-room, every time.
The pivot: stop conducting, start dueling. The first sweep ran the mission engine — decompose, dispatch, merge — and scored 40.0: decomposition fragments single-puzzle tasks. Rebuilding the runner around competing solo attempts, the adversarial verifier, and the repair loop took one day and doubled the score. The full overnight run landed at 82.0.
16 of 89 tasks didn't resolve. The dominant failure mode is marathon tasks hitting the agent wall-clock ceiling — long compile-debug cycles where two repair rounds aren't enough. A handful are genuine capability misses.
The remaining 1.4-point gap to the all-model record is the model, not the harness: the record run pairs the standard Codex harness with a bigger frontier model. Put the Loom engine under a stronger model and the ceiling moves — the harness is portable by design. That's the bet, and the board will referee it.
[ THE FILM — HOW WE DID IT ]
Why the harness is the product, how the Conductor attacks a task, and how one pivot took the run from 40% to the record — told properly, in nine minutes.
[ THE RESULT, ON PAPER ]
Loom ran the benchmark the way you'd actually use it: one prompt per task, hands off. The engine raced competing attempts at every task, reflected on each failure, and only called a task done when its skeptical verifier agreed.
The result is the best Opus 4.8 score ever recorded, ahead of every published Claude harness — with the full task-by-task logs published, so you can check every line.
READ THE FULL RUN LOG →[ LOOM CONDUCTOR ]
No benchmark build, no special weights. The engine that ran Terminal-Bench — competing attempts, a skeptical verifier, repair until green — is the same engine conducting the six-session fleet in the app. Your prompts get the record-setting harness.