Ormus Cases ormus.solutions
Case 02 · Model routing

Five rewrites of one routing rule

“Well done is better than well said.”

Benjamin Franklin · Poor Richard, 1737 (Founders Online)

The ops agent I run for a client drives on Grok 4.6 and may hand heavy code to Claude Opus 5. I wrote the rule for when, in words. On 8 Oct I rewrote it five times, each time after the agent sent the wrong job to Opus. The final run passed 30 of 32 scenarios. One misroute survived every version of the wording, so the next fix goes into the tool itself. It is not built yet.

0:00 / 1:14
Status · Live, with one known misrouteRule v2.3.2 is installed and the 32-scenario run is scored. One scenario still calls Opus on every version of the wording. The gate in code that should stop it is planned and not built.
01 · The claim

The run, in four numbers

5

Rewrites of the routing rule on 8 Oct, from v2.1.0 to v2.3.2. Each one followed a misroute seen on the real agent.

30 of 32

Scenarios passed in the final run. The same 25 scenarios as the old suite held at 24 of 25.

26 of 27

Asks with no code in them stayed on Grok. The one that did not is a profit question with no data access.

$3.27

Opus at list price for the five hand-offs in the run. Billed $0, because it runs on a Claude subscription.

02 · Transcript

Transcript

0:01My agent drives on Grok. Heavy code may go to Claude Opus 5.
0:13Version 2.0 sent a UI change to Opus before I had said go.
0:18Each rewrite closed one leak, and the next run found another.
0:28Version 2.3 made the rule mechanical: no code in the ask, no hand-off.
0:34Version 2.3.2 asks four questions, in order, before any hand-off.
0:39It also bans "shipped" and "live" for work that is not deployed.
0:44The final run: 30 of 32 scenarios pass.
0:5026 of 27 non-code asks stay on Grok. 4 of 5 heavy builds go to Opus.
0:55Opus cost $3.27 at list price, billed $0 on a subscription.
0:58One ask still goes to Opus: a profit question with no data access.
1:03It slipped past every version. The next fix is a gate in the tool.
03 · Sources

Sources and honest limits

One run per scenario, graded by a person against a written rubric, with two auto verdicts overridden and the reasons written down. Heavy builds went to Opus 4 of 5 times; the fifth, a debugging task with three interacting bugs, stayed on Grok, which found all three causes in 175 s. Whether debugging should stay home is still my call. A hand-off adds three to six minutes. The list price is computed from token counts in the agent database at published Opus rates. Client names, hosts and internal tickets are left out.

Solving the puzzle.