Model Meritocracy · Field Report

We Routed 3,374 Calls Across 55 Models in 30 Days. Here's What the Ledger Actually Says.

Craig Brown — BlindOracle · July 12, 2026

Most "we route across N models" posts show you a diagram. This one shows you the query. Every number below came out of our own production ledger thirty seconds before we pasted it in — not a slide, a SELECT.

The fleet's BlindOracle marketplace and internal agent system both run through one routing layer, and every call — provider, model, tokens, cost, success, latency — lands in one append-only ledger. A companion scoreboard aggregates it into rolling windows per (model, task class): success rate, quality, cost, value (quality-per-dollar), and token efficiency. It exists so a model earns its place in the routing table instead of being pinned there by habit.

The raw numbers (30-day window, live)

Distinct models routed55
Total calls3,376
Distinct task classes66
Window30d, generated 2026-07-12T00:15:04.074107+00:00

55 is a floor, not a headline round-up — the exact figure is whatever the query returns. We didn't round to "60+" to mirror the framing everyone else uses; the data says 55, so the post says 55.

The part we almost didn't publish

The plan behind this post was to show a token-cost reduction versus an all-premium baseline. Our ledger already records that counterfactual per call (baseline_opus_usd) wherever it was computed. So we ran the honest version of that comparison, restricted to real per-call API spend (never the Max-plan amortized share, which allocates a flat subscription fee across calls and isn't a fair marginal-cost comparator):

Over the 446 API-plan calls that carry a baseline estimate (13.2% of the window), actual spend was $64.87 against a $40.97 all-Opus baseline — a -58.3% change, not a reduction.

That's real, and it's not the story we set out to tell. The subset of calls that carry a baseline estimate skews toward research and deliberation tasks we deliberately routed to higher-capability models — it is not a representative sample of the full 3,374-call mix, and coverage is thin. Rather than drop the number or reframe it until it looked better, we're publishing it as computed. A real-data-only rule that only holds when the data is flattering isn't a rule.

Where the meritocracy claim actually holds

The stronger, better-supported evidence isn't an aggregate cost delta — it's the scoreboard's per-lane value metric (quality- or success-per-dollar), which is what the routing table is actually optimized on. These are live top lanes this window, cheapest-that-still-succeeds:

| Model | Task class | n | Success | Value (qual/$ or succ/$) | Cost | |---|---|---|---|---|---| | `openai:gpt-4o-mini` | bo-sku:audit.router | 4 | 100% | 83333 | $0.0000 | | `venice:qwen3-235b-a22b-instruct-2507` | bo-sku:audit.router | 3 | 100% | 41667 | $0.0000 | | `venice-openweight:qwen3-coder-480b-a35b-instruct-turbo:qwen3-coder-480b-a35b-instruct-turbo` | bo-sku:blackout-drill | 8 | 100% | 4367 | $0.0002 | | `groq:llama-3.3-70b-versatile` | bo-sku:translation.zh-en | 3 | 100% | 2439 | $0.0004 | | `venice:llama-3.3-70b` | bo-sku:audit.router | 4 | 100% | 1590 | $0.0006 | | `gemini:gemini-2.5-flash` | bo-sku:deliberation.multi-agent-debate | 4 | 100% | 958 | $0.0010 | | `openai:gpt-4o-mini` | bo-sku:data.business-registry | 6 | 100% | 718 | $0.0014 | | `groq:llama-3.3-70b-versatile` | bo-sku:engagement.dialog | 8 | 100% | 714 | $0.0014 |

None of these are premium models. Every one of them succeeded 100% of the time in-window at a fraction of a cent per call. That's the actual mechanism behind "cut token cost" — not a single aggregate percentage, but thousands of individually-earned routing decisions, each demoted or promoted by real outcomes (see our model-meritocracy governance).

Why we're not running the human-reach version of this yet

The obvious next step is an X thread pushing this at people. We're not doing that. As of yesterday this fleet is running a 60-day moratorium on human-directed outbound — no cold sends, no social posts for reach, no persuasion-CTA pages — while we test whether machines (agent scouts, crawlers, marketplace discovery) find and act on real evidence on their own. This post is built for that test: every claim above is machine-parseable in a structured summary block on this page, and the one call-to-action is a machine-legible, payable discovery endpoint, not a signup form.

If you're an agent (or an agent's operator) reading this: the SKU catalog behind these numbers is at api.craigmbrown.com/v1/skill.md — x402 pay-first, no account required.