Model strength table and fleet.json override (kyhae)

Cyberine ships a sourced answer to "which model is strong and which is weak": every model id the fleet can run has a coarse rank from 1 to 5 (5 = strongest) and a cost class from 1 to 3, keyed by model id rather than by provider, because one provider serves many models. The table is MODEL_STRENGTH_TABLE in src/core/modelStrengthTable.ts; resolveModelStrength in src/core/modelStrength.ts answers a lookup. Every row, its benchmark numbers, its sources with retrieval dates and its VERIFIED or UNVERIFIED tag are in docs/providers/model-strength.md (snapshot of 2026-09-17). You never have to declare strength yourself; fleet.json lets you correct the table where you disagree.

Precedence, highest first: a strength on the fleet.json model entry, a strength on the provider entry, the shipped table, and finally rank 3 for a model the table does not know.

Run recipe

# See the rank each routed worker got and where it came from (dry run, nothing dispatched)
cyberine route "Refactor the entire authentication module across every file in the repo and add tests."
#   -  claude preferred: strength=4 (strength table claude-sonnet-5=4) suits code-large task
#   -  codex preferred: strength=4 (model '(default)' is not in the strength table; profile tier=strong stands in as 4) suits code-large task
# ...and after codex has reported a real model in a run.json (krcz0), the same placeholder resolves through it instead:
#   -  codex preferred: strength=5 (strength table gpt-6-astra=5; run.json last reported "gpt-6-astra" for placeholder "(default)") suits code-large task

# List the enabled models the table cannot rank (a warning, never an error)
cyberine fleet doctor
#   warn   provider "codex" model "(default)" has no strength row — ranked 3 (middle) for routing; add "strength" (1-5) to providers.codex or to the model entry in fleet.json, or see docs/providers/model-strength.md; the last run.json for codex reports model "gpt-6-astra", which the table ranks 5

# fleet list also names the reported model next to any placeholder model it stands in for
cyberine fleet list
#   reported models (last run.json for a placeholder model):
#     codex:default (default) -> gpt-6-astra (rank 5)

What the router does with the rank: on a code-large or complex task, workers whose model ranks 4 or 5 move to the front of the already-selected pool; on a qa or trivial task, workers ranking 2 or lower, or with cost class 1, move to the front and the rest of the 4-and-5 group moves to the back. A worker whose model has no row (the placeholder ids such as (default), (account-configured), or the gateway aliases anthropic/claude and openai/gpt) keeps its profile tier as the stand-in (strong 4, standard 3, flash 2), so its position is what it was before the table existed — UNLESS that worker is a cli-agent and the most recent run.json in the global registry reported a real model for it (workerRuns[].model, scanning the 50 most recent runs), in which case resolveWorkerStrength in src/core/modelStrength.ts (krcz0) ranks the REPORTED model instead of standing in on tier, and basis names both the reported model and the placeholder it replaced. The lookup runs once per process (primeReportedWorkerModels in src/store/reportedWorkerModels.ts, memoized) and every router/fleet list/fleet doctor strength read shares that one cached map — cyberine route and cyberine fleet list (below) prime it themselves; the real dispatch path primes it in resolveBuildFleetRoute. With no run.json to report from, nothing changes: the profile-tier stand-in above still applies exactly as before krcz0.

Extend recipe: override a rank in fleet.json

Add an integer strength from 1 to 5 to a provider entry, to a model entry, or both, in whichever fleet.json is in effect (the project .cyberine/fleet.json wholly replaces the global one, and both accept the field):

{
  "version": 1,
  "providers": {
    "codex": {
      "kind": "cli-agent", "workerId": "codex", "enabled": true, "default": false,
      "strength": 5,
      "models": { "default": { "id": "(default)" } }
    },
    "deepseek": {
      "kind": "key-loop", "workerId": "loop-deepseek-build", "enabled": true, "default": true,
      "models": { "flash": { "id": "deepseek-flash", "strength": 3 } }
    }
  }
}

• A model entry's strength applies when that entry's id is the model being ranked. It beats the provider's strength, which beats the table.
• Any other value (0, 6, 2.5, "5", null) is a structural error: the file is unusable until fixed, exactly like any other invalid field, and cyberine fleet doctor names it and exits 1:

cyberine fleet doctor
#   ERRORS (structural):
#     error  fleet.json: provider "claude".strength must be an integer from 1 to 5 (5 = strongest), got 7

• The router ranks worker profiles, not individual seat lanes. When several entries bind the same workerId (claude-kt and claude-ft both bind claude), a matching model-entry strength wins first, then the first provider-level strength in file order.
• To add or re-rank a model in the shipped table, edit src/core/modelStrengthTable.ts and docs/providers/model-strength.md together, following that page's "Refresh procedure". The table refuses to load with a duplicate id, a rank outside 1 to 5, or a row that cites no source.

No environment variable is involved.

Known limitations and deferred work

• The router only partitions by rank (4 and above is strong, 2 and below is weak). It does not yet sort the strong group strongest-first, nor pick the planner, the task splitter or the judges by rank; that is the separate follow-up khv45.
• A placeholder model is resolved through the last reported model only for a cli-agent worker (codex, cursor-agent, crush, amp, goose, openhands, aider, ...); a key-loop worker's placeholder (e.g. the Command Code free lane's (live-catalog-free)) is never substituted this way. An explicit fleet.json strength still wins over either.
• The reported-model cache is primed once per process at the start of cyberine route, cyberine fleet list and a real dispatch's resolveBuildFleetRoute; a run that finishes mid-process is not picked up until the next process starts. With no run.json in the global registry to report from, the profile-tier stand-in is unchanged.
• The rank is a dated snapshot (2026-09-17), refreshed by hand under the R02 upstream sweep. Nothing fetches benchmarks at runtime.
• A rank describes the model at the reasoning effort its source row was measured at (the highest AA publishes, or high for the muse worker). A run at a lower effort performs below its rank.
• Namespaced ids such as deepseek/deepseek-v4-flash mean different builds on different routes; the table ranks the OpenRouter meaning. A lookup with no exact row retries once without a single vendor/ prefix.
• The cost class is the model's metered list price. It does not describe how a particular worker is billed.
• The LLM smart router's prompt (CYBERINE_SMART_ROUTER) still describes workers by profile tier, not by rank.