Providers
Model strength reference (2026-09-17)
One sourced answer to "which enabled model is strongest", used by the router today and meant for planning, task splitting and judging. It ranks every model id the fleet can run (every WORKER_PROFILES model and variant, every modelCatalog.ts entry, and the model codex reports in run.json) on a coarse 1 to 5 scale, where 5 is strongest. The data lives in src/core/modelStrengthTable.ts; resolveModelStrength in src/core/modelStrength.ts reads it.
A dated snapshot, not a live feed. Every number below was retrieved on 2026-09-17, except the
gpt-6.1-sol,gpt-6-solandgpt-6-lunarows, which were added from a re-fetch of the same leaderboard on 2026-10-02 (S9, S10). Benchmarks move every few weeks, so the table is re-derived under the R02 upstream sweep (the "Model strength benchmarks" row in CHANGELOG-WATCH.md). Until then, trust afleet.jsonoverride over this page.
Tags
- VERIFIED: a page fetched on 2026-09-17 shows a measured number for this exact model (the same weights and version; a routing namespace such as
deepseek/or:clouddoes not change that). - UNVERIFIED: the rank is inherited or inferred, and the Notes column says why. The usual reasons are an alias that now answers as another model, a faster serving of a measured model, a snapshot date that does not match the benchmark row, or an index that Artificial Analysis itself labels as estimated.
How a rank is derived
The spine is the Artificial Analysis Intelligence Index (II) on S1: one composite, measured by one independent party, with 652 model rows on the page. Its per-model data also carries the coding and agentic evals used as cross-checks here: Terminal-Bench 4.0, Terminal-Bench 2.1 and Terminal-Bench Hard (TB4.0, TB2.1, TB-Hard), SciCode, GPQA and HLE among them. For each model the row at the highest reasoning effort AA publishes is used (usually max or xhigh; the Gemini Flash rows are high), except where Cyberine itself pins a lower effort for the worker that runs the model: then the row at that effort, or the nearest higher one AA publishes, is used. Today that exception is only muse, which passes --reasoning-effort high (muse-cli.md).
| Rank | Band on II | Meaning |
|---|---|---|
| 5 | 48 and above | frontier: plans, splits and judges |
| 4 | 36 to 47.9 | strong: complex implementation |
| 3 | 24 to 35.9 | capable: well-specified tasks; also the rank given to an unknown model |
| 2 | 12 to 23.9 | limited: small, single-module work |
| 1 | below 12 | legacy or small models |
The coding cross-checks (AA's own Terminal-Bench runs, vendor SWE-bench Verified via S2, and vendor model cards S3 to S8) never move a rank in this snapshot. A band edge is a judgement call either way; stating the rule and listing every disagreement below keeps the table reproducible, and the operator settles a disputed model with a fleet.json override. Ties are intended: this is an ordinal, not a score.
Cost class
Same semantics as WorkerProfile.costClass (1 cheap, 3 expensive), set from the model's metered output list price per 1M tokens: class 1 up to $2.50, class 2 up to $12, class 3 above $12. Prices come from src/core/modelPricing.ts (verified live by earlier tasks), openai.md for gpt-6-astra ($10/$50), and S1's listed price where the repo has none. The Muse Spark ids are class 3 even though S1 lists $1.25/$4.25: the loop-muse-build profile in src/core/workerProfiles.ts records that Meta publishes no readable rate for the raw API and pins class 3 so an unknown rate never wins a cheapest-first pick; this table follows the repo. Where neither the repo nor S1 has a price (gemini-2.0-flash-001, command-r-plus, sonar-pro), the class is copied from the worker profile or the model family and the row is UNVERIFIED.
The class describes the model's list price, not how a particular worker is billed: a subscription seat running claude-sonnet-5 still reports class 2 here.
Sources
| Key | Source | What it shows | Fetched |
|---|---|---|---|
| S1 | https://artificialanalysis.ai/leaderboards/models | Intelligence Index plus per-eval scores (TB4.0, TB2.1, TB-Hard, SciCode, GPQA, HLE) and list prices for 652 model rows, embedded as JSON in the page | raw HTML with curl, 2026-09-17 |
| S2 | https://llm-stats.com/benchmarks/swe-bench-verified | SWE-bench Verified, 116 models, every score marked self_reported: true with the vendor page it came from | raw HTML with curl, 2026-09-17 |
| S3 | https://openai.com/index/gpt-6-astra/ | Terminal-Bench 4.0: GPT-6 Astra 57.9, GPT-5.6 Sol 37.3, Claude Fable 5.1 55.8, Claude Fable 5 44.5, Claude Opus 5 52.6 | through the r.jina.ai reader (direct fetch returns 403), 2026-09-17 |
| S4 | https://www.anthropic.com/claude-fable-and-mythos-5-1 | Terminal-Bench 4.0: Fable 5.1 55.8, Fable 5 42.0, Opus 5 52.3, GPT-5.6 Sol 37.3; states Fable 5.1 defaults to High effort in Claude Code | through the r.jina.ai reader (direct fetch returns 403), 2026-09-17 |
| S5 | https://openai.com/index/gpt-5-6/ | SWE-Bench Pro: Sol 64.6, Terra 63.4, Luna 62.7 | through the r.jina.ai reader, 2026-09-17 |
| S6 | https://huggingface.co/deepseek-ai/DeepSeek-V4.1-Flash | Terminal-Bench 2.1 (max effort): Opus 5 89.1, GPT-5.6 Sol 88.8, Kimi K3 88.3, DeepSeek V4 Pro 87.9, V4 Flash 82.7, V4.1 Flash 90.6 | raw HTML with curl, 2026-09-17 |
| S7 | https://huggingface.co/deepseek-ai/DeepSeek-V4-Pro-0813 | Terminal Bench 2.1: DeepSeek V4 Pro 0813 87.9 | raw HTML with curl, 2026-09-17 |
| S8 | https://huggingface.co/moonshotai/Kimi-K3 | Terminal-Bench 2.1: Kimi K3 88.3 | raw HTML with curl, 2026-09-17 |
| S9 | https://artificialanalysis.ai/leaderboards/models | S1 re-fetched for the GPT-6 tiers only: GPT-6.1 Sol (Max) II 51.8, GPT-6 Sol (Max) II 47.5, GPT-6 Luna (Max) II 37.3, all intelligenceIndexIsEstimated: false; the rest of the table is not re-derived from it | raw HTML with curl, 2026-10-02 |
| S10 | https://developers.openai.com/api/docs/models/gpt-6.1-sol, /gpt-6-sol, /gpt-6-luna | list prices per 1M in/out: 6.1 Sol $2/$10 (cached $0.10), 6 Sol $2/$10 (cached $0.20), 6 Luna $0.10/$0.50 (cached $0.01); 6.1 Sol 1,050,000 context, 128,000 max output | raw HTML with curl, 2026-10-02 |
Vendor numbers (S2 to S8) are self-reported with each vendor's own harness, so they are corroboration, not the ranking input. FLEET-BENCH (cyberine eval bench) is not a source here: it spends quota by design and has no published per-model numbers for most of these ids.
Strength table
Numbers are percentages except II, which is AA's index. Rows are sorted by rank, then by II.
| Model id | Rank | Cost class | Status | Evidence | Sources (retrieved) | Notes |
|---|---|---|---|---|---|---|
claude-fable-5-1 | 5 | 3 | VERIFIED | AA "Claude Fable 5.1 (Adaptive Reasoning, Max Effort, Default Fallback)": II 53.4; TB4.0 52.0, TB2.1 91.4; vendor: TB4.0 55.8 (Anthropic, OpenAI tables) | S1, S3, S4 (2026-09-17) | |
gpt-6-astra | 5 | 3 | VERIFIED | AA "GPT-6 Astra (max)": II 52.8; TB4.0 59.1, TB2.1 88.4; vendor: TB4.0 57.9 (OpenAI) | S1, S3, S4 (2026-09-17) | |
gpt-6.1-sol | 5 | 2 | VERIFIED | AA "GPT-6.1 Sol (Max)": II 51.8; TB4.0 56.1 | S9, S10 (2026-10-02) | the xhigh row is II 51.0 (band 5) and the medium row II 47.8 (band 4), so a worker run below xhigh performs below this rank |
claude-opus-5 | 5 | 3 | VERIFIED | AA "Claude Opus 5 (Adaptive Reasoning, Max Effort)": II 50.7; TB4.0 49.0, TB2.1 89.1; vendor: TB4.0 52.3 (Anthropic) / 52.6 (OpenAI); TB2.1 89.1 (DeepSeek table) | S1, S3, S4, S6 (2026-09-17) | |
claude-fable-5 | 5 | 3 | VERIFIED | AA "Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback)": II 49.7; TB4.0 42.4, TB2.1 84.6, TB-Hard 62.9; SWE-bench Verified 95.0 (vendor, "Claude Fable 5"); vendor: TB4.0 42.0 (Anthropic) / 44.5 (OpenAI) | S1, S3, S4, S2 (2026-09-17) | |
gpt-6-sol | 4 | 2 | VERIFIED | AA "GPT-6 Sol (Max)": II 47.5; TB4.0 43.9 | S9, S10 (2026-10-02) | the xhigh row is II 44.1 and the medium row II 39.8, both still band 4 |
gpt-5.6-sol | 4 | 3 | VERIFIED | AA "GPT-5.6 Sol (max)": II 47.1; TB4.0 39.9, TB2.1 88.0, TB-Hard 65.9; vendor: SWE-Bench Pro 64.6 (OpenAI); TB2.1 88.8 (DeepSeek table) | S1, S5, S6 (2026-09-17) | |
gpt-5.6 | 4 | 3 | UNVERIFIED | AA "GPT-5.6 Sol (max)": II 47.1; TB4.0 39.9, TB2.1 88.0, TB-Hard 65.9; vendor: SWE-Bench Pro 64.6 (Sol, OpenAI) | S1, S5 (2026-09-17) | alias, routes to gpt-5.6-sol (modelCatalog) |
muse-spark-1.3 | 4 | 3 | VERIFIED | AA "Muse Spark 1.3 (xhigh)": II 45.2; TB4.0 16.7, TB2.1 85.4 | S1 (2026-09-17) | Cyberine's muse worker passes --reasoning-effort high (muse-cli.md) and AA publishes no high row, so the nearest higher row (xhigh, II 45.2) sets the rank; the max row is II 48.2 |
muse-spark-1.3-contributor | 4 | 3 | UNVERIFIED | AA "Muse Spark 1.3 (xhigh)": II 45.2; TB4.0 16.7, TB2.1 85.4 | S1 (2026-09-17) | same weights as muse-spark-1.3 under contributor data terms; ranked at the same effort |
kimi-k3 | 4 | 3 | VERIFIED | AA "Kimi K3 (max)": II 43.8; TB4.0 12.6, TB2.1 85.0; vendor: TB2.1 88.3 (Moonshot card and DeepSeek table) | S1, S8, S6 (2026-09-17) | |
gpt-5.6-terra | 4 | 2 | VERIFIED | AA "GPT-5.6 Terra (max)": II 42.3; TB4.0 35.4, TB2.1 88.0, TB-Hard 57.6; vendor: SWE-Bench Pro 63.4 (OpenAI) | S1, S5 (2026-09-17) | |
claude-opus-4-8 | 4 | 3 | VERIFIED | AA "Claude Opus 4.8 (Adaptive Reasoning, Max Effort)": II 42.0; TB4.0 21.7, TB2.1 84.6, TB-Hard 58.3; SWE-bench Verified 88.6 (vendor, "Claude Opus 4.8") | S1, S2 (2026-09-17) | |
muse-spark-1.2 | 4 | 3 | VERIFIED | AA "Muse Spark 1.2 (xhigh)": II 39.8; TB4.0 7.1, TB2.1 80.1 | S1 (2026-09-17) | |
muse-spark-1.2-contributor | 4 | 3 | UNVERIFIED | AA "Muse Spark 1.2 (xhigh)": II 39.8; TB4.0 7.1, TB2.1 80.1 | S1 (2026-09-17) | same weights as muse-spark-1.2 under contributor data terms |
deepseek-flash | 4 | 1 | VERIFIED | AA "DeepSeek V4.1 Flash (Reasoning, Max Effort)": II 39.5; TB4.0 26.8; vendor: TB2.1 90.6 (DeepSeek card) | S1, S6 (2026-09-17) | |
deepseek/deepseek-v4.1-flash | 4 | 1 | VERIFIED | AA "DeepSeek V4.1 Flash (Reasoning, Max Effort)": II 39.5; TB4.0 26.8; vendor: TB2.1 90.6 (DeepSeek card) | S1, S6 (2026-09-17) | |
deepseek-v4-flash | 4 | 1 | UNVERIFIED | AA "DeepSeek V4.1 Flash (Reasoning, Max Effort)": II 39.5; TB4.0 26.8 | S1, S6 (2026-09-17) | alias since 2026-09-11, answers as deepseek-flash (modelCatalog) |
deepseek-v4-flash-vision-exp | 4 | 1 | UNVERIFIED | AA "DeepSeek V4.1 Flash (Reasoning, Max Effort)": II 39.5; TB4.0 26.8 | S1, S6 (2026-09-17) | alias since 2026-09-11, answers as deepseek-flash and ignores images (modelCatalog) |
deepseek-v4-pro | 4 | 1 | UNVERIFIED | AA "DeepSeek V4.1 Flash (Reasoning, Max Effort)": II 39.5; TB4.0 26.8 | S1, S6, S7 (2026-09-17) | routed to deepseek-flash on api.deepseek.com from 2026-09-14 (modelCatalog); the 0813 Pro it replaced also sits in band 4 (II 36.3) |
gemini-3.7-flash | 4 | 2 | VERIFIED | AA "Gemini 3.7 Flash (high)": II 39.4; TB4.0 13.6, TB2.1 85.8 | S1 (2026-09-17) | |
claude-sonnet-5-5 | 4 | 2 | UNVERIFIED | not in the 2026-09-17 snapshot (released 2026-09-28) | Anthropic models overview (2026-10-01) | inherits claude-sonnet-5's rank until the next snapshot; same list price ($2/$10), so the cost class is measured, the rank is not |
claude-sonnet-5 | 4 | 2 | VERIFIED | AA "Claude Sonnet 5 (Adaptive Reasoning, Max Effort)": II 38.4; TB4.0 14.1, TB2.1 80.5; SWE-bench Verified 85.2 (vendor, "Claude Sonnet 5") | S1, S2 (2026-09-17) | |
gpt-5.6-luna | 4 | 1 | VERIFIED | AA "GPT-5.6 Luna (max)": II 37.5; TB4.0 11.6, TB2.1 80.9; vendor: SWE-Bench Pro 62.7 (OpenAI) | S1, S5 (2026-09-17) | |
gpt-6-luna | 4 | 1 | VERIFIED | AA "GPT-6 Luna (Max)": II 37.3; TB4.0 12.6 | S9, S10 (2026-10-02) | band 4 only at max effort: the xhigh row is II 33.9 and the medium row II 29.5, both band 3 |
deepseek/deepseek-v4-pro-0813 | 4 | 1 | VERIFIED | AA "DeepSeek V4 Pro 0813 (Reasoning, Max Effort)": II 36.3; TB4.0 14.1, TB2.1 78.7; vendor: TB2.1 87.9 (DeepSeek card) | S1, S7 (2026-09-17) | |
deepseek/deepseek-v4-flash-0731 | 3 | 1 | VERIFIED | AA "DeepSeek V4 Flash 0731 (Reasoning, Max Effort)": II 34.5; TB4.0 12.1, TB2.1 78.7 | S1 (2026-09-17) | |
gemini-3.6-flash | 3 | 2 | VERIFIED | AA "Gemini 3.6 Flash (high)": II 34.3; TB4.0 7.1, TB2.1 77.5 | S1 (2026-09-17) | |
gemini-3.5-flash | 3 | 2 | VERIFIED | AA "Gemini 3.5 Flash (high)": II 33.0; TB4.0 6.6, TB2.1 78.7, TB-Hard 40.9 | S1 (2026-09-17) | |
deepseek/deepseek-v4-pro | 3 | 1 | UNVERIFIED | AA "DeepSeek V4 Pro 0424 (Reasoning, Max Effort)": II 30.9; TB4.0 14.6, TB2.1 64.0, TB-Hard 46.2 | S1 (2026-09-17) | OpenRouter undated alias frozen on the 2026-04-23 GA; AA labels its row 0424 |
claude-sonnet-4-6 | 3 | 3 | VERIFIED | AA "Claude Sonnet 4.6 (Adaptive Reasoning, Max Effort)": II 30.5; TB4.0 3.0, TB2.1 71.2, TB-Hard 53.0; SWE-bench Verified 79.6 (vendor, "Claude Sonnet 4.6") | S1, S2 (2026-09-17) | |
gemini-3.1-pro-preview | 3 | 2 | VERIFIED | AA "Gemini 3.1 Pro Preview": II 30.4; TB4.0 4.0, TB2.1 73.8, TB-Hard 53.8; SWE-bench Verified 80.6 (vendor, "Gemini 3.1 Pro") | S1, S2 (2026-09-17) | |
MiniMax-M3 | 3 | 1 | VERIFIED | AA "MiniMax-M3": II 29.6; TB4.0 2.0, TB2.1 65.2, TB-Hard 42.4; SWE-bench Verified 80.5 (vendor, "MiniMax M3") | S1, S2 (2026-09-17) | |
kimi-k2.6 | 3 | 2 | VERIFIED | AA "Kimi K2.6": II 27.5; TB4.0 0.5, TB2.1 65.9, TB-Hard 43.9; SWE-bench Verified 80.2 (vendor, "Kimi K2.6") | S1, S2 (2026-09-17) | |
mimo-v2.5-pro | 3 | 1 | VERIFIED | AA "MiMo-V2.5-Pro": II 26.4; TB4.0 0.0, TB2.1 65.2, TB-Hard 43.2; SWE-bench Verified 78.9 (vendor, "MiMo-V2.5-Pro") | S1, S2 (2026-09-17) | |
kimi-k2.7-code | 3 | 2 | VERIFIED | AA "Kimi K2.7 Code": II 26.3; TB4.0 1.0, TB2.1 67.4, TB-Hard 44.7 | S1 (2026-09-17) | |
moonshotai/kimi-k2.7-code | 3 | 2 | VERIFIED | AA "Kimi K2.7 Code": II 26.3; TB4.0 1.0, TB2.1 67.4, TB-Hard 44.7 | S1 (2026-09-17) | OpenCode route name for kimi-k2.7-code |
kimi-k2.7-code-highspeed | 3 | 2 | UNVERIFIED | AA "Kimi K2.7 Code": II 26.3; TB4.0 1.0, TB2.1 67.4, TB-Hard 44.7 | S1 (2026-09-17) | faster serving of kimi-k2.7-code, not measured separately |
deepseek/deepseek-v4-flash | 3 | 1 | UNVERIFIED | AA "DeepSeek V4 Flash 0420 (Reasoning, Max Effort)": II 24.6; TB4.0 2.5, TB2.1 61.8, TB-Hard 35.6; SWE-bench Verified 78.6 (vendor, "DeepSeek-V4-Flash-0423") | S1, S2 (2026-09-17) | OpenRouter undated alias frozen on the April GA; the same string on Command Code is that vendor's own build |
kimi-k2.5 | 2 | 2 | UNVERIFIED | AA "Kimi K2.5 (Reasoning)": II 23.5 (AA estimate); TB2.1 45.7, TB-Hard 34.8; SWE-bench Verified 76.8 (vendor, "Kimi K2.5") | S1, S2 (2026-09-17) | |
MiniMax-M2.7 | 2 | 1 | VERIFIED | AA "MiniMax-M2.7": II 23.2; TB4.0 0.0, TB2.1 55.4, TB-Hard 39.4 | S1 (2026-09-17) | |
MiniMax-M2.7-highspeed | 2 | 1 | UNVERIFIED | AA "MiniMax-M2.7": II 23.2; TB4.0 0.0, TB2.1 55.4, TB-Hard 39.4 | S1 (2026-09-17) | faster serving of MiniMax-M2.7, not measured separately |
MiniMax-M2.5 | 2 | 1 | UNVERIFIED | AA "MiniMax-M2.5": II 22.8 (AA estimate); TB-Hard 34.8; SWE-bench Verified 80.2 (vendor, "MiniMax M2.5") | S1, S2 (2026-09-17) | |
gemini-3.5-flash-lite | 2 | 1 | VERIFIED | AA "Gemini 3.5 Flash-Lite": II 22.7; TB4.0 1.0, TB2.1 53.6 | S1 (2026-09-17) | |
mimo-v2.5 | 2 | 1 | VERIFIED | AA "MiMo-V2.5": II 22.3; TB4.0 0.0, TB2.1 63.7, TB-Hard 41.7 | S1 (2026-09-17) | |
kimi-k2-thinking | 2 | 2 | UNVERIFIED | AA "Kimi K2 Thinking": II 22.0 (AA estimate); TB-Hard 31.1 | S1 (2026-09-17) | DISCONTINUED 2026-05-25 per moonshot-kimi.md but still listed in modelCatalog.ts |
anthropic/claude-sonnet-4.5 | 2 | 3 | VERIFIED | AA "Claude 4.5 Sonnet (Reasoning)": II 21.2; TB4.0 0.0, TB2.1 55.8, TB-Hard 35.6 | S1 (2026-09-17) | |
MiniMax-M2.1 | 2 | 1 | UNVERIFIED | AA "MiniMax-M2.1": II 20.9 (AA estimate); TB-Hard 28.8; SWE-bench Verified 67.0 (vendor, "MiniMax M2.1") | S1, S2 (2026-09-17) | |
meituan/LongCat-2.0 | 2 | 1 | VERIFIED | AA "LongCat 2.0": II 19.7; TB4.0 0.0, TB2.1 50.2 | S1 (2026-09-17) | Command Code paid id; the :free tier was retired (Command Code v1.58.0, verified 2026-09-19), kz9mk renamed the row 2026-09-25 |
MiniMax-M2 | 2 | 1 | UNVERIFIED | AA "MiniMax-M2": II 18.6 (AA estimate); TB-Hard 25.8 | S1 (2026-09-17) | |
glm-4.6 | 2 | 1 | UNVERIFIED | AA "GLM-4.6 (Reasoning)": II 18.5 (AA estimate); TB2.1 49.4, TB-Hard 25.0; SWE-bench Verified 68.0 (vendor, "GLM-4.6") | S1, S2 (2026-09-17) | |
claude-haiku-4-5 | 2 | 2 | VERIFIED | AA "Claude 4.5 Haiku (Reasoning)": II 17.6; TB4.0 0.0, TB2.1 44.2, TB-Hard 27.3; SWE-bench Verified 73.3 (vendor, "Claude Haiku 4.5") | S1, S2 (2026-09-17) | |
gemini-2.5-pro | 2 | 2 | VERIFIED | AA "Gemini 2.5 Pro": II 16.7; TB4.0 0.0, TB2.1 28.5, TB-Hard 26.5; SWE-bench Verified 63.2 (vendor, "Gemini 2.5 Pro") | S1, S2 (2026-09-17) | |
google/gemini-2.5-pro | 2 | 2 | VERIFIED | AA "Gemini 2.5 Pro": II 16.7; TB4.0 0.0, TB2.1 28.5, TB-Hard 26.5 | S1 (2026-09-17) | |
gemini-3.1-flash-lite | 2 | 1 | UNVERIFIED | AA "Gemini 3.1 Flash-Lite": II 16.0; TB4.0 0.5, TB2.1 31.1, TB-Hard 24.2 | S1 (2026-09-17) | AA row slug is the preview build |
kimi-k2-turbo-preview | 2 | 1 | UNVERIFIED | AA "Kimi K2 0905": II 15.3 (AA estimate); TB-Hard 23.5; SWE-bench Verified 65.8 (vendor, "Kimi K2-Instruct-0905") | S1, S2 (2026-09-17) | DISCONTINUED 2026-05-25 per moonshot-kimi.md but still listed in modelCatalog.ts; no benchmark row of its own, ranked by the K2 0905 generation of the same period |
gemini-2.5-flash | 2 | 1 | UNVERIFIED | AA "Gemini 2.5 Flash (Reasoning)": II 13.1 (AA estimate); TB-Hard 13.6; SWE-bench Verified 60.4 (vendor, "Gemini 2.5 Flash") | S1, S2 (2026-09-17) | |
gpt-oss:120b-cloud | 2 | 1 | VERIFIED | AA "gpt-oss-120b (high)": II 12.3; TB4.0 0.0, TB2.1 26.2, TB-Hard 23.5 | S1 (2026-09-17) | Ollama Cloud tag of gpt-oss-120b |
grok-3 | 2 | 3 | UNVERIFIED | AA "Grok 3": II 12.1 (AA estimate); TB-Hard 11.4 | S1 (2026-09-17) | |
qwen3-coder-plus | 1 | 2 | UNVERIFIED | AA "Qwen3 Coder 480B A35B Instruct": II 11.9 (AA estimate); TB-Hard 18.9; SWE-bench Verified 69.6 (vendor, "Qwen3-Coder 480B A35B Instruct") | S1, S2 (2026-09-17) | DashScope hosted Qwen3-Coder; ranked by the 480B-A35B open release |
mistral-large-latest | 1 | 1 | UNVERIFIED | AA "Mistral Large 3": II 9.7; TB4.0 0.0, TB2.1 12.0, TB-Hard 15.9 | S1 (2026-09-17) | moving alias; ranked by Mistral Large 3, the newest Large on AA |
accounts/fireworks/models/deepseek-v3 | 1 | 1 | UNVERIFIED | AA "DeepSeek V3 (Dec '24)": II 8.5; TB4.0 0.0, TB2.1 16.9, TB-Hard 6.8; SWE-bench Verified 42.0 (vendor, "DeepSeek-V3") | S1, S2 (2026-09-17) | Fireworks snapshot not confirmed; V3 0324 (II 9.7) lands in the same band |
gemini-2.0-flash-001 | 1 | 1 | UNVERIFIED | AA "Gemini 2.0 Flash (Feb '25)": II 8.9 (AA estimate); TB-Hard 3.8 | S1 (2026-09-17) | |
gpt-4o | 1 | 2 | UNVERIFIED | AA "GPT-4o (Nov '24)": II 8.4 (AA estimate); TB-Hard 8.3; SWE-bench Verified 33.2 (vendor, "GPT-4o") | S1, S2 (2026-09-17) | |
openai/gpt-4o | 1 | 2 | UNVERIFIED | AA "GPT-4o (Nov '24)": II 8.4 (AA estimate); TB-Hard 8.3; SWE-bench Verified 33.2 (vendor, "GPT-4o") | S1, S2 (2026-09-17) | |
anthropic.claude-3-5-sonnet-20241022-v2:0 | 1 | 3 | UNVERIFIED | AA "Claude 3.5 Sonnet (Oct '24)": II 7.9 (AA estimate); SWE-bench Verified 49.0 (vendor, "Claude 3.5 Sonnet") | S1, S2 (2026-09-17) | |
llama-3.3-70b-versatile | 1 | 1 | UNVERIFIED | AA "Llama 3.3 Instruct 70B": II 7.7 (AA estimate); TB2.1 4.9, TB-Hard 3.0 | S1 (2026-09-17) | Groq serving |
llama-3.3-70b | 1 | 1 | UNVERIFIED | AA "Llama 3.3 Instruct 70B": II 7.7 (AA estimate); TB2.1 4.9, TB-Hard 3.0 | S1 (2026-09-17) | Cerebras serving |
meta-llama/Llama-3.3-70B-Instruct-Turbo | 1 | 1 | UNVERIFIED | AA "Llama 3.3 Instruct 70B": II 7.7 (AA estimate); TB2.1 4.9, TB-Hard 3.0 | S1 (2026-09-17) | Together serving, FP8 turbo build |
meta/llama-3.3-70b-instruct | 1 | 1 | UNVERIFIED | AA "Llama 3.3 Instruct 70B": II 7.7 (AA estimate); TB2.1 4.9, TB-Hard 3.0 | S1 (2026-09-17) | NVIDIA NIM serving |
sonar-pro | 1 | 2 | UNVERIFIED | AA "Sonar Pro": II 7.6 (AA estimate) | S1 (2026-09-17) | |
llama3.1:8b | 1 | 1 | UNVERIFIED | AA "Llama 3.1 Instruct 8B": II 6.9 (AA estimate); TB2.1 1.5, TB-Hard 0.8 | S1 (2026-09-17) | local Ollama; quantization depends on the pull |
openai/gpt-4o-mini | 1 | 1 | UNVERIFIED | AA "GPT-4o mini": II 6.7 (AA estimate); TB2.1 5.6; SWE-bench Verified 8.7 (vendor, "GPT-4o mini") | S1, S2 (2026-09-17) | |
gemini-2.5-flash-lite | 1 | 1 | UNVERIFIED | AA "Gemini 2.5 Flash-Lite (Non-reasoning)": II 6.7 (AA estimate); TB-Hard 2.3; SWE-bench Verified 31.6 (vendor, "Gemini 2.5 Flash-Lite") | S1, S2 (2026-09-17) | |
command-r-plus | 1 | 2 | UNVERIFIED | AA "Command-R+ (Apr '24)": II 5.3 (AA estimate) | S1 (2026-09-17) |
75 rows: 38 VERIFIED, 37 UNVERIFIED.
Not ranked: unknown, rank 3
These ids have no row, so resolveModelStrength returns rank 3 with source unknown, and cyberine fleet doctor warns about every enabled provider that uses one.
| Id | Used by | Why there is no row |
|---|---|---|
(default) | codex, aider, crush | the CLI's own configured default. For codex the effective model is recorded per run as workerRuns[].model (kd7ax); on this machine it reported gpt-6-astra (rank 5) on 2026-09-17 and gpt-6-sol (rank 4) on 2026-10-02, and fleet doctor prints that reported model's rank next to the warning |
(account-configured) | cursor-agent | chosen in the Cursor account, not per run |
(smart-default) | amp | Amp picks the model; no public model flag |
(provider-default) | goose | depends on CYBERINE_GOOSE_PROVIDER |
(configured-at-first-run) | openhands | picked in OpenHands' own first-run setup |
(live-catalog-free) | loop-commandcode-free-build | resolved from Command Code's live free list, which rotates |
anthropic/claude, openai/gpt | loop-claude-build, loop-gpt-build | LiteLLM gateway aliases; the gateway config decides the model |
MiniMax-Text-01 | mmx-text-build (retired) | no row for it on S1, and no vendor benchmark was fetched |
Inside the router a placeholder does not fall to 3. The capability reorder substitutes the worker profile's tier for an unranked model (strong 4, standard 3, flash 2), so codex and cursor-agent keep the position they had before this table existed. The fix for a real ranking is a fleet.json override.
Where the sources disagree
- DeepSeek V4.1 Flash (
deepseek-flash) is rank 4, not 5. DeepSeek's own card (S6) puts it at 90.6 on Terminal-Bench 2.1, above Opus 5's 89.1 in the same table. AA's independent run puts it at 26.8 on Terminal-Bench 4.0, against Opus 5's 49.0 and GPT-6 Astra's 59.1, and its II is 39.5. The independent measurement decides the rank. - Muse Spark 1.3 is rank 4 at the effort Cyberine runs it. Its
maxrow (II 48.2, TB4.0 33.3) would be band 5, but themuseworker passes--reasoning-effort high, and the nearest published row,xhigh, is II 45.2 with TB4.0 16.7. An operator who runs muse at a higher effort (CYBERINE_MUSE_REASONING_EFFORT) can set"strength": 5on the provider. - Kimi K3 is band 4 on II (43.8) and TB2.1 (85.0; 88.3 on its own card), but only 12.6 on TB4.0.
- Gemini 3.1 Pro Preview and Claude Sonnet 4.6 are band 3 on II (30.4 and 30.5), yet their TB-Hard (53.8 and 53.0) and vendor SWE-bench Verified (80.6 and 79.6) look like band 4 coders.
- GPT-5.6 Luna reaches band 4 only at max effort (II 37.5). AA's medium-effort row is 25.5, which is band 3, and a run at a lower effort than the ranked row performs below its rank. The same caveat applies to every row; S4 notes that Claude Code runs Fable 5.1 at High by default, where AA's row (51.2) is still band 5.
- Vendor SWE-bench Verified has saturated around 80 for many band 3 models (MiniMax M3 80.5, Kimi K2.6 80.2, MiMo-V2.5-Pro 78.9), so it rarely separates bands. The newest frontier models (GPT-6 Astra, Fable 5.1, Opus 5, GPT-5.6) report SWE-Bench Pro or Terminal-Bench 4.0 instead.
- Namespaced ids depend on the route.
deepseek/deepseek-v4-proanddeepseek/deepseek-v4-flashare ranked as OpenRouter's frozen April GA aliases (see openrouter.md); the same strings on Command Code or OpenCode name those vendors' own builds. The lookup also strips a singlevendor/prefix when there is no exact row, so a user-typedmoonshotai/kimi-k2.6resolves tokimi-k2.6. - Catalog versus docs.
kimi-k2-turbo-previewandkimi-k2-thinkingare listed inmodelCatalog.tsbut marked DISCONTINUED 2026-05-25 in moonshot-kimi.md. They keep their rows, at rank 2, until the catalog drops them.
Override from fleet.json
fleet.json accepts an optional integer strength (1 to 5) on a provider entry and on a model entry. Precedence, highest first: the model entry, the provider entry, this table, unknown (3). Any other value is a structural error that cyberine fleet doctor reports and that makes the file unusable, like every other invalid field. The recipe is in USAGE.md under "Model strength table and fleet.json override (kyhae)".
Refresh procedure
- Download S1 as raw HTML (the WebFetch summarizer truncates it) and read each model's
intelligenceIndex,intelligenceIndexIsEstimated,terminalBench40,terminalBench21andterminalbenchHardfrom the embedded JSON. - Add a row for every id newly listed in
modelCatalog.tsorWORKER_PROFILES, and for any new modelcodexreports inrun.json. - Re-band every row with the rules above, update
src/core/modelStrengthTable.tsand this page together under a new date, and bumpMODEL_STRENGTH_RETRIEVED. - Re-fetch the vendor cards (S3 to S8) only to refresh the disagreement list; they do not set ranks.
Upstream Changelog Watch List
Every upstream Cyberine follows, in one table, so "anything new upstream?" resolves to the right changelog in one hop. URLs are pulled from each provider's page in this folder (its "Official docs"...
DeepSeek API (direct)
All paths are relative to https://api.deepseek.com. Every one below was called live on 2026-08-25 (T679) and returned the status shown.