cyberine
UsageRun recipes

Hard spend cap — opt-in, race-free ceiling across ALL providers

CYBERINECREDITMINUSD above is a SOFT, per-provider gate: it reads the folded balance and down-ranks/skips a worker, but two concurrent dispatches can each read "balance ok" moments apart and,...

CYBERINE_CREDIT_MIN_USD above is a SOFT, per-provider gate: it reads the folded balance and down-ranks/skips a worker, but two concurrent dispatches can each read "balance ok" moments apart and, together, spend past any cap. CYBERINE_SPEND_CAP_USD closes that race with an append-then-verify reservation (src/store/spendCap.ts): every dispatch appends a reservation event UNCONDITIONALLY, then re-reads the fully folded total (cumulative realized spend across all providers + every still-active reservation, including its own) before granting. Whichever dispatch's read lands later always sees every reservation appended before it, so the cap can never be collectively exceeded (a dispatch may be conservatively denied in a rare simultaneous race, but nothing is ever let through over the cap).

The cumulative total is tracked by spendCap.ts itself (recordRealizedSpend, appended once per completed dispatch) — it does NOT depend on a per-provider credits set balance existing in creditLedger.ts. Earlier versions summed only ledger-tracked providers' spend, which silently stayed at $0 forever for a provider with no configured balance; this is fixed, so the cap works correctly with zero setup beyond CYBERINE_SPEND_CAP_USD itself.

Gated dispatch paths — every dispatch in the repo funnels through one of exactly two choke points, and both reserve/release/record identically: Orchestrator.dispatch (cyberine run/cli.ts, the MCP orchestrate tool, npm run demo, and autonomyLoop.ts's non-code gate) and WorkflowEngine.runWorker (npm run wf, isolate builds, autonomy-loop sync-verify dispatch).

WHICH calls count (khz46): only those whose resolved billing class is credits — a per-token paid API. A subscription or free call (a Claude seat, codex, local Ollama, a Command Code call drawing the plan's credit pool) takes no reservation and records no USD here at all; its credits are tracked apart. See "Dollars and credits are counted apart" below.

Unset (the default) = disabled, zero overhead, behaviour identical to pre-T482 Cyberine. Set it to turn on the hard ceiling:

# Never let total accounted+reserved spend across ALL providers exceed $20
CYBERINE_SPEND_CAP_USD=20 cyberine run "refactor the auth module"

# Same, for a workflow dispatch, with a shorter reservation TTL (default 10 minutes)
CYBERINE_SPEND_CAP_USD=20 CYBERINE_SPEND_CAP_TTL_MS=120000 npm run wf -- .local/dogfood/some.workflow.ts

When a dispatch is denied, that worker's Result is status: 'error' with a message naming the cap and the current accounted+reserved total — the run continues with whatever other workers were already dispatched (a denial is not a run-level crash). A reservation is released as soon as its worker's dispatch finishes (success, error, or cancellation); an unreleased reservation (e.g. a killed process) still frees itself once its TTL elapses, so a crash can never permanently eat budget.

This is a headless-orchestrator design: there is no interactive "approve this spend" prompt — "asking the user" means setting CYBERINE_SPEND_CAP_USD up front.

spendcap-events.jsonl (under ~/.cyberine/credits/) rotates/compacts into spendcap-summary.json once it exceeds CYBERINE_SPENDCAP_ROTATION_BYTES (default 256 KB), mirroring creditLedger.ts's sealed-segment fold, so a fan-out-heavy fleet run does not pay an ever-growing full-file fold on every reservation check. A still-open (unreleased) reservation is carried forward across a rotation rather than dropped.

Run the regression smoke (spawns real concurrent OS processes to prove the race is actually closed, and exercises the Orchestrator.dispatch gate, cumulative tracking, rotation, and the composed spendCapStatus() / dailyCapStatus() field values the four spend-visibility surfaces below render):

npm run selftest:spend-cap

Spend visibility (T636) — see both caps before either one blocks a dispatch

Both the hard cap above and the daily cap (CYBERINE_DAILY_USD_CAP, OFF by default since 2026-09-12 — see "Resource guards" / dailySpendCapUsd()) compute accurate numbers continuously, but by default nothing shows them until the moment dispatch is actually refused. Four surfaces now report the live posture proactively:

cyberine doctor                    # "Spend & cap:" section — daily cap %, hard cap % (if set),
                                    # and any credit-ledger entry at low/exhausted status
cyberine cost                      # header lines for both caps + a "daily cap%" column per day
curl -s 127.0.0.1:41700/metrics    # always includes spendCaps: { daily, hard } (previously the
                                    # daily snapshot only appeared once the cap had ALREADY fired)
cyberine tui                       # fleet header shows an "acct daily $spent/$cap (NN%)" chip always,
                                    # plus an "acct cap $spent/$cap (NN%)" chip when CYBERINE_SPEND_CAP_USD is set

Prompt-cache hit rate per worker (kayhj)

cyberine cost <runId> carries a cache column: the share of each worker's prompt tokens the provider served from its prompt cache (DeepSeek prompt_cache_hit_tokens, Anthropic-shaped cache_read_input_tokens, both already normalised into Usage.cacheHitTokens), and a total row that is token-weighted across the workers that reported it (90% (2/3 reporting)), never a mean of percentages. A worker whose provider reports no cache figures shows -.

cyberine cost <runId>              # worker | in/out | cost | cache
npm run selftest:billing-cache-hit # the helpers + a positive-controlled check that the command renders them

Known Limitations and Deferred Work: only the per-run view carries the column; the per-day / per-provider summary (cyberine cost --days N) does not aggregate cache figures yet, and CLI workers (claude / codex / muse) report cache reads only where their stream exposes them.

Cache-hit-aware cost estimation (k92fe)

The cache column above used to be display-only: creditLedger.estimateCostUsd() (and the loop worker's own running costUsd, which GlobalRegistry's billableCostUsd and therefore the daily cap read verbatim) priced every input token at the cache-MISS rate, so a DeepSeek run with an 80-95% cache-hit ratio overstated real spend by roughly 3-6x and could trip CYBERINE_DAILY_USD_CAP far earlier than the account was actually being billed. modelPricing.ts now carries an optional cachedInputCostPerMTok (and peak.cachedInputCostPerMTok) alongside each model's normal rate; getModelPricing(), computeChatCost(), and estimateCostUsd() all take an optional cache-hit-token count and price that many input tokens at the cached rate instead, falling back to the normal rate for every model without one. LoopAgentAdapter passes its accumulated cacheHitTokens through automatically — no env var or flag needed.

Rows with a verified published cached rate today: DeepSeek direct deepseek-flash and deepseek-v4-pro (both off-peak and peak, from api-docs.deepseek.com/quick_start/pricing), OpenAI direct gpt-5.6-sol / gpt-5.6-terra / gpt-5.6-luna / gpt-5.6 (from developers.openai.com/api/docs/pricing), and MiniMax direct MiniMax-M3 / MiniMax-M2.7 / MiniMax-M2.5 / MiniMax-M2.1 (from platform.minimax.io/docs/guides/pricing-paygo).

Known Limitations and Deferred Work: every other row — the legacy DeepSeek deepseek-v4-flash / deepseek-v4-flash-vision-exp ids, every OpenRouter-multiplexed row, Anthropic/Google/xAI/Kimi/ Xiaomi direct rows, and any model not in modelPricing.ts at all — has no cached rate and keeps billing 100% of its input tokens at the normal rate, unchanged from before k92fe. computeChatCost callers outside the loop adapter (the chat TUI status line, the oracle tool) do not yet pass a cache-hit count, so their cost chips are unaffected by this fix.

In the TUI fleet header the chips are scope-labeled: the aggregate cost of the listed runs renders as proj $N (or all $N under cyberine agents' cross-project global mode), while the two cap meters carry an acct prefix — those are ALWAYS account-wide (~/.cyberine), regardless of which project's runs are listed, so the numbers can legitimately disagree with the proj figure.

src/store/spendCap.ts's spendCapStatus() composes spendCapUsd() + spendCapSpentUsd() into one { enabled, capUsd, spentUsd, remainingUsd, pctUsed } snapshot (mirrors dailyCapStatus()'s shape) — read-only, never touches reserveSpend/releaseReservation/recordRealizedSpend.

Daily-cap admission reservation (k9wpc) — closing the concurrent-dispatch race

checkDispatchGuards() above (the daily-cap half of "Resource guards") is a READ-ONLY, point-in-time check: it sums today's already-recorded spend and refuses a NEW dispatch once that sum is at/over CYBERINE_DAILY_USD_CAP, but it never re-checks mid-run. Two or three dispatches admitted within the same race window each read the SAME "under cap" snapshot before any of them has posted real spend, so all are admitted and then each spends freely for its own entire run. This was observed 2026-08-26 as a real 22% daily overshoot: three cyberine run dispatches launched ~3 minutes apart took the day from $8.90 to $12.18 against a $10.00 cap, with the cap nominally enforced the whole time.

The fix is reserveDailyBudget() / releaseDailyBudget() (src/store/dailySpend.ts) — an append-then-verify admission RESERVATION for the daily cap, the same discipline the hard cap above already uses (spendCap.ts's reserveSpend): append a reservation event UNCONDITIONALLY, then re-read today's recorded spend plus every still-open reservation (including the one just appended) before granting. Whichever caller's read lands later always sees every reservation appended before it, so the cap can never be collectively exceeded by more than the admission math already accounted for at each grant decision.

checkDispatchGuards() itself is deliberately left UNCHANGED — several callers (/health, cyberine reap's status print) use it read-only with no matching release, so turning it into a reservation would leak budget for each of them. Instead the reservation is a SEPARATE layer wired into the specific choke points that actually dispatch:

  • Orchestrator.dispatch() — reserves agents.length * (Task.maxBudgetUsd ?? 0.50) before fanning out, releases in a finally once every agent's dispatch settles.
  • launchBackgroundRun() (cyberine run --bg, chat /run, the TUI's "n" new-task, and — transitively, since they all dispatch through it — cron jobs and the task-queue consumer) — reserves CYBERINE_DAILY_CAP_RUN_ESTIMATE_USD (default $1.00) before spawning the detached child. This path relies on TTL expiry rather than an explicit release (see Known Limitations).
  • The web chat route (POST /api/chat/:id) — mirrors the hard cap's existing reserve/release pair around its own direct dispatch (it bypasses Orchestrator.dispatch).
# Reproduce + verify the fix (spawns real concurrent OS processes against a seeded registry,
# proves the pre-fix race with the unmodified checkDispatchGuards() alone, then proves
# reserveDailyBudget() closes it, then proves it end-to-end through Orchestrator.dispatch)
npm run selftest:daily-cap-reservation

# Tune the reservation's own knobs (rarely needed — defaults match the observed incident):
CYBERINE_DAILY_CAP_RUN_ESTIMATE_USD=2.00 cyberine run --bg "a pricier multi-step task"
CYBERINE_DAILY_CAP_RESERVATION_TTL_MS=1800000 cyberine run --bg "..."   # default 30 min

Known Limitations and Deferred Work: this closes the ADMISSION race, not estimate accuracy — a run whose real cost runs far above its own admission estimate can still push the day over cap; size Task.maxBudgetUsd (or CYBERINE_DAILY_CAP_RUN_ESTIMATE_USD for background dispatch) to the run's actual expected cost for a tighter bound. launchBackgroundRun()'s reservation is opened before the PreDispatch hook / permission-mode / smart-spawn-budget checks further down that function — but since T700 (see below) a dispatch rejected by one of THOSE gates releases immediately instead of waiting out CYBERINE_DAILY_CAP_RESERVATION_TTL_MS, and the reservation for a dispatch that DID spawn travels to the detached child in CYBERINE_DAILY_RESERVATION_ID and is released when that child exits. TTL remains the backstop for a child that is SIGKILLed or never starts. cron.ts / taskQueue.ts / serve.ts's own read-only checkDispatchGuards() calls are unmodified — they inherit the admission gate transitively through launchBackgroundRun, but do not reserve a second time themselves.

Spend-cap truth (T700) — what the caps count, which paths they cover, what reservations are open

Six related fixes to the money guards, all shipped together because they overlap in src/store/dailySpend.ts, src/store/spendCap.ts and src/store/GlobalRegistry.ts.

(a) A subscription CLI worker no longer counts as spend. A claude / codex / cursor-agent / loop-copilot-build worker draws on a flat subscription: the total_cost_usd it reports is usage telemetry, not money owed. That figure used to be folded into the registry's totalCostUsd and summed straight into the daily cap. Measured on this machine's own history, claude CLI workers had contributed $47.02 across 59 worker runs, and 2026-06-10 alone carried $31.04 of CLI-notional cost against $0.00 of API-real spend — enough to pause every dispatch on the machine three times over against the default $10 cap.

A run record now carries two figures: totalCostUsd (notional, unchanged, what cyberine cost and the fleet Cost column render) and billableCostUsd (real money owed to a metered provider, what the daily cap sums). The split is keyed on each worker's costModel (src/core/workerProfiles.ts) — the same field CHEAP_BUILD_FLEET's drift guard and implementTaskWorkflow's design-step whitelist already trust — stamped onto the WorkerRun at dispatch, with the profile registry as the fallback for records written before T700. Note the deliberate asymmetry with the hard cap's older kind === 'loop' test: a METERED CLI worker (the opencode-* profiles) is real money and still counts.

(b) Auxiliary LLM calls now move both caps and appear in run cost. Judges (LlmJudge, AdversarialJudge's N skeptics plus its synth pass, SecurityJudge, BehaviorJudge, DiffJudge, the swarm/lane merge judges), the LLM router, the candidate re-ranker, chat session titling, the NL goal planner, the non-code verify gate, the swarm/smart planners, self-heal and integration-fix used to spend entirely outside both caps — makeOpenAICompletion returned only a string and discarded res.usage. An AdversarialJudge round can cost more than the fleet dispatch it judges.

src/store/auxSpend.ts accounts each such call in one place: the hard cap's tally, a day-bucketed ~/.cyberine/credits/aux-spend.jsonl the daily cap sums, and — when the call happens inside a run — that run's new auxCostUsd, folded into totalCostUsd so run cost is honest. Every makeOpenAICompletion consumer inherits this with no call-site change; makeOpenAICompletionWithUsage returns the token counts and accounted cost for a caller that wants them. consult_oracle is deliberately NOT routed here: it runs inside a loop worker's tool loop and its cost is already folded into that worker's own Usage via ToolContext.reportUsage, so an aux record would bill the same tokens twice.

(c) cyberine chat is gated. Both local surfaces — the Ink TUI (via plannerTurn) and the non-TTY readline REPL — now run the same guard/reserve/record trio the web chat route already did, through the shared src/chat/chatSpendGate.ts. A blocked turn refuses with the guard's own message before any provider call. The web route keeps its own trio and opts out (spendGate: false) rather than gating twice.

(d) npm run wf and every workflow run are gated. WorkflowEngine.run() now runs the daily-cap guard plus one admission reservation for the whole run, and runWorker re-checks read-only before each worker so a long DAG that crosses the cap mid-flight stops dispatching NEW workers instead of spending its whole admission on one $1.00 reservation. Previously the daily cap (default ON) never reached src/engine/ at all, so the path this repo dogfoods ran with zero spend enforcement. That per-worker re-check is memoized for CYBERINE_DAILY_CAP_RECHECK_MS (default 15000; 0 disables the memo) because its cap branch reads the whole cross-project registry — without it a 30-chunk swarm step would fire 30 full registry scans in the same instant, all seeing the same recorded spend.

(e) Reservations are released, not left to expire. launchBackgroundRun hands its reservation id to the detached child in CYBERINE_DAILY_RESERVATION_ID; the child releases it on exit (src/cyberine-bg-run.ts) and does not take a second one. Every early-abort path between the reservation and the spawn — PreDispatch hook block, plan/dontAsk mode, the smart-spawn budget gate, a router failure — releases immediately. Before this, 11 background runs in any 30-minute window denied every dispatch on the machine regardless of real spend, and a plan-mode /run that dispatched nothing still burned $1.00 of headroom for half an hour.

(f) Reservations are visible. Both caps deny on recorded + reserved, but every read surface reported only the recorded half — with 11 open reservations the gate said ~$11.0000 already recorded+reserved of $10.00 while cyberine cost printed 3% and doctor reported healthy. DailyCapStatus and SpendCapStatus now carry reservedUsd, openReservations, admissionBlocked and reservationTtlMs, rendered everywhere the recorded figure appears:

cyberine cost      # an indented line under each cap: "N open reservations holding $X (each expires after ~30m)"
cyberine doctor    # a "daily cap reservations:" / "hard cap reservations:" check, warn when admission is blocked
cyberine tui       # a "reserved $X xN" chip in the fleet header, warn-toned when it is blocking
curl -s 127.0.0.1:41700/metrics   # spendCaps.daily / spendCaps.hard carry the new fields

admissionBlocked is deliberately distinct from reached: reached is a spend fact ("raise the cap or wait for the UTC day"), admissionBlocked is an admission fact ("wait for a reservation to release or expire"), and the remedies differ.

# Regression coverage (both smokes were extended for T700)
npm run selftest:spend-cap              # sections 10-12: the cost-model split, aux accounting, reservation visibility
npm run selftest:daily-cap-reservation  # sections 4-8: CLI exclusion end-to-end, aux moves the cap, release paths

Known Limitations and Deferred Work. The aux ledger is day-bucketed and pruned to today's entries past CYBERINE_AUX_SPEND_PRUNE_BYTES (64 KB) — it answers "how much aux spend today", not "how much this month"; there is no per-provider or per-source aux report. An aux call against an endpoint outside the provider registry (a LiteLLM gateway, a self-hosted URL) prices at creditLedger's $1/$5 fallback rate rather than the real one — conservative, but not accurate. Run attribution for aux spend uses an AsyncLocalStorage context installed by WorkflowEngine.run, so a judge call made from a front-end that never opens a run (orchestrate()'s judge phase in cli.ts/mcp.ts) still moves both caps but is attributed to no run. src/store/MemoryStore.ts's two opt-in LLM calls (CYBERINE_MEMORY_MODEL distillation, embeddings) are still unaccounted; they default to a local Ollama endpoint priced at $0 and were out of T700's scope. The costModel split governs the DAILY cap only — the hard cap's own recordRealizedSpend still gates on adapter.info.kind === 'loop' (now in addition to khz46's billing-class gate below), so a metered opencode-* CLI worker's real spend counts against the daily cap but not the hard one. And the chat gate reserves per TURN with a flat $0.50 / $1.00 estimate; a single very expensive turn can still exceed it (the same estimate-accuracy caveat the admission reservation carries above).

Dollars and credits are counted apart (khz46) — which calls move a USD cap

T700 above split NOTIONAL from BILLABLE cost by each worker's static costModel. kxsw9 then made the real question answerable per call: src/core/billingClass.ts resolves the credential a dispatch would actually authenticate with into one of subscription, credits or free. khz46 is what the caps do with that answer.

The rule, one function (countsAgainstUsdCap in src/core/billingClass.ts). A call counts against CYBERINE_SPEND_CAP_USD and the daily USD cap only when its resolved billing class is credits — a per-token paid API, where the tokens really are dollars owed. A call whose class is subscription or free spends a plan pool, not money: it takes no hard-cap reservation, records no realized USD, and adds nothing to the daily cap. Its credit units go to a separate ledger instead, and are never converted to USD.

The bug this fixes: a paid Command Code judge or worker call was priced through creditLedger's USD rate table and posted to both caps, so an account that spends credits paused every dispatch on dollars it never owed — the same defect class T700(a) fixed for subscription CLI workers, one layer further in (judges and the hard cap, which T700 left on adapter.info.kind === 'loop').

Where the rule is applied — the same three points that convert a call into money:

SiteClass creditsClass subscription / free
Orchestrator.dispatchOne (cyberine run, MCP orchestrate, npm run demo, the autonomy loop's non-code gate)reserves, then records realized USDno reservation, no USD; records credits
WorkflowEngine.runWorker (npm run wf, isolate builds, sync-verify)reserves, then records realized USDno reservation, no USD; records credits
auxSpend.recordAuxSpend (every judge, the LLM router, the re-ranker, chat titling, the planners)prices in USD exactly as beforerecords credits, returns $0.00

A caller that passes no billing class keeps the pre-khz46 dollar behaviour byte-for-byte, which is what selftest:spend-cap and selftest:daily-cap-reservation assert.

The credits ledger. src/store/creditUnits.ts appends one line per credit-denominated call to ~/.cyberine/credits/credit-units.jsonl, day-bucketed and pruned to today's entries past CYBERINE_CREDIT_UNITS_PRUNE_BYTES (64 KB), mirroring the aux ledger. There is deliberately no cap here: Command Code's real ceiling is its rolling 5-hour / weekly window, already enforced by the headroom parking in src/core/commandCodeHeadroom.ts.

Run recipe.

cyberine cost              # a "credits" line under the two cap lines: units today, call count, per provider
cyberine cost <runId>      # a "credits" line for that run when it drew any
cyberine run "..."         # the run NOTE names every worker that spends no dollars

The run note (the usual NOTE: / [engine] NOTE: channel) gains one line next to the billing: lines kxsw9 already prints:

billing: loop-commandcode-build=subscription (endpoint), loop-commandcode-free-build=free (profile), loop-deepseek-build=credits (auth)
billing: loop-deepseek-build is credits via auth-mechanism -- DEEPSEEK_API_KEY present: DeepSeek bills the API key per call
billing: no USD cap for loop-commandcode-build=subscription, loop-commandcode-free-build=free -- not reserved against CYBERINE_SPEND_CAP_USD, not summed into the daily USD cap; loop-commandcode-build records credits instead (`cyberine cost`)

cyberine cost prints the credits line whether or not either cap is enabled, because credits are an account fact and the caps are not:

daily cap  $0.0000 / $10.0000  (0%)  remaining $10.0000, resets 18h 2m
hard cap   not set (CYBERINE_SPEND_CAP_USD unset)
credits    0.0326 today (estimated)  1 call — commandcode 0.0326; never counted against either USD cap

Extend recipe. A provider is credit-denominated when its prefix is listed in CREDIT_PROVIDERS in src/store/creditUnits.ts (today: commandcode). To add one, put its prefix there and give estimateCreditUnits a rate for it; a caller that knows the real figure passes units to recordCreditSpend and the record is then flagged reported rather than estimated. A worker of any other provider whose class is subscription or free (a claude seat, codex, local Ollama) simply records nothing — there are no credits to record, and it already added no dollars.

New environment variable.

VariableDefaultMeaning
CYBERINE_CREDIT_UNITS_PRUNE_BYTES65536 (64 KB)prune threshold for ~/.cyberine/credits/credit-units.jsonl; past it the file is rewritten down to today's entries, mirroring CYBERINE_AUX_SPEND_PRUNE_BYTES

Known Limitations and Deferred Work.

  • Command Code credits are ESTIMATED, always. /provider/v1/chat/completions carries no cost field and no x-ratelimit-* headers (docs/providers/command-code.md, VERIFIED), so no per-call credit figure is observable. Units are derived from token counts at COMMAND_CODE_CREDITS_PER_MTOK (0.0333 credits per 1M tokens, from the one VERIFIED measurement with token counts: 965,446 in + 13,655 out drew 0.0326 credits) and every such record carries estimated: true, rendered as (estimated). That measured run was 87-93% cache-served, so the rate under-states an uncached call. Use GET /alpha/billing/credits (what cyberine route prints) for the authoritative remaining balance; this ledger answers "how much did Cyberine draw today", not "what does the account hold".
  • Purchased Command Code credits resolve to class credits and therefore DO move the USD cap. That follows kxsw9's endpoint rule (included monthly pool left → subscription; only purchased credits left → credits) and the khz46 decision applied on top of it, so once the monthly pool is exhausted a paid call is priced in dollars again. Declare "billing": "subscription" on the provider entry in fleet.json to override it.
  • Credits are attributed to a run only on the WorkflowEngine path. The ambient AsyncLocalStorage sink is installed by WorkflowEngine.run, exactly like auxCostUsd; a judge or dispatch from a front-end that never opens a run (orchestrate() in cli.ts / mcp.ts) still lands in the day ledger but run.json has no creditUnits for it.
  • No cross-day report and no cap. The ledger is pruned to today, there is no --days N credits view, and nothing refuses a dispatch on credits; window parking remains the only credit-side guard.
  • run.creditUnits is never folded into totalCostUsd or billableCostUsd. A credit is not a currency; converting one would put a made-up dollar figure straight back into the caps this separation exists to keep it out of.