CLI reaper — cyberine reap [--dry-run]
cyberine reap is the command-line companion to the TUI's stale-run detection. It sweeps the global
registry for runs whose engine subprocess is alive but has produced no events.jsonl progress for
longer than CYBERINE_STUCK_IDLE_MS (default 10 min), or that have exceeded the hard wall-clock cap
CYBERINE_STUCK_WALL_MS (default 2 hours). Runs with genuine recent activity are never touched.
cyberine reap --dry-run # list stuck runs without killing anything
cyberine reap # cooperative teardown: SIGTERM -> 5s grace -> SIGKILL; marks state cancelledTeardown mirrors Claude Code's cooperative shutdown: the engine gets SIGTERM first, then (after
CYBERINE_STUCK_GRACE_MS, default 5 s) SIGKILL if it is still alive. After the process is gone, the
reaper marks the run cancelled with a killed:stuck — <reason> detail in run.json and
tombstones it in the global registry so cyberine agents stops showing a phantom-busy worker.
A reused PID is never signalled (kcwzb). Before any signal the reaper checks that the recorded enginePid still belongs to the engine: the process must answer kill(0) AND have started no later than the run's last write (updatedAt, 2 s slack), read with ps -o lstart=. A run whose PID now belongs to a process that started after that write is marked cancelled with no signal sent. Crash-resume (scanAndResumeDeadRuns) and cyberine wait use the same check, so a reused PID no longer keeps a dead run "running". On 2026-09-28 the old check signalled two unrelated processes that had inherited the PIDs of 4.6-day-old autopilot runs. npm run selftest:reaper-pid-identity (a security smoke) pins it with a live unrelated sleeper.
Known Limitations and Deferred Work: where ps cannot be read (Windows) the check degrades to the plain kill(0) probe. A command that writes run.json after the engine died and after the PID was reused moves updatedAt forward and defeats the check. The TUI's zombie detection (src/tui/zombie.ts) still uses the plain probe, so it may show such a run as live.
Env knobs:
| Variable | Default | Purpose |
|---|---|---|
CYBERINE_STUCK_IDLE_MS | 600000 (10 min) | idle timeout — how long a live process may emit no events.jsonl progress before being considered stuck |
CYBERINE_STUCK_WALL_MS | 7200000 (2 h) | wall-clock cap — a run older than this is always reaped regardless of recent activity |
CYBERINE_STUCK_GRACE_MS | 5000 (5 s) | grace window between SIGTERM and SIGKILL |
Silence is not a stall: what the idle clock actually sees (kfg1n)
Audited 2026-09-07. Three killers can reach a headless CLI worker on silence alone: this reaper
(events.jsonl mtime, 10 min), the autopilot stall watchdog (--stall-timeout, 20 min of no stdout
byte AND no busy descendant), and nothing in CliAgentAdapter (its timeoutMs is wall-clock).
Measured on real Claude Code transcripts, an AUTOMATIC context compaction is 103-133 s of stream
silence before the compact_boundary event (two auto boundaries at ~970k pre-tokens; a manual one at
437k took 133 s) -- there is no compaction-START event to exempt on, so the defaults stay 4-9x
clear of it by size, not by detection. What was broken: while the model generates, stream-json emits
system/thinking_tokens about once a second, and until kfg1n that mapped to nothing, so a long
thinking phase wrote no events.jsonl line and looked to the idle clock exactly like a hang. The
adapter now emits a heartbeat progress event once per CLAUDE_THINKING_HEARTBEAT_TOKENS (500)
estimated thinking tokens and once per compact_boundary (context compacted (auto, N tokens before)), which the engine records as a progress event and the reaper reads as activity.
Known Limitations and Deferred Work: do not lower CYBERINE_STUCK_IDLE_MS below ~3 minutes for a
fleet with claude workers -- a compaction would be reaped mid-flight and there is no event that
could exempt it; codex and muse streams were not measured for an equivalent quiet phase; the
autopilot watchdog needs no change (any stdout byte, thinking_tokens included, already counts).
npm run selftest:cli-worker-liveness pins the mapping.
To stop programmatically (e.g. from another tool) without the TUI, drop the same file:
printf '%s' '{"action":"stop","target":{"type":"run"}}' \
> .local/runs/<runId>/control.json
# or target one worker:
# {"action":"stop","target":{"type":"worker","agentId":"opencode"}}Stale / zombie runs
If the engine process is killed mid-run (crash, Ctrl-C, kill), its run.json is left stuck in running with nothing driving it. The TUI detects this: the engine records its PID (enginePid) when it...
Always-on operations — graceful drain, health, queue intake, maintenance
The API-first 24/7 model keeps cyberine web supervised and pokes it (or the CLI) with thin external triggers instead of running a fat internal daemon. The pieces below harden that path.