Contrastive reflection into a lesson library — cyberine lessons (k79pw)

Reflect on one completed step only when the record contains a passed worker and a failed worker with the same brief. A judge selecting one of several successful workers is not a failure signal. The filter makes no model call for identical or unknown outcomes, running steps, or chunk/lane workers with potentially different briefs.

Run recipe:

cyberine lessons reflect <runId> --step implement --dry-run --json
cyberine lessons reflect <runId> --step implement --json
cyberine lessons list --json
cyberine lessons show <lessonId> --json

Omit --step to reflect once per eligible step. --dry-run builds the evidence directory and prints eligibility, file names, byte count and missing harness fingerprints without calling a model. CYBERINE_RUNS_DIR selects the input run root; otherwise the command reads .local/runs/ under the current repo. Run ids are full ids, not prefixes. Run the command from the repo that owns the lesson library.

Evidence is written once under .local/reflections/<runId>-<step>-<unique-id>/: the brief, verdict and recorded verification details, worker outputs/diffs, raw transcripts and trajectories rendered as [ACTION], [OBSERVATION] and [ERROR], harness fingerprints and the current lesson INDEX. Secrets are redacted and all evidence is fenced as untrusted text. Absent verification exit codes and old runs without fingerprints stay unknown. A CLI worker (a Claude seat, codex, muse) keeps no transcript, so its recorded output stands in as its evidence file, <step>__<worker>.output; only loop workers carry .transcript.jsonl and a rendered trajectory. A participant with neither a transcript nor any output blocks reflection instead of inventing evidence.

The model returns behaviour rows with behaviour, worker, effect, rating (helped | hurt | no observed effect | confounded) and transcript citations, plus either one lesson or NO_ENTRY. Adoption is not proof; model differences and simultaneous actions can confound a contrast. A lesson contains cue, lesson, target, evidence and numeric confidence in 0..1. Each evidence item is {worker, file, quote}; file is the exact transcript or output basename and quote must literally occur in that raw file, including any JSON escapes. Lesson citations must cover both the passed and failed workers.

Program gates run before storing a lesson: required fields, allowed targets rules | skill | brief-note, at most 600 lesson characters, literal transcript citations, a secret scan, and rejection of proposals about invariants, permissions, credentials, authentication or sandboxing. Invalid output reports the failed gates and exits 1. Valid behaviour rows are saved as fenced text in reflection.json beside the evidence; NO_ENTRY creates no lesson library entry.

Lesson file format: .cyberine/lessons/<id>.md is write-once Markdown with a JSON object as YAML-compatible frontmatter between --- lines, followed by the lesson text. Fields are id, status (candidate | active | retired), cue, lesson, target, confidence, evidence, support, contradict, source and created. source contains run, step, workers and harness, a map from worker id to its recorded fingerprint or null. New lessons always start as candidate with support 1 and contradict 0.

Matching cues are normalized by Unicode normalization, case and whitespace. Repeated cues append support/contradict observations to observations.jsonl instead of creating another lesson file; the same run/step is counted only once. The model may return assessment: "contradict" when evidence refutes an existing cue. INDEX.md is regenerated, and list/show compute current counts from the journal. Concurrent writers fail clearly on a directory lock rather than overwriting each other. The library is untracked; the existing skill extraction and clustering remain separate.

Graded input from outside a run: reflectOutcomes() in src/store/reflection.ts takes a brief, a grader description and per-worker {worker, outcome: passed|failed, basis, evidence} rows from any program oracle (the replay bench's hidden smoke is the first caller, k3kts) and runs the same evidence directory, prompt, gates and library write; worker ids must be distinct and the rows must contain both a pass and a fail.

Extend recipe: add routing.reflect to fleet.json as a non-empty list of worker ids in precedence order, using the same parsing and validation as routing.judges. Reflection tries that chain, then routing.judges, then the default judge backends, while respecting enabled workers and reusing the provider-neutral completion backend and Claude CLI transport. The default reflection model is the alias opus on a Claude CLI backend (any other backend keeps its own model); set CYBERINE_REFLECT_MODEL to a model id supported by the selected transports. It reuses existing fleet credentials and billing; a non-dry reflection can spend quota.

Known Limitations and Deferred Work. Reflection is on demand; there is no automatic post-run trigger. Lesson injection is OFF and no worker prompt changes. The later measurement task kp1mv owns activation, injected lesson ids in run.json, benefit measurement and feedback from actual injection. The reader can project an already-active lesson as retired when contradict exceeds support, but this command neither activates lessons nor measures injection. Existing records do not expose every candidate's typecheck outcome, so unsupported outcomes remain unknown. Context size and model availability depend on the chosen backend; no live provider verification is implied. A stale .write-lock after a crashed writer needs operator inspection.