cyberine
UsageRun recipes

Blind candidate ids in the LLM and harvest judges

llm-judge (and DiffJudge's tie-break, which runs it) shows candidates as anonymous ids A, B, C in a deterministic shuffle seeded by the task id and prompt, never as the worker's agent id or in adapter order, so a worker's brand or its slot in the fleet cannot tilt the pick. The judge answers with an id; the verdict maps it back to the real result and its rationale ends with (winner: <agentId>; blind ids A=<agentId>, B=...) for the audit trail. The harvest judge labels lanes Lane 1, Lane 2 with no agent id in the harvester's prompt, and its rationale maps each label back with the lane's seat.

npm run selftest:judge-fallback         # LlmJudge rules on a blind id
npm run selftest:harvest-judge          # harvester prompt carries no agent id

Known Limitations and Deferred Work: the shuffle is per task, so repeated judging of the same task always sees the same order; it removes adapter-order bias, not position bias within one order. A judge reply with an integer winner is still accepted and read as a position in the shuffled list. A candidate's own output can still name its worker; that text is fenced as untrusted but not rewritten. AdversarialJudge and SecurityJudge already kept agent ids out of their prompts and are unchanged.