ContextQ Integration — Cyberine long-term memory on the Agent Memory Layer

Status: design spec (drives implementation). Grounded against ContextQ as it ships today (shared-context-server, agent layer migration 0043) and Cyberine memory waves 1-4.

This document maps Cyberine's file-based episodic memory onto ContextQ's now-shipped Agent Memory Layer, defines the vendor-independent MemoryProvider port Cyberine will introduce, and lays out a phased rollout. Every claim below cites a real symbol or file:line in one of the two repos.

• Cyberine memory internals: src/store/MemoryStore.ts, src/store/memoryRetrieval.ts, src/launchBg.ts, src/store/runTypes.ts, USAGE.md section "Long-term memory".
• ContextQ surface: src/routes/agent.ts, src/services/agent.service.ts, src/services/agent-boot.service.ts, src/db/schema.ts (the agent_* tables), mcp/src/tools.ts (the agent_* tools), src/plugins/auth.ts.

1. Capability-fit summary

Cyberine today owns a self-contained, file-based episodic memory subsystem. ContextQ ships a server-side, multi-tenant, embedding-backed Agent Memory Layer that is a near-superset of Cyberine's primitives plus capabilities Cyberine does not have yet (mid-run checkpoints, verifiable task trees, kNN-recalled lessons, LLM handoffs).

Net: ContextQ can back every Cyberine memory primitive and adds the four Cyberine is missing (checkpoint/resume, evidence-gated tasks, recalled lessons, handoff). The one structural mismatch Cyberine must bridge is the outcome model — Cyberine stores one flat outcome blob per run; ContextQ splits it into tasks + lessons + a handoff body.

2. Field-by-field mapping

The unit of correspondence: one Cyberine run (one MemoryRecord) ~ one ContextQ agent_session. Everything else hangs off that.

2.1 Identity / scoping

2.2 Run -> session

2.3 Lineage -> resume chain

Caveat: Cyberine lineage is a richer graph (4 distinct links) than ContextQ's single parent_session_id. The mapping keeps parent_session_id for the primary resume edge and parks the other three in metadata rather than forcing them onto one FK.

2.4 Outcome -> tasks + lessons + handoff

This is the structural split. MemoryRecord.outcome (MemoryStore.ts:33-38) is one blob; ContextQ wants it decomposed:

2.5 Retrieval (preamble) -> Boot Pack

ContextQ's boot is richer (it adds a dedicated handoff slot and embedding-recalled lessons), but the injected prompt shape stays an Cyberine responsibility: the adapter must serialize BootPack into the existing preamble envelope so stripMemoryPreamble (MemoryStore.ts:329) still round-trips and memory never re-ingests itself.

2.6 The one Cyberine gap and the one ContextQ caveat

• Cyberine gap — no mid-run checkpoint/resume. Cyberine writes a record only at terminal state (writeRecord early-returns unless state is terminal, MemoryStore.ts:346). There is no equivalent of agent_checkpoint / /resume. A crashed Cyberine run leaves no resumable state — the next run starts from the preamble only. ContextQ's agent_checkpoints (schema.ts:1400) + monotonic seq optimistic lock (agent.service.ts:404) is net-new capability Cyberine gains by adopting the layer (rollout phase 4).

• ContextQ caveats:

  1. Auth scope. Reads (agent_boot, agent_resume, get session/tasks) need requireScope('read'); writes (session start/end, checkpoint, task upsert/tick, lesson add, handoff) need requireScope('write') (agent.ts:78,120,213,...). A programmatic sk_live_ key therefore needs BOTH read and write scopes (or admin, which subsumes both, auth.ts:331). See section 3.
  2. Tenant/workspace must pre-exist. Sessions are tenant-scoped from the key; the workspace + project slugs must already exist in ContextQ — resolveWorkspaceId returns null for unknown slugs (agent.service.ts:61) and the session silently falls back to no workspace, which then breaks agent_handoff (it 400s with no_workspace, agent.ts:438). Cyberine must provision the workspace once, out of band.
  3. Search type-enum is narrower than create. POST /api/contexts accepts 9 types incl. synthesis/map/moc (contexts.ts:41), and ctx_save (MCP) advertises the same 9 (tools.ts:55). But POST /api/search still rejects anything beyond the original 6 — z.enum(['reference','feedback','project','incident','lesson','user']) (search.ts:12). Consequence for phase 1 (generic ctx_save/ctx_search): Cyberine can WRITE a record under any of the 6 search-visible types, but if it writes synthesis/handoff-shaped contexts it cannot later filter-search them by type. Stick to one of the 6 (recommend reference for run records, lesson for issues) until the search enum is widened. The dedicated agent_* endpoints are unaffected — they do not go through the search type-enum.

3. Auth / provisioning model

Cyberine authenticates to ContextQ as a programmatic client with a sk_live_ API key.

• The user mints the key manually in the ContextQ dashboard. The key must carry BOTH read and write scopes (or admin). Do NOT attempt to find, generate, or derive a key from code. requireScope('write') gates every write endpoint and requireScope('read') every read; a key missing a scope gets 403 scope_denied (auth.ts:356). The key is presented as Authorization: Bearer sk_live_... (auth.ts:131) and binds the request to its issuing tenant (auth.ts:201).

• Where Cyberine stores it: under .local/ (gitignored) per the repo convention — e.g. .local/contextq/contextq.env. Never inline a key in source or commit it (git-workflow.md NOT-authorized list forbids secret commits). Leave a breadcrumb if a key file moves.

• Env vars (new, all read by the ContextQ adapter):

  Env var	Purpose	Default
  CYBERINE_CONTEXTQ	master switch for the ContextQ backend; unset/0 keeps local files	(off)
  CYBERINE_MEMORY_BACKEND	selects the provider: local (default) | contextq | mcp	local
  CYBERINE_CONTEXTQ_URL	base URL of the ContextQ API (e.g. https://ctx.example.com)	(none)
  CYBERINE_CONTEXTQ_API_KEY	the sk_live_ write+read key	(none)
  CYBERINE_CONTEXTQ_WORKSPACE	workspace slug all Cyberine sessions/contexts scope to	(none)

  CYBERINE_MEMORY_BACKEND=contextq requires CYBERINE_CONTEXTQ_URL + CYBERINE_CONTEXTQ_API_KEY

  • CYBERINE_CONTEXTQ_WORKSPACE; on any missing value or transport error the provider factory falls back to local and logs (best-effort, mirroring MemoryStore's never-throw contract, MemoryStore.ts:480). The existing CYBERINE_MEMORY=0 master switch (MemoryStore.ts:317) still hard-disables all memory regardless of backend. The mcp backend reuses the same key via the ContextQ MCP server's CONTEXT_API_KEY / CONTEXT_API_URL envs (mcp/src/tools.ts:8,32).

4. The MemoryProvider port

Cyberine introduces a MemoryProvider interface so memory is backend-agnostic. The local file store is refactored behind it (LocalMemoryProvider wrapping today's MemoryStore functions); ContextQ is the FIRST alternate adapter, not a hardcoded dependency. Selection is config-only via CYBERINE_MEMORY_BACKEND.

Update 2026-09-22: the shipped seam differs from this spec. The local MemoryStore stays canonical, not refactored behind a MemoryProvider port; a backend instead implements the augmentation interface MemoryAugmentationBackend (src/store/memoryBackend.ts:29), chosen via selectMemoryBackend() (src/store/memoryBackend.ts:329). There is no MemoryProvider/LocalMemoryProvider symbol in source.

4.1 Required methods (every backend MUST implement)

These are exactly the surface launchBg.ts and the cyberine memory CLI consume today, so the local store satisfies them by definition:

interface MemoryProvider {
  // writes (terminal-state record)
  writeRecord(run: Run, cwd: string): Promise<MemoryRecord | null>;       // MemoryStore.ts:344
  // reads
  listRecords(cwd: string): Promise<MemoryRecord[]>;                      // MemoryStore.ts:487
  getRecord(cwd: string, runId: string): Promise<MemoryRecord | null>;    // MemoryStore.ts:503
  latestRecord(cwd: string): Promise<MemoryRecord | null>;               // MemoryStore.ts:513
  listOpenIssues(cwd: string): Promise<IssueEvent[]>;                     // MemoryStore.ts:544
  // retrieval -> prompt
  buildMemoryPreamble(                                                    // memoryRetrieval.ts:202
    cwd: string, query: string,
    opts?: { topK?: number; maxChars?: number; chatSessionId?: string; parentRunId?: string },
  ): Promise<string>;
}

Contract notes: all methods are best-effort (return null/[]/'', never throw into the caller) so a backend outage degrades to "no memory", never a failed run. The ContextQ adapter satisfies these by translating to/from agent_* calls + the Boot Pack (see the mapping in section 2) and re-rendering the MEMORY (...) envelope.

4.2 Optional capabilities (a backend MAY implement; callers feature-detect)

Only the ContextQ/MCP backends implement these initially; local returns not-supported and Cyberine keeps its current behavior. Exposed via an optional capabilities bag + narrow methods:

interface MemoryProviderCapabilities {
  boot?: boolean;          // server-assembled Boot Pack (agent_boot)
  checkpoint?: boolean;    // mid-run snapshot + resume (agent_checkpoint / /resume)
  tasks?: boolean;         // evidence-gated verifiable task tree (agent_task_*)
  lessons?: boolean;       // embedded, kNN-recalled lessons (agent_lesson_add)
  handoff?: boolean;       // LLM handoff document (agent_handoff)
}

Vendor-independence rule: Cyberine core (engine, launchBg, CLI) depends ONLY on the MemoryProvider interface. No agent_* symbol, no ContextQ URL, and no sk_live_ handling leaks above the adapter boundary — a future backend (e.g. a different memory server) drops in by implementing the same port.

5. Phased rollout

Each phase is independently shippable and degrades to the prior phase on failure. "Cyberine seam" names the file/function to change; "ContextQ" names the tool/endpoint.

Phase 1 — generic ctx_save / ctx_search (works against ContextQ today)

• What: define the MemoryProvider port; refactor today's MemoryStore calls behind LocalMemoryProvider; add a ContextqMemoryProvider that on writeRecord POSTs a single context (ctx_save -> POST /api/contexts, tools.ts:88) carrying the serialized MemoryRecord, and on buildMemoryPreamble runs ctx_search (POST /api/search, tools.ts:151) and renders hits into the existing envelope.
• Cyberine seam: src/store/MemoryStore.ts (writeRecord), src/store/memoryRetrieval.ts (buildMemoryPreamble), src/launchBg.ts:80 (the only preamble call site), plus a new src/store/MemoryProvider.ts + src/store/ContextqMemoryProvider.ts.

  Update 2026-09-22: shipped as src/store/memoryBackend.ts (ContextqBackend class) plus src/store/contextq.ts (HTTP wrappers); neither MemoryProvider.ts nor ContextqMemoryProvider.ts exists.

• ContextQ: POST /api/contexts, POST /api/search.
• Caveat: use type:"reference" for run records and type:"lesson" for issues — both are in the search-visible 6 (search.ts:12). Do NOT use synthesis/map/moc yet (not search-filterable, section 2.6.3). No agent layer needed — proves the transport, key, and workspace before adopting the richer surface.

Phase 2 — agent_boot for the preamble

• What: buildMemoryPreamble on the ContextQ backend calls agent_boot and renders the returned BootPack (resume + handoff + lessons + facts) into the MEMORY (...)/END MEMORY envelope instead of hand-rolling ctx_search.
• Cyberine seam: ContextqMemoryProvider.buildMemoryPreamble; the call site at launchBg.ts:80 is unchanged (still calls the port).
• ContextQ: agent_boot / POST /api/agent/boot (agent.ts:76, agent-boot.service.ts:255). Pass goal=finalTask, workspace=$CYBERINE_CONTEXTQ_WORKSPACE, token_budget from CYBERINE_MEMORY_MAX_TOKENS.

Phase 3 — runs -> agent_sessions + lineage -> resume chains

• What: map each Cyberine run to a session: agent_session_start at dispatch, agent_session_end at terminal state with the status mapping (section 2.2). Thread RunLineage.parentRunId -> parent_session_id; park previousRunId/chatSessionId/ rootRunId in metadata (section 2.3).
• Cyberine seam: src/launchBg.ts (session start at dispatch, stash session id keyed by runId), src/store/MemoryStore.ts writeRecord (session end). Resolve parent session id from the parent run's stored metadata.cyberineRunId.
• ContextQ: agent_session_start (agent.ts:118), agent_session_end (agent.ts:173).

Phase 4 — agent_checkpoint / agent_resume mid-run (closes the Cyberine gap)

• What: add periodic checkpoints during long runs (the capability Cyberine lacks today). Snapshot working state after each step; on restart, agent_resume reloads the latest checkpoint + open tasks.
• Cyberine seam: the WorkflowEngine step loop (src/engine/) emits checkpoint(sessionId, state, summary) per completed step; a new resume path reads agent_resume on relaunch. This is net-new Cyberine behavior, gated on capabilities.checkpoint.
• ContextQ: agent_checkpoint / POST /api/agent/sessions/:id/checkpoint (agent.ts:211), agent_resume / GET /api/agent/sessions/:id/resume (agent.ts:239). Mind the monotonic seq optimistic lock (agent.service.ts:404) under concurrent writers.

Phase 5 — agent_task / agent_lesson / agent_handoff writers

• What: decompose outcome per section 2.4 — accomplished -> verified tasks (with synthesized evidence), next -> pending tasks, issues -> lessons (embedded for recall) or blocked tasks; then agent_handoff at session end to synthesize the workspace handoff doc.
• Cyberine seam: a new src/store/outcomeToContextq.ts mapping helper invoked from writeRecord on the ContextQ backend; gated on capabilities.tasks/lessons/handoff.

  Update 2026-09-22: shipped as src/store/outcomeMapping.ts (outcomeToTaskPayloads/outcomeToLessonPayloads), invoked from ContextqBackend.pushSession (src/store/memoryBackend.ts:133-136), not from writeRecord.

• ContextQ: agent_task_upsert (agent.ts:282), agent_task_tick (agent.ts:336, evidence-gated), agent_lesson_add (agent.ts:372), agent_handoff (agent.ts:413, needs an LLM provider configured server-side or it 503s, agent.ts:467).

6. Gaps vs the design draft

Differences between what the earlier agent-memory-layer design draft predicted and what ContextQ actually ships, plus the Cyberine-side mismatches to plan around:

1. Search type-enum did NOT get widened to match create. The draft surface treats context types uniformly, but in the shipped code POST /api/contexts and ctx_save accept 9 types (incl. synthesis/map/moc, contexts.ts:41, tools.ts:55) while POST /api/search still hard-rejects anything beyond the original 6 (search.ts:12). Phase 1 must avoid the extra types or run records become un-search-filterable. Flag for the ContextQ team: widen searchSchema.type to the full set.

2. Embedding dimension is 768, not 384. The Drizzle custom type is named vector384 but its SQL type is vector(768) (schema.ts:28-32) and lessons embed at 768 (agent.service.ts). Any Cyberine-side embedding parity work must target 768-dim; the name is misleading.

3. Lessons are written without the vector through Drizzle. addLesson inserts the row first, then writes the embedding via a raw UPDATE ... ::vector SQL (agent.service.ts:483-501) because the custom type has no toDriver serializer. If embeddings are disabled server-side, lessons store with a null vector and boot falls back to recency ordering (agent.service.ts:572) — Cyberine should not assume kNN recall is always available.

4. Single parent_session_id vs Cyberine's 4-way lineage. ContextQ ships exactly one self-referential session link (schema.ts:1342); Cyberine's RunLineage has four distinct edges (runTypes.ts:22). The mapping parks three of them in metadata — there is no first-class home for previousRunId/chatSessionId/rootRunId.

5. agent_handoff is not self-contained — it needs a server LLM + a workspace. It wraps the dream-handoff generator and 503s when no LLM provider is configured (agent.ts:467), 400s (no_workspace) when neither the body nor the session carries a workspace (agent.ts:438). Phase 5 must provision both; it is not a pure data write.

6. No first-class column for runId, workflowName, workers, or verdict. All ride in agent_sessions.metadata jsonb (schema.ts:1349). The runId-as-metadata join key is the load-bearing link between Cyberine's UUID world and ContextQ's serial ids — get it wrong and resume-chain resolution (phase 3) breaks.