Ollama (local models)

• Kind: local inference server; two API surfaces — OpenAI-compatible Chat Completions (/v1) and native REST (/api)
• Official docs: https://docs.ollama.com (API reference: https://github.com/ollama/ollama/blob/main/docs/api.md)
• Cyberine adapter: LoopAgentAdapter via the OpenAI-compat path — wired 2026-06-07 as loop-ollama (read-only) and the opt-in build worker loop-ollama-build (env opt-in: any OLLAMA_* set; static bearer ollama; thinking via reasoning delta handled in the adapter).
• Use case: free, private, offline fleet worker; also local embeddings for the memory subsystem (docs/design/long-term-memory.md).
• Researched/verified: 2026-06-07 — confidence high (deep-research with adversarial claim verification + live empirical tests against local Ollama v0.30.6 with qwen3:0.6b on this machine; empirical results below are version-specific).

Base URL & auth

OpenAI-compat base	http://localhost:11434/v1
Native base	http://localhost:11434/api
Auth header	none required; compat path accepts any Authorization: Bearer <anything> (use static 'ollama')
Keys	none — local server (ollama serve, or the desktop app)

The two integration paths

Path A — OpenAI-compat (/v1) — RECOMMENDED for Cyberine

Drop-in for the existing LoopAgentAdapter (openai SDK): swap baseURL, static apiKey. Empirically verified on v0.30.6:

• POST /v1/chat/completions with stream:true + tools works: tool_calls arrive in an SSE chunk in OpenAI shape — {id, type:"function", function:{name, arguments:"<json-string>"}} — atomically (one chunk), finish_reason:"tool_calls". The whole Cyberine tool loop runs unchanged.
• stream_options.include_usage works: final chunk before [DONE] carries usage{prompt_tokens, completion_tokens} (added in ollama PR #6784, Dec 2024).
• Thinking models stream the trace as delta.reasoning — NOT reasoning_content (see Gotchas).
• GET /v1/models → 200, lists pulled models (availability probe).
• POST /v1/embeddings available for the memory subsystem.

Path B — native (/api/chat) — only if these are needed

• NDJSON wire format (not SSE): one JSON object per line, done:false chunks then a final done:true chunk with done_reason + stats (prompt_eval_count, eval_count, total_duration, load_duration, eval_duration). Verified 3-0 + empirically.
• Streaming + tools: two-chunk pattern — done:false chunk carries the complete tool_calls object (never streamed token-by-token; reliable since ollama PR #10415, merged 2025-05-23). Empirical: native arguments come as a JSON object ({"city":"Tokyo"}) with nesting {id, function:{index, name, arguments}} — NOT the OpenAI string convention; a native adapter must convert.
• Thinking trace in message.thinking (chat) / thinking (generate); "think": false request field disables reasoning (useful to force tool behavior on small thinking models — empirically needed for qwen3:0.6b).
• options.num_ctx per-request WORKS here (and only here — see Gotchas).
• Model management: GET /api/tags (pulled models), POST /api/show (model info), POST /api/pull, GET /api/ps (loaded runners incl. effective context_length + VRAM), GET /api/version.
• Mid-stream errors arrive as an {"error": ...} NDJSON object AFTER HTTP 200 is committed — a native consumer must check every chunk for an error key.

Verdict: Path A for chat/tool workers (zero new adapter code). Path B only for: per-request num_ctx, message.thinking access, model management/diagnostics (/api/tags, /api/ps, /api/pull). Cyberine needs B solely for the availability probe and (optionally) context-size pinning.

Endpoints

Method + path	Purpose
POST /v1/chat/completions	OpenAI-compat chat — what LoopAgentAdapter calls
GET /v1/models	List pulled models (compat availability probe)
POST /v1/embeddings	OpenAI-compat embeddings
POST /api/chat	Native chat (NDJSON, options, think)
GET /api/tags	Pulled models + sizes (native availability probe)
POST /api/show	Model details (404 if not pulled)
GET /api/ps	Loaded runners: effective context_length, VRAM, expires_at
POST /api/pull	Pull a model programmatically
GET /api/version	Server version

Models — what actually works as an Cyberine worker

No fixed catalog — whatever is pulled locally. The tag tools on https://ollama.com/search?c=tools is a catalog hint only; whether a model emits structured tool_calls (which the Cyberine loop requires) is NOT guaranteed by the tag and must be verified per model.

Verified on this repo 2026-06-07 (Ollama v0.30.6; "read package.json via the read_file tool, report name") — tested both at the raw /v1/chat/completions layer and through LoopAgentAdapter's full tool loop:

Model	Size (Q4)	Structured tool_calls?	Full-loop time	Verdict
llama3.1:8b	4.9GB	YES, finish_reason:tool_calls	25s	DEFAULT — fast, no thinking overhead, reliable tools
qwen3:8b	5.2GB	YES (cleaner args)	99s	Stronger reasoning, but ALWAYS thinks on the compat path (no think toggle there) → ~4x slower. Use for hard tasks only.
qwen2.5-coder:7b	4.7GB	NO — emits the call as raw JSON in content, never as tool_calls	n/a	Unusable as a worker. Good at code completion, but Ollama's template does not surface its tool calls; the Cyberine loop sees a plain text answer.
qwen3:0.6b	522MB	YES, but reasons instead of acting	n/a	Wire-format testing only; too small for real work.
nomic-embed-text	274MB	n/a (embeddings)	n/a	For the memory subsystem (CYBERINE_MEMORY_EMBEDDINGS=ollama).

Takeaway for adopters: default OLLAMA_MODEL=llama3.1:8b. Pick a model that emits structured tool_calls — a "tools"-tagged or even "coder" model is NOT a guarantee (qwen2.5-coder fails). When trying a new model, test it the same way: curl /v1/chat/completions with a tools array and check the response has a tool_calls field (not a JSON blob in content). An 8B Q4 model needs roughly 6GB RAM loaded; size the model to the host's FREE memory, not its total — overcommitting drives the machine into swap.

Key request params (compat path)

• model, messages, tools, stream, stream_options.include_usage — all as OpenAI.
• max_tokens, temperature, top_p — honored.
• options (native-style extra body) — silently ignored on /v1 (empirically: options.num_ctx: 8192 → model still loaded at its default 40960). Do not rely on it.

Notable features

• Free + private: no API key, no egress; cost class 1 in fleet terms.
• Tool calling: OpenAI-identical schema on both paths (type:"function", function:{name, description, parameters}) — portable definitions (verified 3-0).
• Thinking models: native think toggle; trace via delta.reasoning (compat) / message.thinking (native).
• Structured outputs: native format field accepts a JSON schema; compat response_format exists but schema-level support was NOT confirmed by verification — treat as unverified.
• keep_alive: runner unload timeout (default ~5m; /api/ps expires_at shows it). First request after unload pays a load penalty (empirical: ~0.7-1.2s for 0.6b; tens of seconds for large models) — relevant for OLLAMA_TIMEOUT style budgets.

Gotchas

• CRITICAL — thinking field name differs from every other Cyberine provider: the compat shim emits delta.reasoning, while LoopAgentAdapter reads delta.reasoning_content (DeepSeek/Kimi convention). Without a 1-line adapter tweak (reasoning_content ?? reasoning), thinking-model output streams as silence until the answer. (ollama issues #15288/#15293; empirically confirmed.)
• CRITICAL — context length cannot be set per-request on the compat path. options.num_ctx works only on /api/chat. The effective default depends on server config (OLLAMA_CONTEXT_LENGTH) and model — empirically qwen3:0.6b loaded at 40960 on v0.30.6, but other models/installs may load as low as 4096 (old folklore says 2048; refuted for current versions). For agent loop work pin the server-side env (OLLAMA_CONTEXT_LENGTH=16384 or higher) or pre-load via a native call with options.num_ctx. Check the effective value with GET /api/ps → context_length.
• Missing model = HTTP 404 on both paths (empirical, v0.30.6): compat body {"error":{"message":"model '<m>' not found","type":"not_found_error"}}, native {"error":"model '<m>' not found"}. Catch and surface "run ollama pull <m>" instead of retrying.
• Small thinking models may think instead of calling tools — qwen3:0.6b empirically burned its turn on thinking and never emitted a tool call until "think": false (native) was set. The compat path has no think toggle; prefer non-thinking models (or large enough thinking models) for tool workers.
• Tool-call arguments type differs by path: compat = JSON string (OpenAI convention), native = JSON object with different nesting. Another reason to stay on compat for the loop.
• Mid-stream NDJSON errors after HTTP 200 (native only) — check every chunk for error.
• Usage on compat path lacks cost (always local/free) — costUsd stays unset; token counts come from include_usage.

Pricing

Free (local compute). Cost class 1; speed/capability bounded by host hardware.

Cyberine integration notes (wired 2026-06-07, dogfood run f64b7856)

Implemented per the DeepSeek-direct template (buildAdapters.ts); verified live: loop-ollama answered pong via local qwen3:0.6b.

• config.ts: add ollamaBaseUrl (OLLAMA_BASE_URL, default http://localhost:11434/v1) and ollamaModel (OLLAMA_MODEL, default llama3.1:8b); add ollamaDirect: boolean to AdapterAvailability.
• Availability: isConfigured() is synchronous; an HTTP probe would be the first async check in it. Decision: gate on explicit env opt-in (any OLLAMA_* set ⇒ register, mirroring the gatewayOptIn pattern) and let the adapter surface the 404/ECONNREFUSED as a clean worker error; optionally add a fast async probe (GET /v1/models, 500ms timeout, then /api/tags includes the model) inside buildAdapters where async is available.
• buildAdapters.ts: register loop-ollama (read-only tools) when availability.ollamaDirect; add case 'loop-ollama-build' to the build-fleet switch via makeLoopImplementer({ id: 'loop-ollama-build', provider: 'ollama:<model>', model: config.ollamaModel, baseURL: config.ollamaBaseUrl, apiKey: 'ollama' }).
• LoopAgentAdapter.ts: 1-line thinking fix — read delta.reasoning_content ?? delta.reasoning (cast). No replayReasoning needed (Ollama does not require reasoning echo; needsReasoningReplay stays Kimi-only).
• workerProfiles.ts: loop-ollama-build → kind: 'key-loop', tier: 'flash', costClass: 1, speed: 'medium', strengths ['free','local','private'], weaknesses ['capability-ceiling','hardware-dependent'].
• .env.example + docs/providers/README.md row + USAGE.md recipe (per feature-docs.md).
• Memory synergy: CYBERINE_MEMORY_EMBEDDINGS=ollama (design doc) uses POST /v1/embeddings with nomic-embed-text.

Sources

• https://docs.ollama.com/api/openai-compatibility
• https://github.com/ollama/ollama/blob/main/docs/api.md
• https://ollama.com/blog/streaming-tool (May 2025)
• https://github.com/ollama/ollama/pull/10415 (streaming tool calls, merged 2025-05-23)
• https://github.com/ollama/ollama/pull/6784 (stream_options.include_usage, merged Dec 2024)
• https://docs.ollama.com/capabilities/thinking ; issues #15288, #15293 (compat reasoning field)
• https://docs.ollama.com/api/errors (mid-stream NDJSON errors)
• Empirical tests 2026-06-07, Ollama v0.30.6 + qwen3:0.6b, this machine (missing-model 404s; compat stream+tools+usage; delta.reasoning; options.num_ctx compat-ignored/native-honored via /api/ps; native 2-chunk tool stream; think:false)