autonomous-ai / engine-ollama

Ollama

Official docs, read 2026-09-29 (source files in github.com/ollama/ollama/tree/main/docs):
FAQ · Context length ·
OpenAI compatibility ·
Thinking · CLI ·
macOS · Linux · Import ·
API reference
Tested: Ollama 0.34.4 (brew install ollama), MacBook Pro M1 Pro 32 GB, macOS 26.6, 2026-09-29.
Tags: [doc] official page above, [run] seen on the tested machine, [?] unverified.

When to use it

• A model in Ollama's store (fleet models → START WITH ollama): Ollama runs it. It downloaded the
  file and ships its own engine for new architectures, which Grid's engine can refuse [?].

  • Answering: adopt it as it is; it is the person's app, so never restart or reconfigure it.

  • (start it): start one yourself (below). An Ollama that is off is the normal case, not a reason
    to switch engines [run].

• Ollama not installed: its blobs are GGUF files, so fleet models names Grid's llama.cpp — link the
  blob into ~/.grid/models and join --serve [run].

• A GGUF from another app enters Ollama only by ollama create (a Modelfile with FROM <file>), and
  that copies the whole file into Ollama's store (+624 MB for a 640 MB GGUF) [run]. So never do it
  unasked; when the person wants Ollama and it has nothing suitable, offer it with the size it adds on
  disk, beside the no-copy choice (the file with its own app). Never ollama pull without the
  go-ahead: it downloads.

MLX inside Ollama (Apple silicon)

Since 0.19 Ollama has its own MLX engine on Apple silicon, announced as a preview on 2026-03-30
(blog) [doc]. It runs on MLX only the architectures registered in its
source — read the list live, never from memory:
https://raw.githubusercontent.com/ollama/ollama/main/mlxrunner/model/architectures/architectures.go
(one import per architecture folder under mlxrunner/model/) [doc]. Compare with model_type from
fleet model-facts. Everything else keeps running on Ollama's GGML engine. The announced models are
Ollama-library tags in their own quantization, and the post asks for more than 32 GB of unified memory [doc];
importing other MLX models is not yet documented [?].

Where its models live / how to list them

• Default dir: macOS ~/.ollama/models, Linux service /usr/share/ollama/.ollama/models, Windows
  C:\Users\%username%\.ollama\models; OLLAMA_MODELS moves it [doc].

• Layout: manifests/registry.ollama.ai/library/<model>/<tag> (a JSON file) and blobs/sha256-<hex>;
  the manifest layer application/vnd.ollama.image.model names the weights blob, a GGUF [run]. Model id is
  <model>:<tag>; other namespaces are <user>/<model>:<tag> [?].

• fleet models already reads the manifests: entries with source: ollama, the blob path, and
  alsoAt when another app links the same file [run].

• From a running server: GET /api/tags (downloaded), GET /api/ps (loaded, with context_length and
  size_vram) [doc], GET /v1/models (owned_by is the Ollama user, library by default) [doc][run].
  POST /api/show {"model":ID} lists capabilities (completion, tools, thinking, vision) [doc].

Installed? Running?

• Installed: command -v ollama or /Applications/Ollama.app. ollama --version with no server prints
  "Warning: could not connect to a running Ollama instance" and the client version [run].

• The macOS app starts the server at login (a login item) [doc]; brew install ollama starts nothing [run];
  the Linux installer creates a systemd service ollama [doc].

• Running: fleet models → engines[] with kind: ollama. Default bind is 127.0.0.1:11434 [doc].

Start an already-downloaded model (Ollama not running, or running with too small a window)

Port P from outside machine.listeningPorts, and never 11434: that is the Ollama app's own, and it must
still start when the person opens it [run]. The port is set only through OLLAMA_HOST (no --port) [doc]:

"$GRIDFLEET" serve ollama-P --env OLLAMAHOST=127.0.0.1:P --env OLLAMACONTEXTLENGTH=65536 -- ollama serve

(fleet serve keeps it alive after your shell returns; log run/ollama-P.log, PID run/ollama-P.pid.)

• Foreground process; the log says Listening on 127.0.0.1:P (version …) when ready [run].
• Every other ollama command must carry the same OLLAMA_HOST, or it talks to 11434 [run].
• Nothing downloads unless you pull; the model loads on its first request [run].

• Context: the default depends on GPU memory — 4K below 24 GiB, 32K at 24–48 GiB, 256K at 48 GiB or
  more [doc] (the FAQ still says 4096 [doc]). Agents need at least 64000 [doc], and every model here is
  used by an agent: always set OLLAMA_CONTEXT_LENGTH to 65536 or more. The OpenAI API cannot set
  context per request [doc].

Ready means

Run "$GRID_FLEET" verify --at http://127.0.0.1:P/v1 --model <model>:<tag> --kind ollama — it performs
these checks with deadlines and prints each one. What it checks:

1. The PID is alive. 2. GET / answers "Ollama is running" [run]. 3. GET /api/version → 200 [run].

2. One bounded request through /v1/chat/completions with the model id, max_tokens 16 and
   "reasoning_effort":"none" for a thinking model → non-empty content [doc][run].

3. GET /api/ps shows the model with context_length ≥ 65536 [doc]. FAIL context means this Ollama
   runs every model with that window [run]: leave the grid, then leave their Ollama as it is and start a
   second one with OLLAMA_CONTEXT_LENGTH (above) [run].

Join Harness Compute

"$GRID_FLEET" run -- join GRID --at http://127.0.0.1:P/v1 -m <model>:<tag> --advertise-as ALIAS

/v1 is required: without it /models and /chat/completions answer 404, and Grid's capability probe
records JSON output as unsupported without any error [run]. Grid's own detector finds Ollama only on
11434 [run].

Tool calls, JSON output, thinking

• /v1/chat/completions supports tools, response_format and reasoning_effort [doc].
  reasoning_effort: "none" asks for no thinking; the native API uses "think": false [doc].

• A model's thinking values and default: /api/show → thinking.values, thinking.default [doc].
• Tool support is per model: check capabilities contains tools before offering it for coding [doc].

Memory and speed knobs (server environment, apply to all models)

brew services start ollama sets OLLAMA_FLASH_ATTENTION=1 and OLLAMA_KV_CACHE_TYPE=q8_0 [run: brew caveat].
ollama ps → PROCESSOR must read 100% GPU; a CPU/GPU split is slow [doc].

Stop

• A server you started: "$GRID_FLEET" stop ollama-P.
• Unload one model but keep the server: OLLAMA_HOST=… ollama stop <model> [doc].
• The person's app or service: ask first. macOS app: quit from the menu bar; Linux: sudo systemctl stop ollama [doc].

Known failures → what to do