Connecting a Model

AGENTOS DOCS / GETTING STARTED

AgentOS works with the model you choose. Run one locally for full privacy, use a server on your network, or connect a hosted API.

Supported providers

Provider	Default host	Good for
Ollama (default)	`http://localhost:11434`	The easiest way to run open models locally or on a GPU box on your LAN
llama.cpp (`llama-server`)	`http://localhost:8080`	Lean, fine-tuned local inference with precise context control
OpenAI-compatible	your provider's base URL	OpenAI, vLLM, LM Studio and any other `/v1/chat/completions` API

// PRIVACY The model receives your messages, workspace context and anything the agent reads. If the model runs on hardware you control, that data never leaves your network. If you use a hosted provider, it is sent to that provider. See the AgentOS Privacy Policy.

Using Ollama

Install Ollama and start it: ollama serve
Pull a model that supports tool calling, for example: ollama pull qwen3
In AgentOS, open Workspace Settings → Model, select Ollama, and pick the model.

The local default (http://localhost:11434) needs no extra configuration.

A remote Ollama server

To use Ollama on another machine (a GPU workstation, a cloud VM or https://ollama.com), set Ollama Host in Workspace Settings, or edit .agentos/MODEL.md:

Host: http://192.168.1.50:11434

To make it the default for new workspaces, set it in Global Settings or in ~/.agentos/agentos.json:

{ "ollamaHost": "http://192.168.1.50:11434" }

Using llama.cpp

Start llama-server with your model, then choose llama.cpp as the provider and enter the server URL. AgentOS reads the loaded model's real context size from the server.

Using an OpenAI-compatible API

Choose OpenAI-compatible as the provider in Workspace Settings.
Enter the base URL (for example https://api.openai.com/v1 or http://localhost:1234/v1 for LM Studio).
Enter your API key, then pick a model from the list.

For reasoning models, you can also set the reasoning effort.

Switching models from chat

/model            show the current model
/model switch     list available models and pick one

Tool calling

AgentOS detects whether a model supports native tool calling and uses it automatically. Models without it still work, through AgentOS's built-in tool-call format. Larger, tool-capable models give noticeably better results on multi-step tasks.

Context size

Set the context window in Workspace Settings (MODEL.md). Ollama and llama.cpp can usually detect it on their own. For other providers, enter it so AgentOS knows when to compact memory. See Memory & Context.