Provider-Agnostic by Design

What provider agnosticism actually means

Provider agnosticism means the core agent logic doesn't depend on a specific LLM API, a specific model family, or a specific company's inference service. Instead, the agent treats "call a language model" as an abstract operation—you pass in a prompt and parameters, you get back a text response and any structured output (like tool calls). The specific provider that fulfills that request is swappable.

In practice, this looks like: OpenCode has an internal model interface (roughly: "given this prompt, call a model and return the response"). You then have concrete implementations of that interface—one for Anthropic's API, one for OpenAI's, one for local Ollama, one for a self-hosted vLLM instance. At startup, you tell OpenCode which one to use, and it works the same way from that point on.

This is different from a closed-source agent that's hard-wired to call a specific vendor's API. If you want to use a different provider, you'd have to modify the agent's internal code—something most users can't do, or aren't allowed to do.

How the abstraction works: a concrete layer

For this to work, OpenCode (or any provider-agnostic agent) separates the agent loop logic from the model-calling logic. The loop says: "I need to call the model with this context and these tools." The provider layer says: "Okay, I'll handle that using Anthropic's Claude API" (or Ollama, or anything else).

The abstraction is usually a common interface with methods like:

interface ModelProvider {
  call(prompt, systemPrompt, tools): Promise<ModelResponse>
  countTokens(text): number
  getModelName(): string
}

Each provider (Anthropic, OpenAI, Ollama, vLLM, etc.) implements this interface in its own way. The rest of the agent code doesn't care—it just calls modelProvider.call(...) and handles the response. As long as the response format is consistent (which it is, because the interface enforces it), the agent loop doesn't need to change.

The cost of generality: some providers are better

Because the abstraction has to work across many providers, OpenCode can't take advantage of provider-specific optimizations. A few examples:

Token counting. Anthropic's API has a dedicated token-counting endpoint that's very accurate. OpenAI's doesn't. A local Ollama instance has no token counter at all. An OpenCode implementation might use Anthropic's counter when available, fall back to a heuristic when not, and adjust its context-window strategy based on which provider it's talking to. But it can't just use the most optimized path unconditionally—it has to work with all of them.

Caching. Anthropic's API recently introduced prompt caching, which can reduce latency and cost for repeated queries. OpenAI doesn't have an equivalent feature (yet). OpenCode can use caching when it's available, but it can't rely on it or bake it into the core loop logic, because other providers don't support it.

Tool calling. Every provider handles tool calls slightly differently—how they encode tool definitions in the request, how they return tool calls in the response. OpenCode normalizes these differences behind the interface so the agent loop doesn't see them. But normalization means losing some of the nuance each provider offers.

The upside: you can swap providers and the agent works. The downside: you're never getting the absolute best performance from any single provider, because the architecture has to work with all of them.

A worked example: swapping from Claude to Llama

Say you start with OpenCode using Claude API. Your config looks something like:

provider: anthropic
model: claude-3-5-sonnet
api_key: ${ANTHROPIC_API_KEY}
max_tokens: 4096

After running for a month, you realize you want to try a local Llama instance to avoid API costs and keep code private. You change your config to:

provider: ollama
model: llama2-13b
endpoint: http://localhost:11434
max_tokens: 2048

Restart OpenCode. The exact same agent loop now calls Ollama instead of Anthropic. The request format changes (Ollama doesn't use the same JSON structure as Anthropic's API), but that translation happens inside the Ollama provider layer. The agent logic sees no difference.

Of course, the model itself is different—Llama 2 might behave differently than Claude, might be slower, might produce different code. But the structure of the agent is unchanged, and you didn't have to rewrite any agent code to make the switch.

How to add a new provider

If you want to support a model provider that OpenCode doesn't have built-in support for yet, you implement the provider interface:

1. Create a new file (e.g., providers/my-provider.ts)
2. Implement the ModelProvider interface with your provider's specifics—how to construct requests, parse responses, count tokens, handle errors
3. Register it in the provider registry so OpenCode can load it by name
4. Update your config to use it

You don't modify the core agent loop, the orchestrator, or the tool-calling logic. You just teach OpenCode how to talk to your new model. This is why provider agnosticism is so powerful for extensibility—adding support for a new provider is a contained task, not a rewrite.

The tradeoff: generality vs. optimization

This is the fundamental tradeoff in any provider-agnostic system. You can build an agent that works with any LLM provider, but you'll never be able to fully exploit the specific strengths of each one. The more providers you support, the more you have to stay in the "common ground" of features they all share.

For OpenCode, this tradeoff is worth it—the ability to swap providers and stay in control of your data and costs outweighs the loss of provider-specific optimizations. But it's worth understanding the tradeoff exists, so you know what you're trading away when you choose provider agnosticism over a tightly optimized, single-provider agent.

Common mistake

Assuming that because OpenCode is provider agnostic, all providers will work equally well for all tasks. They won't. Claude might be better at reasoning through complex refactors; Llama might be faster for simple edits and cheaper to run long-running tasks. Mistral might excel at specific languages. The abstraction makes it easy to swap providers, but it doesn't make all providers equivalent. Spend time finding the right provider for your workflow, then stick with it long enough to form a real opinion.