Local Models in Claude Code

This content was generated with the assistance of AI and should be verified against the official documentation of each tool and framework before being relied on in production.

This section’s bibliography lists the reference material consulted while preparing these pages.

This page was written against Claude Code 2.1.284, Ollama 0.34.4, vLLM 0.30.0, LM Studio 0.4 and LiteLLM 1.103.0, as checked on 2026-09-28 (see the Running LLMs Locally baseline). The Claude Code pages quoted below were read on code.claude.com on that date. Installing and using Claude Code is on Getting Started with Claude Code; its settings files and scopes are on Permissions & Settings.

The supported enterprise routes

Claude Code talks to Anthropic’s API by default. Organisations that cannot or will not use it directly have four documented alternatives, all of which still serve Claude models:

Route Enable with Notes

Route	Enable with	Notes
Amazon Bedrock	`CLAUDE_CODE_USE_BEDROCK=1` (plus AWS credentials and `AWS_REGION`)	Uses the Bedrock Invoke API; a `/setup-bedrock` wizard exists.
Google Vertex AI	`CLAUDE_CODE_USE_VERTEX=1`, `ANTHROPIC_VERTEX_PROJECT_ID`, `CLOUD_ML_REGION`	The Claude Code docs now call it Google Cloud’s Agent Platform; global, multi-region and regional endpoints.
Microsoft Foundry	`CLAUDE_CODE_USE_FOUNDRY=1`, `ANTHROPIC_FOUNDRY_RESOURCE`	Environment variables are the only configuration path.
An LLM gateway	`ANTHROPIC_BASE_URL` plus a credential	Any gateway that exposes a supported API format.

Amazon Bedrock

CLAUDE_CODE_USE_BEDROCK=1 (plus AWS credentials and AWS_REGION)

Uses the Bedrock Invoke API; a /setup-bedrock wizard exists.

Google Vertex AI

CLAUDE_CODE_USE_VERTEX=1, ANTHROPIC_VERTEX_PROJECT_ID, CLOUD_ML_REGION

The Claude Code docs now call it Google Cloud’s Agent Platform; global, multi-region and regional endpoints.

Microsoft Foundry

CLAUDE_CODE_USE_FOUNDRY=1, ANTHROPIC_FOUNDRY_RESOURCE

Environment variables are the only configuration path.

An LLM gateway

ANTHROPIC_BASE_URL plus a credential

Any gateway that exposes a supported API format.

For a gateway, the credential goes in one of three places: ANTHROPIC_AUTH_TOKEN (sent as Authorization: Bearer … — the default choice), ANTHROPIC_API_KEY (sent as x-api-key), or an apiKeyHelper — a command in your settings file that prints a fresh credential, for keys that rotate or come from a vault. ANTHROPIC_CUSTOM_HEADERS adds headers (Name: Value, newline-separated) for gateways that need them. Set the variables in the env block of ~/.claude/settings.json or .claude/settings.local.json so background sessions see them too — and never in a project’s committed .claude/settings.json:

{
  "env": {
    "ANTHROPIC_BASE_URL": "https://llm-gateway.internal.example.com"
  },
  "apiKeyHelper": "~/bin/get-gateway-token.sh"
}

Model pinning decides what each alias resolves to: ANTHROPIC_DEFAULT_OPUS_MODEL, ANTHROPIC_DEFAULT_SONNET_MODEL, ANTHROPIC_DEFAULT_HAIKU_MODEL (also used for background tasks) and ANTHROPIC_DEFAULT_FABLE_MODEL, with ANTHROPIC_DEFAULT_MODEL choosing the model new sessions start on. CLAUDE_CODE_ENABLE_GATEWAY_MODEL_DISCOVERY=1 fills the /model picker from the gateway’s /v1/models. Run /status to confirm which base URL and credential are active. See Enterprise deployment overview, Other LLM gateways, Connect Claude Code to an LLM gateway, Amazon Bedrock, Google Vertex AI, Microsoft Foundry, Model configuration and Settings.

What Anthropic supports — and what it does not

The same mechanism — ANTHROPIC_BASE_URL pointed at something that speaks the Anthropic Messages API — also lets Claude Code talk to an open-weight model served locally, because Ollama, vLLM, SGLang, LM Studio, LocalAI, llama.cpp and LiteLLM all implement /v1/messages. Before you do that, know where Anthropic stands. Its gateway documentation says, in so many words, that Anthropic "doesn’t endorse, maintain, or audit third-party gateway products, and doesn’t support routing Claude Code to non-Claude models through any gateway".

In practice that means: it can work, the engines above document it, and it is useful for offline work, private code and experiments — but you are on your own. Claude Code’s prompts, tools and features are built and tested against Claude models; when a local model mis-handles a tool call or a new Claude Code feature needs a header the engine does not forward, there is no support channel at Anthropic for it, and a behaviour that works today can break on the next Claude Code release. For production use of Claude Code in an organisation, the supported routes are the four in the table above, serving Claude.

Pointing Claude Code at a local endpoint

Ollama

The Ollama-documented setup is one command, which configures and launches Claude Code against a local (or :cloud) model:

ollama launch claude
ollama launch claude --model qwen3.6        # choose the model up front

The manual equivalent is two variables — Ollama requires a token but ignores its value:

export ANTHROPIC_BASE_URL=http://localhost:11434
export ANTHROPIC_AUTH_TOKEN=ollama
claude --model qwen3.6

Ollama’s Anthropic compatibility has gaps (no tool_choice, prompt caching, token counting endpoint or PDF blocks; see Ollama). Set the context length to at least 64k, which Ollama itself recommends for Claude Code. See Ollama — Claude Code and Ollama — Anthropic compatibility.

vLLM and SGLang

Serve a tool-calling model with the right parser, give it a served model name without a /, then map every alias to it. vLLM’s own Claude Code guide warns that a Hugging Face-style name such as openai/gpt-oss-120b cannot be used directly in this setup, which is why it renames the model with --served-model-name:

vllm serve openai/gpt-oss-120b --served-model-name my-model \
  --enable-auto-tool-choice --tool-call-parser openai --api-key "$VLLM_API_KEY"

ANTHROPIC_BASE_URL=http://gpu-server.internal:8000 \
ANTHROPIC_AUTH_TOKEN="$VLLM_API_KEY" \
ANTHROPIC_DEFAULT_OPUS_MODEL=my-model \
ANTHROPIC_DEFAULT_SONNET_MODEL=my-model \
ANTHROPIC_DEFAULT_HAIKU_MODEL=my-model \
claude

Claude Code places an attribution block (client version and a prompt fingerprint) at the start of the system prompt; because it changes per request, it can defeat the engine’s prefix cache. vLLM handles this automatically from version 0.17.2; for older vLLM, and for SGLang and LM Studio, set CLAUDE_CODE_ATTRIBUTION_HEADER=0. SGLang also recommends raising API_TIMEOUT_MS for long reasoning outputs. See vLLM — Claude Code, SGLang — Anthropic-compatible API and Claude Code — Environment variables.

LM Studio

lms server start --port 1234
export ANTHROPIC_BASE_URL=http://localhost:1234
export ANTHROPIC_AUTH_TOKEN=lmstudio              # or your LM Studio API token if authentication is on
export CLAUDE_CODE_ATTRIBUTION_HEADER=0
claude --model openai/gpt-oss-20b

The / warning above is vLLM’s: LM Studio’s own Claude Code guide passes its model key openai/gpt-oss-20b to --model unchanged. If a name with a / is rejected by your setup, map it through the ANTHROPIC_DEFAULT_*_MODEL variables as in the vLLM example, or load the model under a slash-free identifier. LM Studio recommends a context length above about 25k tokens for Claude Code. See LM Studio — Claude Code.

LiteLLM

Through a gateway, Claude Code uses a virtual key and the gateway’s model names (see LLM Gateways):

export ANTHROPIC_BASE_URL=http://localhost:4000
export ANTHROPIC_AUTH_TOKEN="$LITELLM_VIRTUAL_KEY"
export ANTHROPIC_DEFAULT_SONNET_MODEL=docs-chat
claude

A gateway also lets the same Claude Code configuration route to Claude on Bedrock, Vertex AI or Foundry for production and to a local model for experiments — with the non-support caveat above applying only to the latter. See LiteLLM — /v1/messages.

Practical limits

Context: Claude Code’s system prompt, tool definitions and file contents are large. Ollama recommends at least 64k tokens; below that, sessions truncate or auto-compact constantly. Make sure the engine allocates it — the model’s advertised maximum is not what a local server gives you by default.
Tool use quality: Claude Code is an agent — it edits files, runs commands and chains many tool calls. Models that are good at chat but weak at structured tool calls loop, hallucinate tool arguments or stop early. Use a model and parser the engine documents for tool calling (--jinja in llama.cpp, --tool-call-parser in vLLM), and prefer models their vendors position for agentic coding.
Hardware: a model good enough for agentic coding at 64k context needs a workstation-class GPU or a large unified-memory Mac (see Hardware Sizing); on a laptop, expect long waits on every turn.
Features: anything that depends on Anthropic-side features the engine does not implement — prompt caching, token counting, server tools, extended-thinking budgets — degrades or fails.

Security of local endpoints

Pointing Claude Code at a local server moves the security boundary to that server:

Keep the engine on 127.0.0.1 or a private network. Most local engines are unauthenticated by default, and Claude Code will happily send your source code to whatever ANTHROPIC_BASE_URL names — including a mistyped or hijacked host.
For a shared server, put TLS and per-user keys in front (see authentication in front of local servers) and distribute the credential with apiKeyHelper or managed settings, not a committed file.
Claude Code’s permission rules still apply, and matter more with a weaker model that may propose risky commands; see Permissions & Settings.
Disable cloud fallbacks (OLLAMA_NO_CLOUD=1, no hosted models in the gateway) if the point of the exercise is that code never leaves the network.