Open-Weight Models

This content was generated with the assistance of AI and should be verified against the official documentation of each tool and framework before being relied on in production.

This section’s bibliography lists the reference material consulted while preparing these pages.

This page was written on 2026-09-28 with Ollama 0.34.4 and vLLM 0.30.0 for the examples (see the Running LLMs Locally baseline), and its table was re-checked against the Hugging Face model cards (license: metadata, model card text and config.json) on 2026-09-29. Model generations move fast: treat the table as a snapshot of that date, not a ranking. The Checked against column says where each row was verified — confirm it on the model card before you ship.

The conceptual landscape — closed vs. open models, providers, how model families relate — is on The Model Landscape; this page is about which open weights to download and run.

The families, as of 2026-09-28

Family Current open sizes Architecture Context Licence Checked against

Qwen3.8 (Alibaba)

27B (Qwen/Qwen3.8-27B, dense, vision-language); 2.4T total / 95B active (Qwen/Qwen3.8-2.4T-A95B, the open-weight base of Qwen3.8-Max)

27B: dense; 2.4T-A95B: MoE

262,144 for the 27B, per the vendor’s own vLLM/SGLang launch lines

Differs per model: 27B — Apache-2.0; 2.4T-A95B — custom qwen3.8-max licence. Each -FP8 repository carries the same licence as its parent.

Qwen’s official GitHub repository (release entries of 2026-08-12 and 2026-08-14); licences from each model and -FP8 repository’s license: tag on the Qwen Hugging Face org (the org page itself lists none)

Qwen3.6 (Alibaba)

27B; 35B total / 3B active

dense; MoE

262,144 native

Apache-2.0

QwenLM README (releases 2026-04); SGLang cookbook (context, licence)

Gemma 4 (Google)

E2B, E4B, 12B, 31B; 26B total / 4B active

dense; MoE

128K (E2B, E4B); 256K (12B, 26B-A4B, 31B)

Apache-2.0

Google Open Source Blog announcement (licence); SGLang cookbook (sizes); google/gemma-4-31B-it model card (context)

DeepSeek V4

Flash, Pro

MoE

1M

MIT

SGLang cookbook (released 2026-04-24 under MIT); the deepseek-ai model cards were not reachable

DeepSeek V3.2

671B total / 37B active

MoE

128K

MIT

vendor GitHub repository (code and weights under MIT; inference config config_671B_v3.2.json); V3.2 builds on V3.1-Terminus, whose V3.1 model card lists 671B / 37B / 128K

Mistral Small 4

119B total / 6.5B active

MoE

256K

Apache-2.0

SGLang cookbook; the Mistral model overview was not reachable

Ministral 3, Devstral 2 (Mistral)

Ministral 3 14B; Devstral Small 2 24B, Devstral 2 123B

dense

256K (Devstral 2)

Ministral 3: Apache-2.0; Devstral Small 2: Apache-2.0; Devstral 2 123B: Modified MIT (no rights for companies above USD 20M monthly revenue)

SGLang cookbook; Devstral Small 2 and Devstral 2 model cards and LICENSE (licences, context)

gpt-oss (OpenAI)

20b: 21B / 3.6B active; 120b: 117B / 5.1B active

MoE (MXFP4 experts)

131,072

Apache-2.0

openai/gpt-oss README; gpt-oss model card (context)

Phi-4 (Microsoft)

Phi-4, Phi-4-mini, Phi-4-multimodal, Phi-4-reasoning

dense

16K (Phi-4); 128K (mini, multimodal); 32K (reasoning)

MIT

PhiCookBook (model names); the microsoft/phi-4, Phi-4-mini-instruct, Phi-4-multimodal-instruct and Phi-4-reasoning model cards (context, licence)

Llama 4 (Meta)

Scout 17B-16E (~109B total), Maverick 17B-128E (~400B total), 17B active

MoE

10M (Scout), 1M (Maverick)

Llama 4 Community License

meta-llama/llama-models (sizes, context, licence file)

Qwen3.8-Flash-Next is left out of the table on purpose. Qwen released it on 2026-08-26 as an experimental preview of the architecture Qwen4 will be built on: a multimodal MoE with 125B parameters plus a 51B N-gram embedding table (about 176B in total, 6B active per token), a 262,144-token native context that extends to 1M, and a custom qwen-community-1.0 licence (also on its -FP8 repository). Treat it as something to evaluate, not a current production family; see the model card and the QwenLM/Qwen3.8-Flash-Next repository (news entry of 2026-08-26).

Older generations used in some books — Mistral-7B, Llama 2, Phi-3 — still run, but every family above has superseded them.

Licence classes

Open weights are not the same thing as open source, and the licence decides whether you may use a model at all:

  • Permissive (Apache-2.0, MIT) — use, modify, fine-tune and redistribute, commercially, with attribution. Apache-2.0 adds an explicit patent grant. Qwen3.6, Qwen3.8-27B, Gemma 4, Mistral Small 4, Ministral 3, Devstral Small 2 and gpt-oss (Apache-2.0), and DeepSeek V3.2 / V4 and Phi-4 (MIT) are in this class. This is the class to prefer for anything you ship.

  • Community licences — custom terms written by the vendor. The Llama 4 Community License, for example, is a separate agreement with its own acceptable-use policy and conditions; older DeepSeek models used a "DeepSeek License" model agreement distinct from the MIT code licence; within Qwen3.8 the 2.4T-A95B (qwen3.8-max) and Flash-Next (qwen-community-1.0) models carry custom licences even though the 27B is Apache-2.0; and Devstral 2 123B’s "Modified MIT" licence excludes companies above USD 20M monthly revenue while Devstral Small 2 is Apache-2.0 — so a family name never tells you the licence; the individual repository does. Read the actual text: these licences can restrict use cases, require attribution in product names, or add conditions for very large deployments.

  • Gated — the Hub repository requires you to accept the licence (and sometimes share contact details) before downloading. Gating is a distribution mechanism, not a licence type: a gated model can carry a permissive or a community licence. Pass HF_TOKEN to the engine after accepting.

The licence of a Hub repository is in its model card metadata (license:), with the full text usually in a LICENSE file; see Hugging Face — Licenses. Quantized re-uploads (GGUF, AWQ) inherit the original licence — check the base model’s, not the uploader’s summary. This is engineering guidance, not legal advice; involve your legal team for anything customer-facing.

Choosing by task

Choose by task first, then by what fits (see Hardware Sizing); this site quotes no benchmark numbers, so evaluate the two or three candidates on your own prompts.

  • Coding and coding agents — use the coding-tuned variants (Devstral, Qwen-Coder, and the general Qwen3.6 / Qwen3.8 models, which their vendor positions for agentic coding). Tool-calling reliability and at least 64k context matter more than raw size for the assistants on Local Models in Claude Code and Local Models in GitHub Copilot.

  • Reasoning — models with a thinking mode (gpt-oss with adjustable reasoning effort, DeepSeek, Qwen’s hybrid reasoning, Phi-4-reasoning). They trade latency and tokens for accuracy; see Reasoning Models.

  • Tool use and agents — a model whose chat template supports tools (the Ollama library tags them), served with the right parser (--jinja in llama.cpp, --tool-call-parser in vLLM). MoE models with few active parameters (gpt-oss-20b, Qwen3.6-35B-A3B, Gemma 4 26B-A4B) are fast enough for multi-step loops on one GPU.

  • Multilingual — Qwen3.5 and later state support for 201 languages and dialects; Gemma and Mistral are also strong multilingual families. Test in your languages — 4-bit quantization hurts non-English text first (see Quality vs. size).

  • Embeddings — a separate, much smaller model: DocsAssistant uses nomic-embed-text; see Generating Embeddings in Practice.

  • On-device and in-app — the smallest variants (Gemma 4 E2B / E4B, Qwen3.5 0.8B-4B, Phi-4-mini) are covered in Small LLMs & Embeddings.

Example: serving an Apache-2.0 model in Ollama and vLLM

gpt-oss-20b is Apache-2.0, runs within 16 GB of memory thanks to its MXFP4 experts (per OpenAI’s README), and runs in both engines. In Ollama:

ollama pull gpt-oss:20b
ollama run gpt-oss:20b "Explain the difference between Apache-2.0 and a community licence in three bullets."

See the openai/gpt-oss README (Ollama section) and Ollama. On a GPU server with vLLM:

vllm serve openai/gpt-oss-20b --enable-auto-tool-choice --tool-call-parser openai
curl http://localhost:8000/v1/chat/completions -H "Content-Type: application/json" \
  -d '{"model": "openai/gpt-oss-20b", "messages": [{"role": "user", "content": "Hello"}]}'

See the openai/gpt-oss README (vLLM section) and vLLM — Online serving. The same pattern works for Qwen3.6 (ollama pull qwen3.6, vllm serve Qwen/Qwen3.6-27B) once you have checked its licence on the model card.

References