Running LLMs Locally Cheat Sheet

This content was generated with the assistance of AI and should be verified against the official documentation of each tool and framework before being relied on in production.

This section’s bibliography lists the reference material consulted while preparing these pages.

A single-page index of quick back-links to every page in this sub-section, grouped the same way as the sub-section itself — handy for jumping straight to the topic you need instead of paging through the full navigation.

The downloadable PDF is a separate, single A4 page meant for printing. It states the version baseline this sub-section targets — Ollama 0.34.4, llama.cpp v0.5.0, vLLM 0.30.0, SGLang 0.5.20, LiteLLM 1.103.0, Claude Code 2.1.284 and Copilot CLI 1.0.89, all checked on 2026-09-28 — and covers:

  • the engine decision tree;

  • the sizing formulas and a quick memory table;

  • the GGUF quant ladder;

  • the Ollama commands and a Modelfile;

  • the llama.cpp, vLLM and SGLang launch lines;

  • the endpoint URLs and default ports per engine;

  • a LiteLLM config.yaml snippet;

  • the Copilot BYOK and Custom Endpoint steps and the Copilot CLI variables;

  • the Claude Code environment variables;

  • the Continue, Aider and OpenCode snippets;

  • the open-weight licence table;

  • the security checklist.

Foundations — Running LLMs Locally, Hardware Sizing and Quantization for Inference.

Engines — Ollama, llama.cpp, vLLM & SGLang, Desktop Runners, NVIDIA NIM and Open-Weight Models.

Gateways and optimisation — LLM Gateways and Inference Optimization.

Coding assistants — Local Models in GitHub Copilot, Local Models in Claude Code and Open Coding Assistants with Local Models.

Operations — Running Local LLMs in Production.

Download the Running LLMs Locally Cheat Sheet (PDF)

References

The PDF summarizes these official pages; each is cited in full on the page listed above that covers it, and in Running LLMs Locally’s bibliography.

Engines

Gateways and assistants

Papers