Deployment

This content was generated with the assistance of AI and should be verified against the official documentation of each tool and framework before being relied on in production.

This section’s bibliography lists the reference material consulted while preparing these pages.

This page was written against langchain 1.4.3, langchain-core 1.6.5 and langgraph 1.2.12 on Python 3.12, with langgraph-cli 0.4.32 (which pulled in langgraph-api 0.15.1, langgraph-runtime-inmem 0.35.1 and langgraph-sdk version 0.4.5), langchain-ollama 1.1.0 and FastAPI 0.141.1, as verified on 2026-09-29. The running example is DocsAssistant, which answers questions over this site’s documentation with Ollama’s llama3.1:8b chat model over the pgvector store built in Integrating with LangChain. Ollama was not available while the snippets were checked, so the graph in this page swaps the model for a small scripted fake through an environment variable and the search_docs tool is a stub; every output shown was produced by a real langgraph dev process and real HTTP and SDK calls against it, not by a live model. The Docker-based commands (langgraph up, langgraph build) and the LangSmith-hosted ones (langgraph deploy, Studio) need a Docker daemon or a LangSmith account and were not run; they are described from the official documentation and the CLI’s --help.

Everything before this page ran inside one Python process. That is the right place to build an agent, and the wrong place to leave it: a browser cannot call a Python function, a conversation has to survive a restart, a slow tool call must not block the request that started it, and someone has to run the nightly job. LangChain’s answer is to put the compiled graph behind a ready-made server that already knows about threads, checkpoints, streaming and schedules, instead of writing that plumbing per project. This page shows how the graph is packaged for it, how it is run locally, what the server offers, where it can be hosted, and what to do when you would rather write the server yourself.

Names: what changed from LangGraph Platform

The hosted product and its tooling were renamed, and much of the older material (blog posts, videos, books written against the 0.x libraries) still uses the old names. The LangChain changelog announces that "LangGraph Platform is now LangSmith Deployment and LangGraph Studio is now LangSmith Studio", and the server component is documented as Agent Server. The command-line tool kept its name.

Table 1. Old names and current names
Older name Current name What it is

LangGraph Platform

LangSmith Deployment

The managed and self-hostable runtime for deploying agents; one of the three services of LangSmith, next to observability and evaluation.

LangGraph Studio

LangSmith Studio

The agent IDE for visualizing, running and debugging a graph served by an Agent Server.

The platform’s API server

Agent Server

The process that exposes the assistants, threads, runs, crons and store API, with a task queue and PostgreSQL behind it.

langgraph command

langgraph command (unchanged)

The LangGraph CLI, installed by the langgraph-cli package.

langgraph.json

langgraph.json (unchanged)

The configuration file that tells the CLI and the server what to load.

See LangSmith changelog (the entry on the product naming changes) and LangSmith Deployment.

From graph to deployable application

A deployable application is a directory with four things: the graph code, a file that lists its Python dependencies, an optional .env file, and langgraph.json. DocsAssistant’s layout:

DocsAssistant’s project layout
docs-assistant/
├── docs_assistant/
│   ├── __init__.py
│   ├── agent.py          # builds the agent; exports the compiled graph
│   └── fake.py           # scripted model used while this page was written
├── .env                  # local-only environment variables (never committed)
├── langgraph.json
└── pyproject.toml        # dependencies

The graph module exports a compiled graph. The one rule specific to deployment is that the module does not configure a checkpointer or a store: Agent Server injects the ones configured for the deployment (PostgreSQL by default), because it needs to manage them for its own operations. The build_agent function takes an optional checkpointer only so the self-serving example at the end of this page can reuse the same code.

docs_assistant/agent.py — the graph the server will load
import os

from langchain.agents import create_agent
from langchain.tools import tool


@tool
def search_docs(query: str) -> str:
    """Search this site's documentation and return the best matching section."""
    return f"HNSW ef_search trades recall for latency (query: {query})."     # stub: the real tool queries pgvector


def _model():
    if os.environ.get("DOCS_ASSISTANT_FAKE_MODEL") == "1":      # used only while writing this page
        from docs_assistant.fake import ScriptedChat
        return ScriptedChat()
    from langchain_ollama import ChatOllama
    return ChatOllama(model="llama3.1:8b")


def build_agent(checkpointer=None):
    return create_agent(
        _model(),
        tools=[search_docs],
        system_prompt="You answer questions about this site's documentation. Always call search_docs first.",
        checkpointer=checkpointer,
        name="docs_assistant",
    )


# Agent Server injects its own checkpointer and store, so the deployed graph is built without one.
graph = build_agent()

See LangChain — Agents for create_agent and Agents with create_agent for how it is built up.

The configuration file maps graph IDs to the place the compiled graph lives:

langgraph.json
{
  "dependencies": ["."],
  "graphs": {
    "docs_assistant": "./docs_assistant/agent.py:graph"
  },
  "env": "./.env",
  "python_version": "3.12"
}

The keys used here, as described in the CLI reference:

  • dependencies (required): "." installs the local package described by pyproject.toml; a directory containing pyproject.toml, setup.py or requirements.txt, or a package name, also work.

  • graphs (required): graph ID to path/to/file.py:variable. The variable is either a compiled graph, which the server loads once at startup and reuses for every run, or a function that returns a graph and is called on every run. Prefer the compiled graph; use a factory function only when a run must build a different graph, for example to choose tools from the assistant’s configuration.

  • env: the path of a .env file, or a mapping of variable names to values. It is a local-development convenience; a production deployment sets environment variables in the deployment environment.

  • python_version: 3.11, 3.12 or 3.13 for the image that langgraph build and langgraph dockerfile produce.

Other keys the reference documents, and that this page does not use, include auth, store (semantic search and TTL for the long-term store), checkpointer (backend and TTL), http (custom app, CORS, route switches), webhooks, dockerfile_lines, image_distro and api_version. Check the file before starting anything with the CLI:

Validating the configuration
langgraph validate
Output
Configuration file /.../docs-assistant/langgraph.json is valid. (1 graph found)

One detail from the run: the documentation’s examples begin with a "$schema" line, but langgraph-cli 0.4.32 printed the warning Unknown key '$schema' is not a recognized config field when it was present, so the file above leaves it out. It is only a warning, and an editor that supports the published JSON schema gains completion from it.

See Application structure and LangGraph CLI — Configuration file.

Running it locally with langgraph dev

langgraph dev starts a lightweight Agent Server inside your own Python environment: no Docker, hot reloading, and state kept in memory and pickled to a local directory (.langgraph_api/, which belongs in .gitignore). It needs the inmem extra:

Installing the CLI and starting the dev server
pip install -U "langgraph-cli[inmem]"
langgraph dev --no-browser --port 2247
Startup output (abridged)
- API: http://127.0.0.1:2247
- Studio UI: https://smith.langchain.com/studio/?baseUrl=http://127.0.0.1:2247
- API Docs: http://127.0.0.1:2247/docs

This in-memory server is designed for development and testing.
For production use, please use LangSmith Deployment.

The flags used above and the other ones worth knowing, from langgraph dev --help:

Flag Effect

--port, --host

Where to bind. The host defaults to 127.0.0.1, which is what you want: the development server has no authentication unless you configure it, so only bind 0.0.0.0 on a trusted network.

--no-browser

Do not open the browser (Studio) on startup.

--no-reload

Disable hot reloading. It is on by default: saving a file that a graph imports reloads the server.

--config

Use a configuration file other than ./langgraph.json.

--tunnel

Expose the server through a public Cloudflare tunnel, for a remote front end that cannot reach localhost. Anyone with the URL reaches the server, so combine it with authentication (see below).

--debug-port, --wait-for-client

Listen for a debugger (needs debugpy); the server can wait for the IDE to attach.

--n-jobs-per-worker

Concurrent runs per worker; the default is 10.

Two quick checks that it is alive: GET /ok returns a health response, and GET /info reports the versions. The server also serves an OpenAPI description and interactive documentation at /docs.

Health check and server information
curl -s localhost:2247/ok
curl -s localhost:2247/info
Output
{"ok":true}
{"version":"0.15.1","langgraph_py_version":"1.2.12","flags":{"assistants":true,"crons":true, ... }}

langgraph dev is one of two local modes. The other, langgraph up, builds and runs the real containers (the API server, PostgreSQL and Redis) with Docker, listening on port 8123 by default and needing a LangSmith API key for local testing. It is the pre-release check for things dev cannot show, such as behaviour under the real database.

Table 2. The two local modes
Aspect langgraph dev langgraph up

Docker

Not required

Required

Install

pip install "langgraph-cli[inmem]"

pip install langgraph-cli

State

In memory, pickled to a local directory

PostgreSQL

Hot reload

Yes, by default

Optional (--watch)

Default port

2024

8123

Use for

Daily development and quick tests

Production-like validation

The documented workflow is langgraph dev while writing, langgraph up before a release, then a deployment; the dev server is tested against the same integration suite as the production one, so behaviour matches.

See Local development and testing and LangGraph CLI — dev.

Agent Server concepts

An Agent Server exposes four resources and a store. They fit together in a way that is easy to blur, so it is worth fixing the vocabulary before calling the API.

flowchart LR G[Graph
deployed code] --> A1[Assistant
config v1] G --> A2[Assistant
config v2] A1 --> R1[Run] A2 --> R2[Run] T[Thread
one conversation] --> R1 T --> R2 R1 --> C[(Checkpoints
PostgreSQL)] R2 --> C CR[Cron job] -->|on a schedule| R1
Figure 1. How a graph, its assistants, threads, runs and crons relate
Assistant

A named configuration of a graph — prompts, model choice, tools — kept apart from the graph’s logic. Deploying a graph creates a default assistant for it, called by the graph’s ID. You can create more assistants for the same graph, each versioned: editing one creates a new version that you can promote or roll back. Assistants are a Deployment concept and do not exist in the open-source library.

Thread

The persistent container for one conversation’s state. Every run on a thread starts from the thread’s current state and writes the new state back, which is how "and ef_construction?" can follow "what does ef_search do?". This is the same thread_id that Short-Term Memory & Checkpointers puts in config["configurable"]; the server just manages it for you.

Run

One invocation of an assistant, optionally on a thread. The server creates the run in a durable queue and a worker executes it, so a run can be started in the background and joined or streamed later. At most one run executes on a given thread at a time; what to do when a second message arrives while one is running is the double-texting problem covered in Human-in-the-Loop & Time Travel.

Cron job

A schedule plus an assistant plus an input. On each tick the server sends the input to the assistant — either on an existing thread or, for a stateless cron, on a new thread per execution. Schedules are cron expressions interpreted in UTC.

Store

The long-term memory store from Long-Term Memory & Stores, exposed under /store and backed by PostgreSQL unless you replace it.

The dev server’s OpenAPI description lists these as routes: /assistants, /threads, /runs, /runs/crons, /store/items and more, plus /mcp/ and /a2a/{assistant_id} endpoints that expose the deployed agent over the Model Context Protocol and the Agent2Agent protocol (see Model Context Protocol and Agent-to-Agent Protocol).

The Python SDK is the friendliest way to drive them. The script below, run against the langgraph dev process started above, creates a second assistant for the graph, updates it, opens a thread, runs the agent with wait and with stream, reads the thread’s state and history, starts a background run and joins it, and schedules and deletes a cron job:

Assistants, threads, runs and crons through langgraph_sdk
import asyncio

from langgraph_sdk import get_client


async def main():
    client = get_client(url="http://127.0.0.1:2247")

    # assistants: the default one exists per graph
    found = await client.assistants.search(graph_id="docs_assistant")
    print("default assistants:", [(a["graph_id"], a["name"]) for a in found])

    # a second assistant on the same graph, with its own configuration
    concise = await client.assistants.create(
        graph_id="docs_assistant",
        name="docs-assistant-concise",
        config={"configurable": {"style": "concise"}},
        metadata={"owner": "docs-team"},
    )
    print("created:", concise["name"], concise["version"])
    updated = await client.assistants.update(concise["assistant_id"], name="docs-assistant-concise-v2")
    print("after update: version", updated["version"])

    # threads: one per user conversation
    thread = await client.threads.create(metadata={"user": "alice"})
    print("thread status:", thread["status"])

    # runs: wait for the result
    result = await client.runs.wait(
        thread["thread_id"], "docs_assistant",
        input={"messages": [{"role": "user", "content": "What does ef_search do?"}]},
    )
    print("run.wait ->", result["messages"][-1]["content"])

    # runs: stream
    async for part in client.runs.stream(
        thread["thread_id"], "docs_assistant",
        input={"messages": [{"role": "user", "content": "And ef_construction?"}]},
        stream_mode="updates",
    ):
        print("stream:", part.event, list(part.data)[:2] if isinstance(part.data, dict) else "")

    state = await client.threads.get_state(thread["thread_id"])
    print("messages on thread:", len(state["values"]["messages"]))
    hist = await client.threads.get_history(thread["thread_id"], limit=3)
    print("checkpoints returned:", len(hist))

    # background run, then join
    run = await client.runs.create(
        thread["thread_id"], "docs_assistant",
        input={"messages": [{"role": "user", "content": "background question"}]},
    )
    print("background run status at creation:", run["status"])
    await client.runs.join(thread["thread_id"], run["run_id"])
    print("after join:", (await client.runs.get(thread["thread_id"], run["run_id"]))["status"])

    # cron: every Monday at 06:00 UTC, on this thread
    cron = await client.crons.create_for_thread(
        thread["thread_id"], "docs_assistant",
        schedule="0 6 * * 1",
        input={"messages": [{"role": "user", "content": "Weekly docs gap report"}]},
    )
    print("cron schedule:", cron["schedule"])
    print("crons listed:", len(await client.crons.search()))
    await client.crons.delete(cron["cron_id"])
    print("crons after delete:", len(await client.crons.search()))


asyncio.run(main())
Output (scripted fake model)
default assistants: [('docs_assistant', 'docs_assistant')]
created: docs-assistant-concise 1
after update: version 2
thread status: idle
run.wait -> According to the docs: HNSW ef_search trades recall for latency (query: What does ef_search do?).
stream: metadata ['run_id', 'attempt']
stream: updates ['model']
stream: updates ['tools']
stream: updates ['model']
messages on thread: 8
checkpoints returned: 3
background run status at creation: pending
after join: success
cron schedule: 0 6 * * 1
crons listed: 1
crons after delete: 0

Read the output as a tour of the model. The default assistant carries the graph’s ID as its name. Updating an assistant bumped its version from 1 to 2 without touching the graph. A background run is pending the moment it is created and success after join, because the API server only enqueued it. The updates events are the stream_mode="updates" parts from Streaming, one per graph step. Restarting the dev server and running the script again listed the earlier assistant next to the default one, which shows the pickled local state at work. The cron job was deleted at the end on purpose: a forgotten cron keeps calling a paid model.

The same operations are plain HTTP, which is what a front end or another service would use:

The REST equivalent: create a thread, wait for a run, stream another
THREAD=$(curl -s -X POST localhost:2247/threads -H 'content-type: application/json' -d '{}' \
  | python3 -c "import json,sys; print(json.load(sys.stdin)['thread_id'])")

curl -s -X POST localhost:2247/threads/$THREAD/runs/wait -H 'content-type: application/json' \
  -d '{"assistant_id":"docs_assistant","input":{"messages":[{"role":"user","content":"What does ef_search do?"}]}}'

curl -s -N -X POST localhost:2247/threads/$THREAD/runs/stream -H 'content-type: application/json' \
  -d '{"assistant_id":"docs_assistant","input":{"messages":[{"role":"user","content":"more"}]},"stream_mode":["updates"]}'
Output of the streaming call (abridged): a server-sent event stream
event: metadata
data: {"run_id":"...","attempt":1}

event: updates
data: {"model":{"messages":[{"content":"","additional_kwargs":{}, ...

event: updates
data: {"tools":{"messages":[{"content":"HNSW ef_search trades recall for latency (query: more)." ...

Finally, RemoteGraph makes a deployed graph look like a local one, so client code written against a compiled graph keeps working after the move to a server. Do not point a RemoteGraph at the deployment it runs in: the documentation warns that it can deadlock; compose graphs locally as subgraphs instead.

Calling the deployment as if it were a local graph
from langchain_core.messages import HumanMessage
from langgraph.pregel.remote import RemoteGraph

remote = RemoteGraph("docs_assistant", url="http://127.0.0.1:2247")
out = remote.invoke({"messages": [HumanMessage("What does ef_search do?")]})
print(out["messages"][-1]["content"])
Output
According to the docs: HNSW ef_search trades recall for latency (query: What does ef_search do?).

For a hosted deployment, RemoteGraph also needs a LangSmith API key, read from LANGSMITH_API_KEY or passed as api_key; it was not needed against the unauthenticated local server.

See Agent Server, Assistants, Use threads, Runs, Cron jobs and RemoteGraph.

What runs behind the API

Knowing the moving parts explains the behaviour above and the choices in the next sections. A deployment is one or more graphs, a PostgreSQL database and a task queue. PostgreSQL holds the resource data (assistants, threads, runs, crons), the checkpoints — so a run interrupted by a worker restart resumes from its last checkpoint — and the long-term store. Redis carries only ephemeral traffic between API servers and queue workers: run signals, cancellation and streaming events; no user or run data persists in it.

flowchart LR U[Client] -->|request| API[API servers] API -->|create run| DB[(PostgreSQL)] API -->|notify| RD[(Redis)] RD -->|wake| W[Queue workers] W -->|claim next run| DB W -->|checkpoints, status| DB W -->|publish events| RD RD -->|stream events| API API -->|server-sent events| U
Figure 2. The moving parts behind the Agent Server API

API servers accept requests and never execute agent code; queue workers run the graph and write checkpoints. Both are stateless containers built from the same image, so they scale independently — API servers with request volume, workers with the number of pending runs. Each worker executes up to N_JOBS_PER_WORKER runs concurrently (default 10). The default single host mode merges the two roles; the split API and queue mode and a distributed runtime are for larger installations. langgraph dev collapses all of this into one in-memory process, which is why it is a development tool.

See Agent Server — Parts of a deployment.

Securing the server

The development server ships with no authentication, and a self-hosted Agent Server has none by default either; a LangSmith-hosted deployment defaults to LangSmith API keys in the x-api-key header. For anything a user can reach, add your own authentication handler and point langgraph.json at it. A handler receives the request headers by parameter name and returns the caller’s identity or raises a 401; an @auth.on handler then stamps and filters resources by that identity, which is what stops one user from reading another’s thread.

docs_assistant/auth.py — authentication and per-owner authorization
import os

from langgraph_sdk import Auth

auth = Auth()


@auth.authenticate
async def authenticate(authorization: str | None) -> Auth.types.MinimalUserDict:
    """Accept a bearer token issued to the web client; a real deployment validates a JWT instead."""
    expected = os.environ["DOCS_ASSISTANT_CLIENT_TOKEN"]
    if authorization != f"Bearer {expected}":
        raise Auth.exceptions.HTTPException(status_code=401, detail="invalid token")
    return {"identity": "web-client"}


@auth.on
async def owner_only(ctx: Auth.types.AuthContext, value: dict) -> dict:
    """Stamp every resource with its owner and filter reads to that owner."""
    metadata = value.setdefault("metadata", {})
    metadata["owner"] = ctx.user.identity
    return {"owner": ctx.user.identity}
langgraph.json — turning the handler on
{
  "dependencies": ["."],
  "graphs": {"docs_assistant": "./docs_assistant/agent.py:graph"},
  "env": "./.env",
  "auth": {"path": "./docs_assistant/auth.py:auth"}
}

Started with DOCS_ASSISTANT_CLIENT_TOKEN set in the environment, the dev server logged Using auth of type=custom, and:

An unauthenticated and an authenticated request
curl -s -o /dev/null -w "no header: %{http_code}\n" -X POST localhost:2250/threads \
  -H 'content-type: application/json' -d '{}'
curl -s -X POST localhost:2250/threads -H 'content-type: application/json' \
  -H "Authorization: Bearer $DOCS_ASSISTANT_CLIENT_TOKEN" -d '{}'
Output (second response abridged)
no header: 401
{"thread_id":"...","metadata":{"owner":"web-client"},"status":"idle", ...

The token comes from an environment variable and never from the source file. The health check at /ok stayed reachable without a token, which is what an orchestrator’s liveness probe needs. In production, validate a signed token from your identity provider inside authenticate instead of comparing a shared secret, and see Guardrails & Security for the broader picture. The approval gate that protects DocsAssistant’s issue-opening tool is unaffected by any of this: it lives in the graph, so it is the same on every host (see Human-in-the-Loop & Time Travel).

See Authentication & access control.

LangSmith Deployment and self-hosted options

LangSmith Deployment is the packaged, managed way to run an Agent Server. Every option below runs the same Agent Server runtime; what differs is who operates the control plane (which builds and manages deployments) and the data plane (the Agent Servers and their databases), and where.

Option Who runs what Plan requirement

Cloud

LangChain runs the control plane, the data plane and the databases. Create a deployment from a GitHub repository in the LangSmith UI, or with langgraph deploy.

Plus plan or above

Hybrid

LangChain runs the control plane; you run the Agent Servers and their data plane in your own infrastructure. Traces go to LangSmith Cloud or a self-hosted LangSmith.

See LangChain’s pricing page

Self-hosted with control plane

You run the control plane and the Agent Servers in your own Kubernetes cluster, next to a self-hosted LangSmith.

Enterprise plan with LangSmith Deployment enabled

Standalone server

You run Agent Server containers with Docker, Docker Compose or Kubernetes and provide PostgreSQL, Redis and a LangSmith license key. There is no control plane; tracing to LangSmith is optional.

A LangSmith license key

Plans and prices change; LangSmith pricing is deliberately not repeated here (see LangChain’s pricing page). A standalone server is documented as production-ready and the lightest option, with two cautions: do not run it in a serverless environment, where scale-to-zero can lose queued tasks, and use the Helm chart on Kubernetes for production rather than a single Docker host.

The CLI is how a project reaches any of them. langgraph build builds the server image, langgraph dockerfile emits a Dockerfile derived from langgraph.json for custom builds, and langgraph deploy (marked beta in the CLI’s help) builds and pushes an image and creates or updates a deployment in one step, using a LangSmith API key from LANGSMITH_API_KEY (or LANGGRAPH_HOST_API_KEY). langgraph deploy --help also lists deploy list, deploy logs, deploy revisions and deploy delete. langgraph dockerfile needs no Docker daemon, so it was run against DocsAssistant:

Generating a Dockerfile from langgraph.json
langgraph dockerfile Dockerfile
The generated Dockerfile (abridged)
FROM langchain/langgraph-api:3.12

# -- Adding local package . --
ADD . /deps/docs-assistant
# -- Installing all local dependencies --
RUN for dep in /deps/*; do ... uv pip install --system --no-cache-dir -c /api/constraints.txt -e . ...; done
ENV LANGSERVE_GRAPHS='{"docs_assistant": "/deps/docs-assistant/docs_assistant/agent.py:graph"}'
...
WORKDIR /deps/docs-assistant

Two things are visible in it. The base image is langchain/langgraph-api at the Python version from python_version, and langgraph.json’s `graphs map becomes the LANGSERVE_GRAPHS environment variable the server reads on startup. Pin the base image with --base-image or the base_image and api_version keys when you need a reproducible build; by default a build tracks the latest stable server.

Whichever option is chosen, secrets belong in the deployment’s environment configuration or a secret manager, never in langgraph.json or a committed .env; the env key exists for local runs.

See LangSmith Deployment, Self-host standalone servers, Self-hosted LangSmith and LangGraph CLI.

Studio

LangSmith Studio is an agent IDE for any server that implements the Agent Server API, whether it is a local langgraph dev process or a LangSmith deployment. There is nothing to install: the dev server prints a Studio URL (https://smith.langchain.com/studio/?baseUrl=http://127.0.0.1:2247 above) that opens the hosted Studio and points it at your local server, so a LangSmith account is required to use it. Studio was not opened while this page was written, so its behaviour below is taken from the documentation.

  • Graph mode shows the graph’s architecture, the nodes traversed by a run, intermediate state and the links into LangSmith (adding a run to a dataset, opening the playground).

  • Chat mode is a simpler chat surface for graphs whose state includes or extends MessagesState; it suits testing the overall behaviour or handing the agent to a colleague. DocsAssistant’s state carries a messages list, which is the shape chat mode expects, but Studio was not opened to confirm it.

  • It manages assistants, threads and long-term memory, edits prompts, runs experiments over a dataset, supports time-travel debugging (the replay and fork mechanics of Human-in-the-Loop & Time Travel), and can deploy a tested graph to LangSmith Cloud in one click.

Studio ties into the rest of LangSmith, so its runs are traced like any other; how to read those traces is the subject of Observability with LangSmith.

See LangSmith Studio.

Serving the agent yourself with FastAPI

The Agent Server is not the only way to expose a graph. A LangGraph graph is a normal Python object, so a web framework can call it. There are two variants, and choosing between them is the real decision.

Add routes to the Agent Server. If you want the assistants, threads, runs and store API and a few endpoints of your own — a /healthz, a webhook, a small admin API — give langgraph.json a custom ASGI app. Server routes and custom routes then share one process and one deployment.

docs_assistant/webapp.py
from fastapi import FastAPI

app = FastAPI()


@app.get("/healthz")
def healthz():
    return {"status": "ok", "service": "docs-assistant"}
langgraph.json — mounting the app next to the server’s own routes
{
  "dependencies": ["."],
  "graphs": {"docs_assistant": "./docs_assistant/agent.py:graph"},
  "env": "./.env",
  "http": {"app": "./docs_assistant/webapp.py:app"}
}

Started with langgraph dev --config pointing at that file, GET /healthz returned {"status":"ok","service":"docs-assistant"} while GET /ok still returned {"ok":true}. The http block also carries CORS settings, configurable_headers, middleware_order and switches such as disable_meta that remove groups of built-in routes to shrink the exposed surface.

Own the server entirely. If your service already exists, or the whole Agent Server model is more than you need, wire the graph into a FastAPI application yourself. You then own what the server did for you: a durable checkpointer (the snippet uses InMemorySaver to stay self-contained; use PostgresSaver as in Short-Term Memory & Checkpointers), authentication, a work queue for runs that outlive a request, cron scheduling, and streaming.

selfserve.py — DocsAssistant behind a plain FastAPI endpoint
from fastapi import FastAPI
from langgraph.checkpoint.memory import InMemorySaver
from pydantic import BaseModel

from docs_assistant.agent import build_agent

app = FastAPI()
agent = build_agent(checkpointer=InMemorySaver())     # use PostgresSaver in production


class Ask(BaseModel):
    question: str


@app.post("/threads/{thread_id}/ask")
async def ask(thread_id: str, body: Ask):
    result = await agent.ainvoke(
        {"messages": [{"role": "user", "content": body.question}]},
        config={"configurable": {"thread_id": thread_id}},
    )
    return {"answer": result["messages"][-1].content, "messages_on_thread": len(result["messages"])}

Run under Uvicorn (uvicorn selfserve:app) and called twice on the same thread ID, the second answer sees the first:

Two calls on the same thread
curl -s -X POST localhost:2249/threads/t1/ask -H 'content-type: application/json' -d '{"question":"What is ef_search?"}'
curl -s -X POST localhost:2249/threads/t1/ask -H 'content-type: application/json' -d '{"question":"again"}'
Output
{"answer":"According to the docs: HNSW ef_search trades recall for latency (query: What is ef_search?).","messages_on_thread":4}
{"answer":"According to the docs: HNSW ef_search trades recall for latency (query: again).","messages_on_thread":8}

messages_on_thread went from 4 to 8: the checkpointer replayed the first exchange into the second call. Streaming this endpoint to a browser with server-sent events is worked through in Streaming; it is the piece you would write yourself that the Agent Server’s /runs/stream already provides.

Table 3. Choosing how to serve the agent
Need Agent Server (dev, Deployment or standalone) Your own FastAPI

Threads, checkpoints, background runs, crons, streaming

Built in

You build each with a checkpointer, a queue and a scheduler

Human-in-the-loop pauses that outlive a request

Built in: a run stops at an interrupt and resumes later

You persist the checkpoint and expose a resume endpoint

Studio, RemoteGraph, assistants and versioned configuration

Yes

No

Fit with an existing service and its auth, logging and deployment

Add routes through http.app

Native

Operational cost

A Postgres and Redis deployment, or LangSmith’s managed one

Whatever your service already runs on

Serving an agent that answers questions over a retrieval index as a production API — caching, rate limits, scaling the retriever, latency budgets — is the subject of RAG Systems in Production (see Serving RAG as an API, Caching, Latency & Cost and Production RAG Architecture), so it is not repeated here. For the MCP flavour of the same question, see Python & FastAPI Integration and Deploying Remote Servers.

See Add custom routes.

Summary

Table 4. What to remember
Topic Rule

Package

A deployable app is a graph module, a dependency file, an optional .env and langgraph.json (dependencies, graphs, env). Run langgraph validate before anything else.

Graph module

Export a compiled graph; do not configure a checkpointer or store, because the server injects them.

Local

langgraph dev (in-memory, hot reload, no Docker, 127.0.0.1) for daily work; langgraph up (Docker, PostgreSQL, Redis) to validate before release.

Concepts

An assistant is a versioned configuration of a graph; a thread holds one conversation’s state; a run is one invocation, queued and executed by a worker; a cron job schedules runs in UTC.

Clients

langgraph_sdk, plain HTTP with server-sent events, or RemoteGraph.

Security

The dev and self-hosted servers start unauthenticated. Add an auth handler, keep secrets in environment variables, delete cron jobs you no longer need.

Hosting

LangSmith Deployment on Cloud, Hybrid or self-hosted with a control plane, or a standalone Agent Server; the same runtime everywhere.

Alternatives

Custom routes through http.app, or your own FastAPI service with a durable checkpointer.

References