Inference Providers & Spaces

This content was generated with the assistance of AI and should be verified against the official documentation of each tool and framework before being relied on in production.

This section’s bibliography lists the reference material consulted while preparing these pages.

This page was written on 2026-09-30 against huggingface_hub 2.0.0, @huggingface/inference 4.13.30, gradio 6.29.0, smolagents 1.26.0 and openai 3.20.0. Everything else in this sub-section runs models on your own hardware. This page covers the hosted side of the Hub: a quick way to try a model before downloading it, a fallback when a device cannot run it, and a way to share a demo. The free, anonymous serverless Inference API that older tutorials and books call through an api-inference.huggingface.co URL has been decommissioned and replaced by Inference Providers. Code that still posts to the old URL, or that relies on the old hosted-inference widget, is outdated and must move to the router and clients shown below. Hosting Spaces in production (scaling, custom domains, SLAs) is out of scope here.

Hosted inference is billed by the provider or by Hugging Face, depending on the route. This page deliberately quotes no prices or free-credit amounts because they change; see the official Hugging Face — Pricing page and the billing section of the Inference Providers documentation.

Inference Providers

Inference Providers give one Hub-authenticated API in front of many serverless inference companies and Hugging Face’s own hf-inference backend. You send a request for a Hub model ID; the router picks a provider that serves it. The chat API is OpenAI-compatible, so any OpenAI client works by changing the base URL.

flowchart LR App["Your app
HF_TOKEN in env"] -->|"HTTPS"| Router["Router
router.huggingface.co/v1"] Router -->|"policy or provider suffix"| P1["Provider A"] Router --> P2["Provider B"] Router --> P3["hf-inference"] P1 --> M["Model from the Hub"] P2 --> M P3 --> M
Figure 1. Request path through Inference Providers

Authentication

Create a fine-grained access token in the Hub settings and enable the Make calls to Inference Providers permission, then export it. The token belongs in an environment variable (or a secret store), never in source code or a committed file. The code on this page reads $HF_TOKEN only.

Exporting the token
export HF_TOKEN="<paste your fine-grained token here>"   # keep it out of shell history files and Git

Reference: Hugging Face — Inference Providers and Hugging Face — User access tokens.

The OpenAI-compatible router

The router’s base URL is https://router.huggingface.co/v1. The model name is a Hub ID, optionally followed by a suffix that controls provider selection:

Model suffix Meaning

none

Default: the router picks the fastest provider, as with :fastest.

:fastest

The provider with the highest throughput.

:cheapest

The provider with the lowest cost per output token.

:preferred

The first available provider in your provider order in the Hub’s Inference Providers settings.

:<provider>

A specific provider, for example :together or :sambanova.

The policy names above are the ones the documentation lists at the time of writing; provider names change as companies join or leave, so take them from the current docs or from GET https://router.huggingface.co/v1/models.

Chat completion with the openai client
import os
from openai import OpenAI

client = OpenAI(
    base_url="https://router.huggingface.co/v1",
    api_key=os.environ["HF_TOKEN"],
)

completion = client.chat.completions.create(
    model="Qwen/Qwen3-8B:cheapest",  # Hub model ID + policy suffix
    messages=[{"role": "user", "content": "Explain Antora components in two sentences."}],
)
print(completion.choices[0].message.content)

Reference: Hugging Face — Inference Providers and openai-python repository.

Model IDs on this page are illustrative; on 2026-09-30 each one called through the router had at least one live provider mapping on the Hub. Whether a given model is served, and by which providers, is shown in the Inference Providers panel of its Hub page and can change over time, so check it before relying on an ID. The same router is what the Spring AI and LangChain OpenAI integrations call when you set a custom base URL; see LLM Gateways for the gateway pattern.

InferenceClient in Python

huggingface_hub.InferenceClient wraps the router and adds task methods beyond chat. Its provider argument defaults to "auto" (automatic routing following your preferences); you can pass a provider name to pin one.

InferenceClient with the OpenAI-style chat interface
import os
from huggingface_hub import InferenceClient

client = InferenceClient(provider="auto", api_key=os.environ["HF_TOKEN"])

completion = client.chat.completions.create(
    model="Qwen/Qwen3-8B",
    messages=[{"role": "user", "content": "What is a Hugging Face Space?"}],
)
print(completion.choices[0].message.content)

Reference: Hugging Face — InferenceClient reference and Hugging Face — Run inference on servers.

InferenceClient in JavaScript

The JavaScript counterpart is @huggingface/inference (Node.js 24 or a browser-side proxy; never ship the token to a browser).

chatCompletion with @huggingface/inference
import { InferenceClient } from "@huggingface/inference";

const client = new InferenceClient(process.env.HF_TOKEN);

const out = await client.chatCompletion({
  model: "Qwen/Qwen3-8B",
  messages: [{ role: "user", content: "What is a Hugging Face Space?" }],
});
console.log(out.choices[0].message.content);

Reference: Hugging Face — huggingface.js inference.

Inference Endpoints

Inference Providers are shared, pay-per-use and model-catalogue-driven. Inference Endpoints are the opposite end: a dedicated, managed deployment of one model (Hub repository or your own) on hardware you choose, with autoscaling options and private networking. Use it when you need a model the providers do not serve, a fixed capacity, or a private deployment of a fine-tune; pricing is per hardware, so see the pricing page linked above.

Creating a dedicated endpoint with huggingface_hub
from huggingface_hub import create_inference_endpoint

endpoint = create_inference_endpoint(
    "docs-assistant-llm",
    repository="Qwen/Qwen3-8B",
    framework="pytorch",
    task="text-generation",
    accelerator="gpu",
    vendor="aws",
    region="us-east-1",
    type="protected",
    instance_size="x1",
    instance_type="nvidia-a10g",
)
endpoint.wait()          # blocks until the endpoint is running
print(endpoint.url)
endpoint.pause()         # stop billing while idle; resume() or delete() later

The available vendors, regions and instance types change; list them in the Endpoints UI or API before choosing, and read the signature in the current reference. Reference: Hugging Face — Inference Endpoints (huggingface_hub) and Hugging Face — Inference Endpoints documentation.

Gradio 6, Spaces and ZeroGPU

Gradio turns a Python function into a web UI and an HTTP API; a Space is a Hub repository that hosts such an app on Hugging Face infrastructure. Together they are the quickest way to share a demo of a model. ZeroGPU is a Space hardware option that attaches a GPU only while a decorated function runs.

A Gradio 6 app that answers through Inference Providers
import os
import gradio as gr
from huggingface_hub import InferenceClient

client = InferenceClient(api_key=os.environ["HF_TOKEN"])  # set as a Space secret, not in code


def answer(question: str) -> str:
    out = client.chat.completions.create(
        model="Qwen/Qwen3-8B",
        messages=[{"role": "user", "content": question}],
    )
    return out.choices[0].message.content


demo = gr.Interface(fn=answer, inputs="text", outputs="text", title="DocsAssistant demo")

if __name__ == "__main__":
    demo.launch()

Reference: Gradio — Documentation and Hugging Face — Spaces overview.

Tokens used by a Space go in the Space’s secrets settings, which are exposed to the app as environment variables; they are not visible in the repository. To run a local model on a GPU in a ZeroGPU Space, import spaces and decorate the GPU-dependent function; the GPU is requested when the function is called and released when it returns.

ZeroGPU: requesting a GPU only for the decorated function
import spaces
import torch
from transformers import pipeline

pipe = pipeline("automatic-speech-recognition", model="openai/whisper-small.en", device="cuda")


@spaces.GPU(duration=30)   # maximum seconds for one call; shorter durations queue faster
def transcribe(audio_path: str) -> str:
    return pipe(audio_path)["text"]

The decorator has no effect outside ZeroGPU, so the same file runs locally. ZeroGPU works with the Gradio SDK only. Reference: Hugging Face — Spaces ZeroGPU.

Calling a Space from code

Every Gradio app exposes an API. gradio_client.Client calls a Space by its owner/name ID; the Space’s "Use via API" page shows the exact endpoint names and parameters.

Calling a Space with gradio_client
import os
from gradio_client import Client

client = Client("your-username/docs-assistant-demo", token=os.environ["HF_TOKEN"])  # token only for private Spaces
print(client.predict("What is a Space?", api_name="/predict"))

Reference: Gradio — Getting started with the Python client. Spaces are demos, not production back ends: they sleep, queue and change, which is why production hosting is out of scope here.

smolagents in brief

smolagents is Hugging Face’s small agent library. CodeAgent has the model write its actions as Python code. The model wrapper for hosted inference is InferenceClientModel; older material and books call it HfApiModel, which is the former name, so update such code.

A CodeAgent on Inference Providers
from smolagents import CodeAgent, InferenceClientModel

model = InferenceClientModel(model_id="Qwen/Qwen3-8B")  # uses HF_TOKEN from the environment
agent = CodeAgent(tools=[], model=model)
print(agent.run("How many seconds are there in a week?"))

Generated code runs in a sandbox only if you configure one; see the security guidance in the docs. Agent design in general is covered in AI Agents. Reference: Hugging Face — smolagents.

Scenario: DocsAssistant goes offline, with a cloud fallback

The running scenario of this sub-section is the DocsAssistant app performing four tasks with local models. Here the same four tasks run in the cloud with InferenceClient, as the fallback when a device cannot run a model or while prototyping before you pick one to download. Task support depends on the provider and the model: not every provider serves every task, so check the task page and the model’s provider list, and be ready for an error or an empty result.

Transcribe a spoken question (automatic speech recognition)
import os
from huggingface_hub import InferenceClient

client = InferenceClient(provider="auto", api_key=os.environ["HF_TOKEN"])
asr = client.automatic_speech_recognition("question.wav", model="openai/whisper-large-v3")
question = asr.text

Reference: Inference Providers — Automatic Speech Recognition.

Answer with chat completion
reply = client.chat.completions.create(
    model="Qwen/Qwen3-8B",
    messages=[
        {"role": "system", "content": "You answer questions about the product documentation."},
        {"role": "user", "content": question},
    ],
).choices[0].message.content

Reference: Inference Providers — Chat Completion.

Speak the answer (text to speech)
audio_bytes = client.text_to_speech(reply, model="hexgrad/Kokoro-82M")
with open("answer.audio", "wb") as f:
    f.write(audio_bytes)   # the container format depends on the provider; inspect it before playing

Text-to-speech has the narrowest provider support of the four tasks; if no provider serves the chosen model, pick another model that a provider serves for that task. Reference: Hugging Face — Inference Providers.

Classify a screenshot (image classification)
labels = client.image_classification("screenshot.png", model="google/vit-base-patch16-224")
for item in labels[:3]:
    print(item.label, item.score)

Reference: Inference Providers — Image Classification.

Describe a screenshot with a vision-language model over chat completion
import base64

with open("screenshot.png", "rb") as f:
    data_url = "data:image/png;base64," + base64.b64encode(f.read()).decode()

description = client.chat.completions.create(
    model="Qwen/Qwen2.5-VL-7B-Instruct",   # any VLM that a provider serves for chat completion
    messages=[{
        "role": "user",
        "content": [
            {"type": "text", "text": "Describe this screenshot in one sentence."},
            {"type": "image_url", "image_url": {"url": data_url}},
        ],
    }],
).choices[0].message.content

Reference: Inference Providers — Chat Completion (vision).

Screenshots and recordings leave the device in this fallback, so the decision to fall back is a privacy decision, not only an availability one. The next section gives the criteria.

When to prefer local

Hosted inference is the faster start; local inference is often the better steady state. The deciding factors:

Factor Prefer local when Prefer hosted when

Privacy and compliance

Data must not leave the device or network (screenshots, recordings, customer text).

The data is public or already covered by an agreement with the provider.

Offline use

The app must work without connectivity.

Connectivity is guaranteed.

Latency

Network round trips dominate short requests, or you need streaming from the first token on-device.

The model is too large for the device and the round trip is acceptable.

Cost predictability

Volume is high and steady, so hardware is amortised; cost does not grow per request.

Volume is low or spiky; pay-per-use beats idle hardware.

Licences

The licence allows redistribution or on-device use and you must ship the weights.

You only call a service and the provider’s terms suffice.

Model size and quality

A small or quantized model meets the quality bar.

You need a model larger than your hardware can hold.
flowchart TD Start["New model feature"] --> Priv{"Data must stay
on device?"} Priv -->|"yes"| Local["Run locally"] Priv -->|"no"| Off{"Must work offline?"} Off -->|"yes"| Local Off -->|"no"| Fit{"Model fits
the target hardware?"} Fit -->|"no"| Cloud["Inference Providers
or Inference Endpoints"] Fit -->|"yes"| Vol{"High, steady volume?"} Vol -->|"yes"| Local Vol -->|"no"| Proto["Prototype hosted,
then decide"]
Figure 2. Choosing between local and hosted inference

A common pattern is hybrid: local by default, hosted as the fallback shown in the scenario above, behind one interface so the choice is configuration. The rest of this sub-section covers the local side:

The same scenario on each platform is in Apple Platforms, Android, Windows & .NET and Web & Cross-Platform.

References