Setting the file. One moment.
Subchapter 19.10
references/inference-providers.mdMarkdown4 KBView on GitHub
Some Spaces don’t need a GPU at all. If the model is available through HF Inference Providers (Cerebras, Fireworks, Together, Replicate, OpenRouter, etc.), the Space can be a thin Gradio shell that proxies to a hosted endpoint:
Also bundled
README@spaces.GPU, no model download.curl "https://huggingface.co/api/models/<ns>/<repo>?expand[]=inferenceProviderMapping"For those, host the model yourself on ZeroGPU — see zerogpu.md.
Choose based on who pays for inference.
Set HF_TOKEN as a Space secret. The Space uses InferenceClient directly. Every visitor’s call is billed to the Space creator’s account.
import os, gradio as gr
from huggingface_hub import InferenceClient
client = InferenceClient(api_key=os.environ["HF_TOKEN"], provider="fireworks-ai")
def chat(msg, history):
return client.chat_completion(
model="<org>/<model>",
messages=[*history, {"role": "user", "content": msg}],
max_tokens=512,
).choices[0].message.content
gr.ChatInterface(chat).launch()Use when you want users to “just click and try it” — no sign-in friction. Cost is on you.
gr.LoginButton + gr.load("models/...") with accept_token=button. Each visitor signs in with their HF account; inference is billed to their account.
import gradio as gr
with gr.Blocks(fill_height=True) as demo:
with gr.Sidebar():
button = gr.LoginButton("Sign in")
gr.load("models/<org>/<model>", accept_token=button, provider="fireworks-ai")
demo.launch()README frontmatter needs:
hf_oauth: true
hf_oauth_scopes:
- inference-apiThis is the recommended pattern for public demos — sustainable cost-wise, and visitors get to use their own provider quotas (which most have paid for or get free).
No GPU needed, so cpu-basic is the natural fit — but it requires a paid plan.
On a free account, create the Space with --flavor zero-a10g instead. ZeroGPU refuses to start without at least one decorated function, so add a no-op one and leave the provider calls outside it:
import spaces
@spaces.GPU(duration=1)
def _noop(): # ZeroGPU requires ≥1 decorated function; never called
passNothing ever requests a GPU, so no quota is burned. Just remember the Space still counts against the free 2-Space ZeroGPU cap — don’t spend a slot here if the user is saving it for a real GPU demo.
If you do use Inference Providers, do not wrap the call in @spaces.GPU. The decorator reserves a GPU slot on your Space for the full duration=, but the function does no GPU work — just an HTTP call out. You burn your own ZeroGPU quota for nothing.
Whatever hardware a provider-proxy Space sits on, no provider call belongs inside @spaces.GPU.