Inference API

Note: This service is provided for experimental purposes only and is offered "as is". We have no control over or liability for their behavior.
Performance and availability are not guaranteed, especially during periods of high demand. No backups are created, so you are responsible for backing up your own configurations and should not use the platform for production environments.

The Inference API provides access to open-source large language models via an OpenAI-compatible REST API. You can use it with any OpenAI-compatible SDK or by manually crafting requests with anHTTP client.

Base URL: https://inference.hetzner.com/api/v1

Available Models

You can query the list of available models at any time using the /v1/models endpoint:

curl -s https://inference.hetzner.com/api/v1/models \
  -H "Authorization: Bearer <YOUR_TOKEN>"

Python OpenAI Examples

All examples use the OpenAI Python SDK. Install it with:

pip install openai

Initialize the client:

from openai import OpenAI

client = OpenAI(
    base_url="https://inference.hetzner.com/api/v1",
    api_key="<YOUR_TOKEN>",
)

Chat Completions

Send a conversation and receive a model-generated response.

response = client.chat.completions.create(
    model="Qwen/Qwen3.6-35B-A3B-FP8",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "Please give me a very short description of Hetzner services!"},
    ]
)

print(response.choices[0].message.content)

Image Processing

Send an image URL alongside your prompt to use the models vision capabilities.

response = client.chat.completions.create(
    model="Qwen/Qwen3.6-35B-A3B-FP8",
    messages=[
        {
            "role": "user",
            "content": [
                {"type": "text", "text": "Describe this image in detail."},
                {
                    "type": "image_url",
                    "image_url": {"url": "https://cdn.hetzner.de/cdn/public/Uploads/Finnland_Luftaufnahme-v2.jpg"},
                },
            ],
        }
    ],
)

print(response.choices[0].message.content)

Curl Examples

Chat Completion

curl -s https://inference.hetzner.com/api/v1/chat/completions \
  -H "Authorization: Bearer <YOUR_TOKEN>" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "Qwen/Qwen3.6-35B-A3B-FP8",
    "messages": [
      {"role": "system", "content": "You are a helpful assistant."},
      {"role": "user", "content": "What are the three laws of thermodynamics?"}
    ]
  }'

Image Processing

curl -s https://inference.hetzner.com/api/v1/chat/completions \
  -H "Authorization: Bearer <YOUR_TOKEN>" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "Qwen/Qwen3.6-35B-A3B-FP8",
    "messages": [
      {
        "role": "user",
        "content": [
          {"type": "text", "text": "Describe this image in detail."},
          {
            "type": "image_url",
            "image_url": {"url": "https://cdn.hetzner.de/cdn/public/Uploads/Finnland_Luftaufnahme-v2.jpg"}
          }
        ]
      }
    ]
  }'

FAQ

How do I get an API key?

Go to https://experiments.hetzner.com/inference - in the top right hand corner there is a red button Create API Token.

What limits apply?

The API enforces rate limits to ensure fair usage and service stability. Current limits are applied per API key:

If you exceed any of the rate limits, the API will respond with HTTP Status Code 429.

Why is there only one model?

The Inference API is currently in experimental status. We are starting with a single high-quality model to validate the platform, gather feedback, and ensure reliability.
Additional models will be added based on demand and operational experience.

How can I give feedback?

We'd love to hear from you! Please send us your feedback via a ticket here. Whether it's a bug report, a feature request, or general impressions.

Open-Source Model Licenses & Attribution

The models available through this API are open-source and subject to their respective licenses. By using this API, you acknowledge the applicable license terms of the underlying model(s).

All models are provided by their respective developers "AS IS", without warranties of any kind. See each model's license for full terms.