Hetzner Launches Free Experimental OpenAI-Compatible LLM Inference API
Hetzner has launched an experimental OpenAI-compatible inference API hosted on its European data center infrastructure. Developers can test Qwen3.6-35B-A3B-FP8 with low latency using standard OpenAI SDKs or OpenCode.
Impact: Medium
Why it matters
You can point your agent frameworks and local coding assistants to a fast, free European inference endpoint right now.
TL;DR
- 01Hetzner offers a free OpenAI-compatible endpoint hosting Qwen3.6-35B-A3B-FP8 with 262K context.
- 02Measured generation speeds exceed 220 output tokens per second with a 153ms median TTFT.
- 03Undocumented `enable_thinking` request parameter allows toggling reasoning output.
Key facts
- Hosted Model
- Qwen/Qwen3.6-35B-A3B-FP8
- Context Window
- 262,000 tokens
- Median TTFT
- 153 ms
- Generation Speed
- 224 tokens/sec
- Price
- Free during experiment
Experimental Endpoint Setup
Hetzner exposes an OpenAI-compatible base URL at https://inference.hetzner.com/api/v1. To use it, generate an API token inside the Hetzner Experiments dashboard and configure standard client libraries.
Model Capabilities and Benchmarks
The hosted Qwen/Qwen3.6-35B-A3B-FP8 payload occupies ~38 GB of VRAM. Initial benchmarks indicate:
- Time to First Token (TTFT): 153 ms median over established connections.
- Generation Speed: 224 tokens/sec capped at 512 generation tokens.
- Reasoning Flag: Passing
enable_thinking: trueallows the model to output intermediate reasoning steps before returning visible answers.
Try it in 2 minutes
from openai import OpenAI
client = OpenAI(
base_url="https://inference.hetzner.com/api/v1",
api_key="YOUR_HETZNER_EXPERIMENTS_TOKEN"
)
response = client.chat.completions.create(
model="Qwen/Qwen3.6-35B-A3B-FP8",
messages=[{"role": "user", "content": "Explain LLM prompt caching in one clear sentence."}]
)
print(response.choices[0].message.content)python
✓ When to use
- Benchmarking low-cost open-weights inference against proprietary APIs.
- Running low-latency local agent loops using OpenAI SDK configurations.
✕ When NOT to use
- Production workloads requiring formal SLAs or uptime guarantees.
- Strict regulatory environments requiring documented compliance commitments.
What to do today
- Generate an API token in the Hetzner Experiments dashboard.
- Update OpenAI base_url to https://inference.hetzner.com/api/v1 in local agent test suites.
- Test OpenCode integration against the free European endpoint.
What the community says
“Interesting, didn’t know about them! Weird model selection though, no glm or deepseek?”
Sources