Hetzner is giving away inference. The models spent it thinking.

Hetzner is handing out inference. Four open-weight models behind an OpenAI-compatible API at inference.hetzner.com, free while it stays experimental, no SLA, and an announcement that reads: “No promises this becomes a permanent product.”

I gave each model the same 38-token prompt and 512 tokens to answer it. Three of the four came back with no answer at all.

Where the tokens went

Qwen 3.6, DeepSeek V4 and GLM 5.2 each produced exactly 512 completion tokens and zero visible words. The budget went entirely into thinking. The API reports this honestly: finish_reason: length, and a couple of thousand characters of reasoning. But message.content comes back null, and a client that reads only that field sees a successful request that said nothing.

Only Kimi K2.7 answered, with 266 words and a comparatively frugal 568 characters of thinking.

That is what happens when reasoning models meet a token budget picked for non-reasoning ones, and it is the first thing to know before wiring this into anything. A token budget is a budget in tokens, not in words, and what a token costs is not obvious before you count.

Two places a stock client breaks

“OpenAI-compatible” is doing some work in that sentence.

The stream frames are data:{...} with no space after the colon. Server-sent events allow it, and the OpenAI SDK handles it. A hand-rolled parser that splits on the documented data: prefix reads zero tokens and reports a working request with empty output. That is exactly what my first attempt did, and for a few minutes I believed the models were the problem. Same shape as the reviewer that filed a fabricated finding: the tool reported success, and the only way to know better was to check its output against something outside it.

Thinking arrives on delta.reasoning. vLLM and the OpenAI SDK use reasoning_content. If you count only that field you measure zero throughput on three of these four models, while the tokens are being generated, billed against your rate limit, and thrown away by your own parser.

Throughput is fine. Waiting is the problem.

Once tokens start flowing, the numbers are respectable for free capacity: Qwen 3.6 held 41 tokens per second, DeepSeek V4 38, Kimi K2.7 30.

The wait before that is where it falls apart. Median time to first token ran from 2.1 seconds for Qwen to 26.8 seconds for GLM 5.2. GLM’s five runs, in the order they happened:

• 122.94s · 172.65s · 26.77s · 18.09s · 11.71s

A model does not get fifteen times faster over five minutes. That spread is a queue draining, and it means the number I could quote for GLM depends entirely on when I asked. Not my queue: the whole benchmark spent 17,920 output tokens against a limit of 200,000 per minute, nothing returned 429, and GLM ran last. If I had been the congestion it would have got slower, not faster. The other three are steadier, with Qwen’s five runs between 1.68 and 2.20 seconds, but the same caveat applies to all of them at a smaller scale.

It batches well

Firing eight 256-token requests at Qwen 3.6 at once did not slow any single one down: per-request throughput stayed around 36 tokens per second against 34 for a lone request, while aggregate throughput went from 15 to 217 tokens per second. That is the shape of a backend built to batch, and it is the case where this API is attractive: offline work, bulk classification, anything where nobody is watching a cursor blink.

DeepSeek V4 was much noisier under the same treatment, dropping to 15 tokens per second per request at eight parallel and back up at four. With five samples I would not build an argument on that; I report it because the noise is itself the finding on a service with no capacity guarantee.

The part worth copying

The site’s repository is private, so a pointer at a file in it is no use to you. Here is the piece that carries both findings — the streaming parser and the timing. Everything else in my runner is argument handling and medians.

const res = await fetch(`${BASE}/chat/completions`, {
  method: 'POST',
  headers: { Authorization: `Bearer ${KEY}`, 'Content-Type': 'application/json' },
  body: JSON.stringify({
    model, messages: [{ role: 'user', content: PROMPT }],
    max_tokens: 512, temperature: 0,
    stream: true, stream_options: { include_usage: true },
  }),
});

const reader = res.body.getReader();
const decoder = new TextDecoder();
let buffer = '', first = null, last = null, visible = '', reasoning = '', usage = null;

for (;;) {
  const { done, value } = await reader.read();
  if (done) break;
  buffer += decoder.decode(value, { stream: true });
  const lines = buffer.split('\n');
  buffer = lines.pop();
  for (const line of lines) {
    // 'data:' with no trailing space. Splitting on the documented 'data: '
    // reads zero tokens and reports a working request with empty output.
    if (!line.startsWith('data:')) continue;
    const payload = line.slice(5).trim();
    if (!payload || payload === '[DONE]') continue;
    const frame = JSON.parse(payload);
    if (frame.usage) usage = frame.usage;
    const delta = frame.choices?.[0]?.delta;
    if (!delta) continue;
    const text = delta.content ?? '';
    // delta.reasoning, not the reasoning_content vLLM and the OpenAI SDK use.
    // Counting only delta.content measures zero on three of the four models.
    const think = delta.reasoning ?? delta.reasoning_content ?? '';
    if (!text && !think) continue;
    if (first === null) first = performance.now();
    last = performance.now();
    visible += text;
    reasoning += think;
  }
}

// Active throughput excludes the wait. End-to-end divides by the whole request.
const activeTps = usage.completion_tokens / ((last - first) / 1000);

Point it at any OpenAI-compatible endpoint. If you run it against this API and get different numbers, that is the finding, not a contradiction — see the caveats below.

What this is and is not

• One client, one location, one afternoon. A residential connection in Berlin, five runs per model. Against a service that states it has no SLA, on hardware shared with everyone else who read the same announcement.
• It measures what one user got, not what the hardware can do. Every number here would move on a different day, and GLM’s would move a lot.
• The method is the transferable part. The measurement loop is above, in full. It writes both the raw JSON and the module this figure reads, so a re-run moves the chart rather than leaving it to be copied across by hand.

Would I use it

For batch work where latency does not matter: yes, and gladly. Free capacity on European hardware with a stated policy of not storing request content is a genuinely good offer, and the throughput holds up under load.

For anything interactive: not until the first-token wait stops depending on the hour. And not through a stock OpenAI client, until delta.reasoning either becomes reasoning_content or the docs say plainly that it is not.

Both of those are the kind of thing an experimental platform exists to find out, which is presumably the point of shipping it this way. The announcement asks for exactly this feedback.

Next

The obvious missing measurement is a longer budget. Every number above is bounded by 512 tokens, which is the constraint that produced the headline finding. At 4,096 the three thinking models would answer, and the interesting question becomes how much of the budget the thinking costs rather than whether it eats all of it.

The second is time of day. Five runs in one afternoon cannot separate a slow model from a busy one, and GLM 5.2’s spread says that distinction is the whole story for at least one of these. That wants the same script on a schedule for a week, which is a different post.

If you have pointed a client at this API: did you get visible output on the first try, or did you also spend a while believing the models were broken? I would like to know whether the empty content is a thing everyone hits or a thing I walked into by picking 512.