Google's OpenAI-compat shim has no reasoning channel; thought text leaks into content

high · inference · probed live 2026-07-26 · affects Google (OpenAI-compat)

Reasoning-capable serving separates the model's thinking from its answer: thought text arrives in a dedicated field (reasoning_content on OpenAI-dialect providers; a "thought": true part on Google's native API), and content holds only what the application should show. Google's OpenAI-compat shim has no such field — and it isn't for lack of asking: the request below sends reasoning_effort: "high", the shim accepts it (200), and still returns zero reasoning. The thought text lands inside literal <thought> tags in the answer field.

What was sent

(download)

{
  "model": "gemma-4-31b-it",
  "messages": [
    {
      "role": "user",
      "content": "Think of a number between 1 and 100. Don't say it out loud — keep it in your thoughts and reply with just 'ok'."
    }
  ],
  "reasoning_effort": "high",
  "temperature": 0,
  "max_tokens": 600,
  "stream": true,
  "stream_options": {
    "include_usage": true
  }
}

What came back

HTTP 200. The message object carries only role and content — no reasoning or reasoning_content key exists:

{
  "role": "assistant",
  "content": "<thought>*   Task: Think of a number between 1 and 100.\n    *   Constraint 1: Don't say it out loud (don't reveal it).\n    *   Constraint 2: Reply with just 'ok'.\n\n    *   I need to pick a number. Let's pick 42.\n    *   I must not mention 42.\n    *   I must reply with exactly 'ok'.</thought>ok"
}

The number the prompt asked the model to withhold — 42 — is verbatim in the user-facing answer field. Reproduced byte-identically across 4 runs.

The same model, same host, native protocol

The identical prompt to …/models/gemma-4-31b-it:generateContent with thinking on returns the thought in a separate, marked part:

{
  "parts": [
    {
      "text": "*   Task: Think of a number between 1 and 100.\n    *   Constraint 1: Don't say it out loud (don't reveal it).\n    *   Constraint 2: Reply with just 'ok'.\n\n    *   I need to pick a number. Let's pick 42.\n    *   I must not mention 42.\n    *   I must reply with exactly 'ok'.",
      "thought": true
    },
    { "text": "ok" }
  ],
  "role": "model"
}

The thought text is identical between the two lanes; only the framing differs. Native marks it "thought": true in a separate part; the shim concatenates it into content. Same vendor, same host, same model — a shim translation defect, not model behavior.

Why it matters

Every consumer that treats content as the answer displays or logs the model's private reasoning — including things the prompt explicitly asked the model to keep to itself. Stripping <thought> tags client-side means relying on an undocumented in-band sentinel Google never promised and can change without notice.

Run it yourself

curl "https://generativelanguage.googleapis.com/v1beta/openai/chat/completions" \
  -H "Authorization: Bearer $GEMINI_API_KEY" -H "Content-Type: application/json" \
  -d @reasoning-leak.google-openai.json

Expected: content: "ok" with reasoning in a dedicated field. Received: the <thought>…</thought>ok string shown.