DeepInfra validates tool_choice but does not enforce it

high · inference · probed live 2026-07-26 · affects DeepInfra

DeepInfra parses and validates the tool_choice field — send an invalid value and you get a 400, force a tool that isn't in tools and you get a 422. Every surface signal says the capability exists. But a valid constraint has zero effect on what the model generates. Validation without enforcement is worse than ignoring the field outright: nothing warns you the guarantee is absent.

What was measured

tool_choice: "required" on a prose-inviting prompt (download):

tool-choice-required.deepinfra.json

{
  "model": "google/gemma-4-31B-it",
  "messages": [
    {
      "role": "user",
      "content": "Just say hello."
    }
  ],
  "tools": [
    {
      "type": "function",
      "function": {
        "name": "log_event",
        "description": "Logs an event message.",
        "parameters": {
          "type": "object",
          "properties": {
            "message": {
              "type": "string"
            }
          }
        }
      }
    }
  ],
  "tool_choice": "required",
  "chat_template_kwargs": {
    "enable_thinking": false
  },
  "temperature": 0,
  "max_tokens": 200
}

Received — HTTP 200, prose, zero tool calls — and a self-contradicting envelope:

response · HTTP 200, choices[0]

{
  "index": 0,
  "message": {
    "role": "assistant",
    "content": "Hello!",
    "reasoning_content": null,
    "name": null,
    "tool_calls": null
  },
  "finish_reason": "tool_calls",
  "logprobs": null
}

finish_reason: "tool_calls" with tool_calls: null: the constraint reached the layer that stamps the finish reason, and never reached decoding.

Forcing get_time by name on a weather prompt, with get_weather, get_news and get_time offered (download):

tool-choice-specific.deepinfra.json

{
  "model": "google/gemma-4-31B-it",
  "messages": [
    {
      "role": "user",
      "content": "What's the weather in Paris right now?"
    }
  ],
  "tools": [
    {
      "type": "function",
      "function": {
        "name": "get_weather",
        "description": "Current weather for a city.",
        "parameters": {
          "type": "object",
          "properties": {
            "city": {
              "type": "string"
            }
          }
        }
      }
    },
    {
      "type": "function",
      "function": {
        "name": "get_news",
        "description": "Latest news on a topic.",
        "parameters": {
          "type": "object",
          "properties": {
            "topic": {
              "type": "string"
            }
          }
        }
      }
    },
    {
      "type": "function",
      "function": {
        "name": "get_time",
        "description": "Current time in a timezone.",
        "parameters": {
          "type": "object",
          "properties": {
            "timezone": {
              "type": "string"
            }
          }
        }
      }
    }
  ],
  "tool_choice": {
    "type": "function",
    "function": {
      "name": "get_time"
    }
  },
  "chat_template_kwargs": {
    "enable_thinking": false
  },
  "temperature": 0,
  "max_tokens": 200
}

Received — HTTP 200, one call, to the wrong tool:

response · HTTP 200, choices[0]

{
  "index": 0,
  "message": {
    "role": "assistant",
    "content": "",
    "reasoning_content": null,
    "name": null,
    "tool_calls": [
      {
        "id": "chatcmpl-tool-a52fe694592aca60",
        "type": "function",
        "function": {
          "name": "get_weather",
          "arguments": "{\"city\": \"Paris\"}"
        }
      }
    ]
  },
  "finish_reason": "tool_calls",
  "logprobs": null
}

get_time was forced; get_weather — the obvious tool for the prompt — came back. That is the signature of the constraint being dropped, not misread.

How this breaks the contract

"required" obliges at least one tool call; a named tool_choice obliges a call to that exact function. DeepInfra returns 200 on both and satisfies neither — and because the field is validated at the surface, every cheap client-side check (no 4xx, plausible finish_reason, well-formed JSON) reports success.

Why it matters

Structured extraction and guaranteed tool loops built on forced calls silently get prose, or a call to a different tool with different semantics. The failure only shows up when downstream code tries to use a result that never arrived.

Run it yourself

curl

curl "https://api.deepinfra.com/v1/openai/chat/completions" -H "Authorization: Bearer $KEY" \
  -H "Content-Type: application/json" -d @tool-choice-required.deepinfra.json

Three checks: (1) "required" on "Just say hello." — watch prose come back under finish_reason: "tool_calls"; (2) forced get_time — watch get_weather return; (3) the validation half: change tool_choice to "banana" — 400, or force a name not in tools — 422. That last pair proves the field is read, which makes the non-enforcement a choice, not a parsing gap.