Novita silently ignores tool_choice

high · inference · probed live 2026-07-26 · affects Novita

tool_choice is the Chat Completions field an application uses to constrain tool calling: "required" guarantees at least one tool call, and naming a function forces a call to exactly that function. Novita accepts both constraints and simply ignores them. Only "none" is honored — the one sign the field is read at all; whether it is validated like DeepInfra's is untested.

What was measured

tool_choice: "required" on a prose-inviting prompt (download):

tool-choice-required.novita.json

{
  "model": "google/gemma-4-31b-it",
  "messages": [
    {
      "role": "user",
      "content": "Just say hello."
    }
  ],
  "tools": [
    {
      "type": "function",
      "function": {
        "name": "log_event",
        "description": "Logs an event message.",
        "parameters": {
          "type": "object",
          "properties": {
            "message": {
              "type": "string"
            }
          }
        }
      }
    }
  ],
  "tool_choice": "required",
  "chat_template_kwargs": {
    "enable_thinking": false
  },
  "temperature": 0,
  "max_tokens": 200
}

Received — HTTP 200:

response · HTTP 200

{
  "id": "3c134565d28fa4229d6f1b203a3cab71",
  "object": "chat.completion",
  "created": 1785075405,
  "model": "google/gemma-4-31b-it",
  "choices": [
    {
      "index": 0,
      "message": { "role": "assistant", "content": "Hello!" },
      "finish_reason": "tool_calls"
    }
  ],
  "usage": { "prompt_tokens": 58, "completion_tokens": 3, "total_tokens": 61 }
}

Prose, no tool_calls key at all — under finish_reason: "tool_calls".

Forcing get_time by name on a weather prompt (download):

tool-choice-specific.novita.json

{
  "model": "google/gemma-4-31b-it",
  "messages": [
    {
      "role": "user",
      "content": "What's the weather in Paris right now?"
    }
  ],
  "tools": [
    {
      "type": "function",
      "function": {
        "name": "get_weather",
        "description": "Current weather for a city.",
        "parameters": {
          "type": "object",
          "properties": {
            "city": {
              "type": "string"
            }
          }
        }
      }
    },
    {
      "type": "function",
      "function": {
        "name": "get_news",
        "description": "Latest news on a topic.",
        "parameters": {
          "type": "object",
          "properties": {
            "topic": {
              "type": "string"
            }
          }
        }
      }
    },
    {
      "type": "function",
      "function": {
        "name": "get_time",
        "description": "Current time in a timezone.",
        "parameters": {
          "type": "object",
          "properties": {
            "timezone": {
              "type": "string"
            }
          }
        }
      }
    }
  ],
  "tool_choice": {
    "type": "function",
    "function": {
      "name": "get_time"
    }
  },
  "chat_template_kwargs": {
    "enable_thinking": false
  },
  "temperature": 0,
  "max_tokens": 200
}

Received — HTTP 200, a call to get_weather instead:

response · HTTP 200, choices[0]

{
  "index": 0,
  "message": {
    "role": "assistant",
    "content": "",
    "tool_calls": [
      {
        "id": "chatcmpl-tool-a81f9268ff103aec",
        "type": "function",
        "function": { "name": "get_weather", "arguments": "{\"city\": \"Paris\"}" }
      }
    ]
  },
  "finish_reason": "tool_calls"
}

The failure signature — wrong tool, same arguments, same self-contradicting finish_reason — is byte-for-byte the DeepInfra signature (that finding).

How this breaks the contract — and why it matters

200s that satisfy neither guarantee, detectable only by comparing the returned tool name against the one you forced. Workflows that depend on forced tool calls — structured extraction, guaranteed tool loops — silently receive prose or a call with different semantics.

Run it yourself

curl

curl "https://api.novita.ai/openai/chat/completions" -H "Authorization: Bearer $KEY" \
  -H "Content-Type: application/json" -d @tool-choice-required.novita.json

Expected: a log_event call, then (with the second body) a get_time call. Received: prose, then get_weather.