# SambaNova enforces tool_choice by post-hoc 400, not constrained decoding

high · inference · probed live 2026-07-26 · affects SambaNova ·
HTML version: [https://inferencecanary.com/sambanova-tool-choice-400](https://inferencecanary.com/sambanova-tool-choice-400)

`tool_choice` is the Chat Completions field an application uses to constrain tool calling:
`"required"` guarantees the reply contains at least one tool call, and naming a function forces
a call to exactly that function. The guarantee is the server's to keep — normally by
constrained decoding, where the model is only allowed to generate output satisfying the
constraint. SambaNova accepts the field, lets the model generate freely, and when the output
happens to violate the constraint, fails the request with a 400 error. Whether your request
succeeds depends on what the model felt like emitting.

## What was measured

Three arms at temperature 0, thinking disabled: `tool_choice: "none"`, `"required"`, and
forced-specific. `"none"` passes — there is nothing to violate. The other two arms, request
and response verbatim.

`"required"` with a prompt that invites prose
([https://inferencecanary.com/assets/repro/tool-choice-required.sambanova.json](https://inferencecanary.com/assets/repro/tool-choice-required.sambanova.json)):

```json
{
  "model": "gemma-4-31B-it",
  "messages": [
    {
      "role": "user",
      "content": "Just say hello."
    }
  ],
  "tools": [
    {
      "type": "function",
      "function": {
        "name": "log_event",
        "description": "Logs an event message.",
        "parameters": {
          "type": "object",
          "properties": {
            "message": {
              "type": "string"
            }
          }
        }
      }
    }
  ],
  "tool_choice": "required",
  "chat_template_kwargs": {
    "enable_thinking": false
  },
  "temperature": 0,
  "max_tokens": 200
}
```

Received — HTTP **400**, after 2.45 s (generation ran to completion before the rejection):

```json
{
  "error": "Specified tool_choice: \"required\", but the model output contains zero valid function call.",
  "error_code": null,
  "error_model_output": "Hello!",
  "error_param": null,
  "error_type": "Invalid function calling output."
}
```

Forcing `get_time` by name on a weather prompt, with `get_weather`, `get_news` and `get_time`
offered
([https://inferencecanary.com/assets/repro/tool-choice-specific.sambanova.json](https://inferencecanary.com/assets/repro/tool-choice-specific.sambanova.json)):

```json
{
  "model": "gemma-4-31B-it",
  "messages": [
    {
      "role": "user",
      "content": "What's the weather in Paris right now?"
    }
  ],
  "tools": [
    {
      "type": "function",
      "function": {
        "name": "get_weather",
        "description": "Current weather for a city.",
        "parameters": {
          "type": "object",
          "properties": {
            "city": {
              "type": "string"
            }
          }
        }
      }
    },
    {
      "type": "function",
      "function": {
        "name": "get_news",
        "description": "Latest news on a topic.",
        "parameters": {
          "type": "object",
          "properties": {
            "topic": {
              "type": "string"
            }
          }
        }
      }
    },
    {
      "type": "function",
      "function": {
        "name": "get_time",
        "description": "Current time in a timezone.",
        "parameters": {
          "type": "object",
          "properties": {
            "timezone": {
              "type": "string"
            }
          }
        }
      }
    }
  ],
  "tool_choice": {
    "type": "function",
    "function": {
      "name": "get_time"
    }
  },
  "chat_template_kwargs": {
    "enable_thinking": false
  },
  "temperature": 0,
  "max_tokens": 200
}
```

Received — HTTP **400**:

```json
{
  "error": "Specified tool_choice with function name \"get_time\", but the model output contains function call to another tool with function name \"get_weather\".",
  "error_code": null,
  "error_model_output": "<|tool_call>call:get_weather{city:<|\"|>Paris<|\"|>}<tool_call|>",
  "error_param": null,
  "error_type": "Invalid function calling output."
}
```

## How this breaks the contract

A valid request with a valid constraint must return 200 with a conforming `tool_calls` array;
a 4xx is for malformed requests. Here the error reports a *model output* failure on a
well-formed request — and the `error_model_output` field is raw template text, not a
`tool_calls` structure an application could repair from. The model did emit a valid
`get_weather` call in the second case; the provider discarded it and returned an error
instead.

## Why it matters

Forced-tool workflows that pass in testing throw unrecoverable runtime errors in production,
timed to whenever the model's free-run output happens to disagree with the constraint. There
is nothing to catch and retry against except the same coin-flip.

## Run it yourself

Save either body above and POST it with your key:

```
curl "https://api.sambanova.ai/v1/chat/completions" -H "Authorization: Bearer $KEY" \
  -H "Content-Type: application/json" -d @tool-choice-required.sambanova.json
```

A conforming provider returns 200 with a `log_event` (or `get_time`) call. SambaNova returns
the 400 shown whenever the model's unconstrained output violates the constraint.