# SambaNova ships raw chain-of-thought as the answer when truncation lands mid-thought

medium · inference · probed live 2026-08-02 · affects SambaNova ·
HTML version: [https://inferencecanary.com/sambanova-truncation-leak](https://inferencecanary.com/sambanova-truncation-leak)

With thinking enabled, the model emits its reasoning first, closes the thought channel with a
template marker, then writes the answer. SambaNova only separates the two when that closing
marker actually arrives: if `max_tokens` cuts generation off mid-thought, the entire raw
thought channel — opening marker included — is returned in `content`, the field applications
display as the answer, while the `reasoning` field comes back empty. The answer itself never
arrives.

## What was sent

An arithmetic prompt with thinking on, `temperature: 0`, and a `max_tokens` budget of 60 —
small enough that the cut lands inside the thought
([https://inferencecanary.com/assets/repro/think-truncation.sambanova.json](https://inferencecanary.com/assets/repro/think-truncation.sambanova.json)):

```json
{
  "model": "gemma-4-31B-it",
  "messages": [
    {
      "role": "user",
      "content": "A shop sells pens at 17 cents each. I buy 23 pens and pay with a $5 bill. How many cents do I get back? Answer with the number, then explain the arithmetic in one or two sentences."
    }
  ],
  "chat_template_kwargs": {
    "enable_thinking": true
  },
  "temperature": 0,
  "max_tokens": 60
}
```

## What came back

HTTP 200, `finish_reason: "length"` — and this `choices[0]`, verbatim:

```json
{
  "finish_reason": "length",
  "index": 0,
  "logprobs": null,
  "message": {
    "content": "<|channel>thought\n*   Item: Pens\n    *   Price per pen: 17 cents\n    *   Quantity bought: 23 pens\n    *   Payment: $5 bill\n    *   Goal: Calculate change in cents.\n\n    *   Total cost = Price per",
    "role": "assistant"
  }
}
```

`content` opens with the literal template marker `<|channel>thought` followed by the
unfinished chain of thought; there is no reasoning field on the message at all. Three
identical runs returned this byte-identically, 3 of 3.

## The contrast that isolates it

The same request with `max_tokens: 1200` completes and splits cleanly: 641 characters of
reasoning in the reasoning field, and *"109 — 23 pens at 17 cents each cost 391 cents…"* as
the answer. The same request at `max_tokens` 300 and 289 — budgets where the cut lands
mid-*answer*, past the thought — also splits cleanly, 2 of 2: full reasoning in its field, a
truncated answer in `content`, no marker anywhere. Truncation as such is handled correctly.
Only the cut that lands *inside the thought* leaks, and it leaks every time.

## How this breaks expectations

`content` must carry answer text only; template control markers belong to the provider's
parser. SambaNova's own clean handling of mid-answer truncation proves the split is
implemented — it just isn't applied until the closing marker arrives, so a thought cut short
is never recognized as a thought.

## Why it matters

Running against a token budget is routine in production, and reasoning inflates output length
unpredictably — so this fires under ordinary load, deterministically. When it does, the
user-visible answer becomes the model's raw scratchpad: internal deliberation an application
may deliberately hide, prefixed with template markup, with no answer following it. Any caller
that renders or parses `content` gets it wholesale.

## Run it yourself

```
curl "https://api.sambanova.ai/v1/chat/completions" -H "Authorization: Bearer $KEY" \
  -H "Content-Type: application/json" -d @think-truncation.sambanova.json
```

As shipped (`max_tokens: 60`) the response leads with `<|channel>thought` in `content`. Raise
`max_tokens` to 300 — the cut lands mid-answer and the split is clean; raise it to 1200 — the
response completes with reasoning and answer in their proper fields.