# Lightning leaks template reasoning markers into content

medium · inference · probed live 2026-07-26 · affects Lightning ·
HTML version: [https://inferencecanary.com/lightning-marker-leak](https://inferencecanary.com/lightning-marker-leak)

A chat template's control markers — the delimiters that separate the model's reasoning channel
from its answer — belong to the provider's parser, never to the `content` field an application
renders. On Lightning, on an image-bearing tool turn, the parse intermittently fails in both
directions at once: the reasoning field comes back empty *and* marker text sits at the front
of the answer.

## What was sent

The tool-result-image conversation — a magenta test image returned by `get_image`,
`max_tokens: 300`
([https://inferencecanary.com/assets/repro/tool-image.lightning.json](https://inferencecanary.com/assets/repro/tool-image.lightning.json);
the same conversation as the
[tool-result images finding](https://inferencecanary.com/tool-result-images.md), which Lightning
otherwise passes):

```json
{
  "model": "lightning-ai/gemma-4-31B-it",
  "messages": [
    {
      "role": "user",
      "content": "Call the `get_image` tool. In one short sentence: what is the dominant color of the image?"
    },
    {
      "role": "assistant",
      "content": "",
      "tool_calls": [
        {
          "id": "call_1",
          "type": "function",
          "function": {
            "name": "get_image",
            "arguments": "{}"
          }
        }
      ]
    },
    {
      "role": "tool",
      "tool_call_id": "call_1",
      "content": [
        {
          "type": "image_url",
          "image_url": {
            "url": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAEAAAABACAIAAAAlC+aJAAAATElEQVR42u3PMQkAAAwDsPo33UnoPQjEQNL0tQgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgILAdBp+HSRtACMAAAAABJRU5ErkJggg=="
          }
        }
      ]
    }
  ],
  "tools": [
    {
      "type": "function",
      "function": {
        "name": "get_image",
        "description": "Returns an image.",
        "parameters": {
          "type": "object",
          "properties": {}
        }
      }
    }
  ],
  "temperature": 0,
  "max_tokens": 300,
  "stream": true,
  "stream_options": {
    "include_usage": true
  }
}
```

## What came back

HTTP 200; `content`, exact bytes:

```
thought
<channel|>The dominant color of the image is magenta.
```

The reasoning field is the empty string; the answer itself is correct. The response used 13 of
its 300 tokens — nowhere near truncation. Two of four identical runs leak, byte-identically;
the other two return the clean *"The dominant color of the image is magenta."* Same bytes
sent, same budget.

## The contrast that isolates it

Same provider, same session: a text-returning tool call at a 400-token budget parses cleanly —
reasoning text lands in the reasoning field, content is clean. The user-message-image control
at the same 300-token budget is clean every time. Only the image-bearing *tool* turn leaks,
and only sometimes.

## How this breaks expectations

`content` must carry answer text only. Lightning's own clean responses prove the
marker-stripping normally works; this is a nondeterministic channel-parse failure on one turn
shape.

## Why it matters

Any caller that renders or string-matches `content` intermittently gets a stray `thought` line
and a literal `<channel|>` in its output — the worst kind of bug to chase, because the
identical request usually comes back clean.

## Run it yourself

```
curl "https://lightning.ai/api/v1/chat/completions" -H "Authorization: Bearer $KEY" \
  -H "Content-Type: application/json" -d @tool-image.lightning.json
```

POST the body a few times; some responses will lead with the marker text shown.