# Tool results replayed out of order are mis-paired on most providers

medium · inference · probed live 2026-07-26 · affects Lightning, SambaNova, Cerebras,
Together AI, DeepInfra, Novita, Parasail, Google (native) ·
HTML version: [https://inferencecanary.com/tool-results-out-of-order](https://inferencecanary.com/tool-results-out-of-order)

The Chat Completions contract says a `role:"tool"` message is matched to its call by
`tool_call_id`, in any order. Most OpenAI-dialect providers match by *position* instead —
replay two results in reversed order and the data swaps between the calls.

## What was measured — the access-code test

The model fetches unguessable 6-character codes for Paris and Tokyo via two parallel
`get_code` calls, then must report "Paris=…, Tokyo=…". The two results are replayed in
*reversed* order, each carrying its correct `tool_call_id`. The final answer is a direct
readout of which result landed on which call. The replay turn as sent to Together AI — the ids
are the provider's own, echoed back
([https://inferencecanary.com/assets/repro/tool-order-turn2.togetherai.json](https://inferencecanary.com/assets/repro/tool-order-turn2.togetherai.json)):

```json
{
  "model": "google/gemma-4-31B-it",
  "messages": [
    {
      "role": "user",
      "content": "Fetch the access codes for BOTH Paris and Tokyo using `get_code` — you may call it twice in parallel. Then report them exactly as: Paris=<code>, Tokyo=<code>."
    },
    {
      "role": "assistant",
      "content": "",
      "tool_calls": [
        {
          "id": "chatcmpl-tool-973c9f6d7ae2c668",
          "type": "function",
          "function": {
            "name": "get_code",
            "arguments": "{\"city\": \"Paris\"}"
          }
        },
        {
          "id": "chatcmpl-tool-b2f715c8e3ff7fe8",
          "type": "function",
          "function": {
            "name": "get_code",
            "arguments": "{\"city\": \"Tokyo\"}"
          }
        }
      ]
    },
    {
      "role": "tool",
      "tool_call_id": "chatcmpl-tool-b2f715c8e3ff7fe8",
      "content": "710992"
    },
    {
      "role": "tool",
      "tool_call_id": "chatcmpl-tool-973c9f6d7ae2c668",
      "content": "fb6ede"
    }
  ],
  "tools": [
    {
      "type": "function",
      "function": {
        "name": "get_code",
        "description": "Returns the access code for a city.",
        "parameters": {
          "type": "object",
          "properties": {
            "city": {
              "type": "string"
            }
          },
          "required": [
            "city"
          ]
        }
      }
    }
  ],
  "parallel_tool_calls": true,
  "chat_template_kwargs": {
    "enable_thinking": false
  },
  "temperature": 0,
  "max_tokens": 300
}
```

Paris's true code is `fb6ede`, Tokyo's is `710992`; the ids bind each result correctly — only
the array order is reversed. Received, complete:

```
Paris=710992, Tokyo=fb6ede
```

Swapped. The provider rendered the results positionally and the ids were discarded. The same
test, same reversal, on Google's OpenAI-compat shim: `Paris=7f1ece, Tokyo=ab4931` —
**correct**, proving id-honoring is implementable on this model.

| Provider | Answer | Pairing |
|---|---|---|
| Lightning, SambaNova, Cerebras, Together AI, DeepInfra, Novita, Parasail | codes transposed | swapped |
| Google (native) | codes transposed | positional by design — the protocol has no ids at all (name-keyed `functionResponse`) |
| Google (OpenAI-compat) | correct | pairs by id |
| Both Bedrock APIs | — | untestable: never emit two calls ([separate finding](https://inferencecanary.com/bedrock-parallel-tool-call.md)) |

## Root cause

The gemma-4 chat template — the text program that turns the conversation into the exact input
the model reads — renders calls and results by tool *name* only; ids don't exist in the
model's context. Honoring the id contract therefore requires the provider to re-order results
to call order *before* templating. One provider does; the rest pass the array order straight
through.

## Why it matters

An async agent harness that appends tool results in completion order — the natural thing —
silently swaps payloads between calls. That is data corruption, not degradation: the model
states the wrong value with full confidence.

## Run it yourself

Two steps. First POST the opening turn
([https://inferencecanary.com/assets/repro/tool-order-turn1.togetherai.json](https://inferencecanary.com/assets/repro/tool-order-turn1.togetherai.json))
and take the two returned calls and their ids verbatim:

```json
{
  "model": "google/gemma-4-31B-it",
  "messages": [
    {
      "role": "user",
      "content": "Fetch the access codes for BOTH Paris and Tokyo using `get_code` — you may call it twice in parallel. Then report them exactly as: Paris=<code>, Tokyo=<code>."
    }
  ],
  "tools": [
    {
      "type": "function",
      "function": {
        "name": "get_code",
        "description": "Returns the access code for a city.",
        "parameters": {
          "type": "object",
          "properties": {
            "city": {
              "type": "string"
            }
          },
          "required": [
            "city"
          ]
        }
      }
    }
  ],
  "parallel_tool_calls": true,
  "chat_template_kwargs": {
    "enable_thinking": false
  },
  "temperature": 0,
  "max_tokens": 300
}
```

Then rebuild the replay body above with your provider's ids and two nonce codes of your own,
results in reversed array order. Answer matches your codes → the provider pairs by id.
Transposed → this finding.

```
curl "$BASE_URL/chat/completions" -H "Authorization: Bearer $KEY" \
  -H "Content-Type: application/json" -d @tool-order-turn2.togetherai.json
```