Tool results replayed out of order are mis-paired on most providers

medium · inference · probed live 2026-07-26 · affects Lightning, SambaNova, Cerebras, Together AI, DeepInfra, Novita, Parasail, Google (native)

The Chat Completions contract says a role:"tool" message is matched to its call by tool_call_id, in any order. Most OpenAI-dialect providers match by position instead — replay two results in reversed order and the data swaps between the calls.

What was measured — the access-code test

The model fetches unguessable 6-character codes for Paris and Tokyo via two parallel get_code calls, then must report "Paris=…, Tokyo=…". The two results are replayed in reversed order, each carrying its correct tool_call_id. The final answer is a direct readout of which result landed on which call. The replay turn as sent to Together AI — the ids are the provider's own, echoed back (download):

tool-order-turn2.togetherai.json

{
  "model": "google/gemma-4-31B-it",
  "messages": [
    {
      "role": "user",
      "content": "Fetch the access codes for BOTH Paris and Tokyo using `get_code` — you may call it twice in parallel. Then report them exactly as: Paris=<code>, Tokyo=<code>."
    },
    {
      "role": "assistant",
      "content": "",
      "tool_calls": [
        {
          "id": "chatcmpl-tool-973c9f6d7ae2c668",
          "type": "function",
          "function": {
            "name": "get_code",
            "arguments": "{\"city\": \"Paris\"}"
          }
        },
        {
          "id": "chatcmpl-tool-b2f715c8e3ff7fe8",
          "type": "function",
          "function": {
            "name": "get_code",
            "arguments": "{\"city\": \"Tokyo\"}"
          }
        }
      ]
    },
    {
      "role": "tool",
      "tool_call_id": "chatcmpl-tool-b2f715c8e3ff7fe8",
      "content": "710992"
    },
    {
      "role": "tool",
      "tool_call_id": "chatcmpl-tool-973c9f6d7ae2c668",
      "content": "fb6ede"
    }
  ],
  "tools": [
    {
      "type": "function",
      "function": {
        "name": "get_code",
        "description": "Returns the access code for a city.",
        "parameters": {
          "type": "object",
          "properties": {
            "city": {
              "type": "string"
            }
          },
          "required": [
            "city"
          ]
        }
      }
    }
  ],
  "parallel_tool_calls": true,
  "chat_template_kwargs": {
    "enable_thinking": false
  },
  "temperature": 0,
  "max_tokens": 300
}

Paris's true code is fb6ede, Tokyo's is 710992; the ids bind each result correctly — only the array order is reversed. Received, complete:

response · HTTP 200, content

Paris=710992, Tokyo=fb6ede

Swapped. The provider rendered the results positionally and the ids were discarded. The same test, same reversal, on Google's OpenAI-compat shim: Paris=7f1ece, Tokyo=ab4931 — correct, proving id-honoring is implementable on this model.

Provider	Answer	Pairing
Lightning, SambaNova, Cerebras, Together AI, DeepInfra, Novita, Parasail	codes transposed	swapped
Google (native)	codes transposed	positional by design — the protocol has no ids at all (name-keyed `functionResponse`)
Google (OpenAI-compat)	correct	pairs by id
Both Bedrock APIs	—	untestable: never emit two calls (separate finding)

Root cause

The gemma-4 chat template — the text program that turns the conversation into the exact input the model reads — renders calls and results by tool name only; ids don't exist in the model's context. Honoring the id contract therefore requires the provider to re-order results to call order before templating. One provider does; the rest pass the array order straight through.

Why it matters

An async agent harness that appends tool results in completion order — the natural thing — silently swaps payloads between calls. That is data corruption, not degradation: the model states the wrong value with full confidence.

Run it yourself

Two steps. First POST the opening turn (download) and take the two returned calls and their ids verbatim:

tool-order-turn1.togetherai.json

{
  "model": "google/gemma-4-31B-it",
  "messages": [
    {
      "role": "user",
      "content": "Fetch the access codes for BOTH Paris and Tokyo using `get_code` — you may call it twice in parallel. Then report them exactly as: Paris=<code>, Tokyo=<code>."
    }
  ],
  "tools": [
    {
      "type": "function",
      "function": {
        "name": "get_code",
        "description": "Returns the access code for a city.",
        "parameters": {
          "type": "object",
          "properties": {
            "city": {
              "type": "string"
            }
          },
          "required": [
            "city"
          ]
        }
      }
    }
  ],
  "parallel_tool_calls": true,
  "chat_template_kwargs": {
    "enable_thinking": false
  },
  "temperature": 0,
  "max_tokens": 300
}

Then rebuild the replay body above with your provider's ids and two nonce codes of your own, results in reversed array order. Answer matches your codes → the provider pairs by id. Transposed → this finding.

curl

curl "$BASE_URL/chat/completions" -H "Authorization: Bearer $KEY" \
  -H "Content-Type: application/json" -d @tool-order-turn2.togetherai.json