AWS

2 graded endpoints serving gemma-4-31b · back to the matrix

A vendor is graded once per API it speaks, because a shim and a native protocol routinely behave differently - AWS speaks 2.

Billing

Footnote (customer-favoring, not flagged): Reports reasoning_tokens: 0 beside delivered reasoning - under-counts in the customer's favor.

Findings affecting AWS

• high inference Several providers drop replayed reasoning: the model never sees its own prior thinking
  The gemma-4 chat template renders reasoning replayed on model turns, and the model card mandates preserving thinking across tool-call turns - the reference provider reproduces this live. These providers delete it before...
• high inference Images in tool results: rejected or silently blinded on several providers
  The model reads images returned by tools - the reference provider delivers them, perception-judged, and several OpenAI-compat providers prove the standard shape works. The failing providers fail on schema choice, not...
• medium inference AWS Bedrock never emits parallel tool calls
  parallel_tool_calls: true is accepted on both Bedrock APIs, but only a single call ever comes back; every other provider emits two for the same two-lookup prompt. A model capability suppressed at the serving layer; the...
• medium inference AWS Bedrock un-escapes tool-call arguments in transit
  At temperature 0, every other provider agrees the model returns a tricky string argument with literal escape sequences. Both Bedrock APIs instead return real control characters - one extra unescape pass applied to the...
• medium inference AWS Bedrock (Responses) rejects temperature
  temperature returns 400 unsupported_parameter, so deterministic sampling is unreachable on this provider: every run is stochastic, degrading reproducibility for applications and evals alike.