gemma-4-31b, provider by provider
All cells probed live · latest probe 2026-08-29 · how grading works
One model, many hosted endpoints. The capabilities below are defined by the model's own chat template - the text program, shipped with the model, that turns a conversation into the exact input the model reads. Every provider is graded on whether it delivers them.
inferencecanary · gemma-4-31b · probed 2026-08-29
inferencecanary.com
| Provider | Inference | Billing | Caching | 20K-token agentic turn | |
|---|---|---|---|---|---|
| Speed | Cost | ||||
| Lightning OpenAI chat/completions | A− 1 medium | ✓ | Yes | 1.2s | 0.29¢ |
| SambaNova OpenAI chat/completions | A− 1 medium | ✓ | No | 4.9s | 0.79¢ |
| Google (native)reference generateContent - Google's own protocol | B 1 high | ✓ | Fail | 47s | Fail |
| Parasail OpenAI chat/completions | B 1 high | ✓ | Sometimes | 17.4-25.1s | 0.13-0.31¢ |
| Cerebras OpenAI chat/completions | B 1 high | ✓ | Yes | 0.7s | 2.0¢ |
| Together AI OpenAI chat/completions | B 1 high | ✓ | Sometimes | 3.4-6.6s | 0.80¢ |
| Novita OpenAI chat/completions | B 1 high | ✓ | No | ~30s | 0.29¢ |
| DeepInfra OpenAI chat/completions | C 2 high | ✓ | Sometimes | 24.8-29.7s | 0.27¢ |
| AWS Bedrock (Chat Completions) OpenAI chat/completions on Bedrock Mantle | D 2 high · 2 medium | ✓footnote on card | No | 3.3s | 0.29¢ |
| AWS Bedrock (Responses) OpenAI Responses on Bedrock Mantle | D− 2 high · 3 medium | ✓footnote on card | No | - | 0.29¢ |
| Google (OpenAI-compat) OpenAI chat/completions shim over generateContent | F 4 high · 1 medium | ✓footnote on card | Fail | 48s | Fail |
What the model can do - and who delivers it
- Think in a separate channel. The template defines a reasoning phase ahead of the answer, surfaced by every dialect as a structured reasoning field - except one: Google's own OpenAI-compat shim has no reasoning channel at all, leaking thought text into the answer field.
- Keep its reasoning across tool calls. The template renders the model's prior thinking back to it on later turns. Several providers delete it before the model can read it.
- Read images returned by tools. The reference protocol delivers them and several OpenAI-compat providers prove the standard shape works - yet several providers reject or silently blind them, closing off agentic multimodal.
- Call tools - several at once. Every provider but the Bedrock APIs
emits parallel calls; Bedrock never does.
Constraining the choice with
tool_choiceis honored on most providers and validated-but-unenforced on the rest. - Return byte-exact tool arguments. Every other provider agrees with the reference; both Bedrock APIs apply an extra unescape pass, corrupting any argument whose bytes matter.
Every graded endpoint
- A− Lightning - OpenAI chat/completions
- A− SambaNova - OpenAI chat/completions
- B Google (native) reference - generateContent - Google's own protocol
- B Parasail - OpenAI chat/completions
- B Cerebras - OpenAI chat/completions
- B Together AI - OpenAI chat/completions
- B Novita - OpenAI chat/completions
- C DeepInfra - OpenAI chat/completions
- D AWS Bedrock (Chat Completions) - OpenAI chat/completions on Bedrock Mantle
- D− AWS Bedrock (Responses) - OpenAI Responses on Bedrock Mantle
- F Google (OpenAI-compat) - OpenAI chat/completions shim over generateContent
Vendor roll-ups: Lightning AI · SambaNova · Google · Parasail · Cerebras · Together AI · Novita AI · DeepInfra · AWS.