# Gemma 4 31BHigh > Source: Benchize (https://benchize.com) — https://benchize.com/models/gemma-4-31b-it-high. When citing or reusing this content, credit "Benchize" and link to https://benchize.com/models/gemma-4-31b-it-high. Google Canonical URL: https://benchize.com/models/gemma-4-31b-it-high Last updated: Wed, 21 Oct 2026 18:52:20 GMT Reproduction and quotation must name Benchize, link to the canonical HTML URL, and retain individual source citations. Google ## Benchmarks | Benchmark | Score | ± | Rank | Cost | Latency | Source | | | --- | --- | --- | --- | --- | --- | --- | --- | | [MortgageTax / overall](https://benchize.com/benchmarks/vals-ai%3Amortgage_tax%3Aoverall.md) | 61.37% | SE ±0.97 pp | 24 / 31 | $0 | 41.93source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_mortgage_tax.DCd-oO5Y.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 61.368 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0 \| USD \| per_test \| Lower is better \| \| `latency` \| 41.928 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 32768 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.97 pp sourcePopulation 98 stderr 0.967 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_mortgage_tax.DCd-oO5Y.json) Source Vals AI Original model label google/gemma-4-31b-it Source version 1 Observed 2026-10-21T16:45:58.500Z Evidence SHA-256 `e5c844561f97d119d66d70269e726ecfb68be79eae39acf14ead3e95422e648c` Source locator `/tasks/overall/google~1gemma-4-31b-it` | | | | | | | | | [SAGE / PDEs](https://benchize.com/benchmarks/vals-ai%3Asage%3APDEs.md) | 50.79% | SE ±6.27 pp | 1 / 41 | $0 | 106.92source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_sage.RRwIIzO5.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 50.794 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0 \| USD \| per_test \| Lower is better \| \| `latency` \| 106.921 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 30000 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±6.27 pp sourcePopulation 91 stderr 6.273 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_sage.RRwIIzO5.json) Source Vals AI Original model label google/gemma-4-31b-it Source version 1 Observed 2026-10-21T16:46:07.515Z Evidence SHA-256 `cc5fb3fecdb0382d492ff167a42ad84b9c5c3883373c78ed87baaa972b8b6f7c` Source locator `/tasks/PDEs/google~1gemma-4-31b-it` | | | | | | | | | [BullshitBench v2](https://benchize.com/benchmarks/bullshitbench%3Av2.md) | 25% | — | 123 / 191 | — | — | [bullshitbench · v2](https://api.github.com/repos/petergpt/bullshit-benchmark/contents/data/v2/latest/leaderboard.csv?ref=3c655234d15967738f95da696e3f938d4862dcf7) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `green_rate` \| 0.25 \| score_fraction \| all_attempts \| Higher is better \| \| `red_rate` \| 0.51 \| score_fraction \| all_attempts \| Lower is better \| \| `refusal_rate` \| 0.0 \| score_fraction \| all_attempts \| Lower is better \| \| `avg_score` \| 0.74 \| score \| judge_grade_0_to_2 \| Higher is better \| ### Configuration Reasoning effort high suite v2 thinkingEnabled true ### Reported statistics acceptedCount 51 answeredCount 100 clearCount 25 errorCount 0 partialCount 24 rank 152 refusalCount 0 sampleSize 100 ### Source [Original source](https://api.github.com/repos/petergpt/bullshit-benchmark/contents/data/v2/latest/leaderboard.csv?ref=3c655234d15967738f95da696e3f938d4862dcf7) Source bullshitbench Original model label google/gemma-4-31b-it Source version v2 Observed 2026-10-21T16:33:56.675Z Evidence SHA-256 `98b485e08764d1c97144bc4f7b28ced1a96d50850947c775870dfda0e5d749b5` Source locator `/rows/151` | | | | | | | | | [MortgageTax / numerical extraction](https://benchize.com/benchmarks/vals-ai%3Amortgage_tax%3Anumerical_extraction.md) | 65.66% | SE ±1.34 pp | 25 / 31 | $0 | 55.5source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_mortgage_tax.DCd-oO5Y.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 65.66 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0 \| USD \| per_test \| Lower is better \| \| `latency` \| 55.497 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 32768 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.34 pp sourcePopulation 98 stderr 1.339 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_mortgage_tax.DCd-oO5Y.json) Source Vals AI Original model label google/gemma-4-31b-it Source version 1 Observed 2026-10-21T16:45:58.500Z Evidence SHA-256 `e5c844561f97d119d66d70269e726ecfb68be79eae39acf14ead3e95422e648c` Source locator `/tasks/numerical_extraction/google~1gemma-4-31b-it` | | | | | | | | | [Terminal-Bench 2.0 / medium](https://benchize.com/benchmarks/vals-ai%3Aterminal-bench-2%3Amedium.md) | 45.46% | SE ±6.78 pp | 14 / 23 | — | 1,384.03source duration | [Vals AI · 2.0](https://www.vals.ai/_astro/benchmark_view_terminal_bench_2.T48tPR4O.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 45.455 \| percent \| reported \| Higher is better \| \| `latency` \| 1384.027 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 32768 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±6.78 pp sourcePopulation 67 stderr 6.776 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_terminal_bench_2.T48tPR4O.json) Source Vals AI Original model label google/gemma-4-31b-it Source version 2.0 Observed 2026-10-21T16:46:14.732Z Evidence SHA-256 `38e640d995259778e531f048b2f2c7b3b281cb05b56be643ddc9dc53eb85adf7` Source locator `/tasks/medium/google~1gemma-4-31b-it` | | | | | | | | | [Terminal-Bench 2.0 / overall](https://benchize.com/benchmarks/vals-ai%3Aterminal-bench-2%3Aoverall.md) | 39.33% | SE ±5.21 pp | 13 / 23 | — | 855.3source duration | [Vals AI · 2.0](https://www.vals.ai/_astro/benchmark_view_terminal_bench_2.T48tPR4O.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 39.326 \| percent \| reported \| Higher is better \| \| `latency` \| 855.298 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 32768 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±5.21 pp sourcePopulation 67 stderr 5.207 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_terminal_bench_2.T48tPR4O.json) Source Vals AI Original model label google/gemma-4-31b-it Source version 2.0 Observed 2026-10-21T16:46:14.732Z Evidence SHA-256 `38e640d995259778e531f048b2f2c7b3b281cb05b56be643ddc9dc53eb85adf7` Source locator `/tasks/overall/google~1gemma-4-31b-it` | | | | | | | | | [Terminal-Bench 2.0 / easy](https://benchize.com/benchmarks/vals-ai%3Aterminal-bench-2%3Aeasy.md) | 75% | SE ±25 pp | 9 / 23 | — | 19,030.38source duration | [Vals AI · 2.0](https://www.vals.ai/_astro/benchmark_view_terminal_bench_2.T48tPR4O.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 75 \| percent \| reported \| Higher is better \| \| `latency` \| 19030.377 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 32768 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±25 pp sourcePopulation 67 stderr 25 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_terminal_bench_2.T48tPR4O.json) Source Vals AI Original model label google/gemma-4-31b-it Source version 2.0 Observed 2026-10-21T16:46:14.732Z Evidence SHA-256 `38e640d995259778e531f048b2f2c7b3b281cb05b56be643ddc9dc53eb85adf7` Source locator `/tasks/easy/google~1gemma-4-31b-it` | | | | | | | | | [SAGE / overall](https://benchize.com/benchmarks/vals-ai%3Asage%3Aoverall.md) | 55.03% | SE ±3.41 pp | 2 / 41 | $0 | 93.06source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_sage.RRwIIzO5.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 55.034 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0 \| USD \| per_test \| Lower is better \| \| `latency` \| 93.058 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 30000 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±3.41 pp sourcePopulation 91 stderr 3.411 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_sage.RRwIIzO5.json) Source Vals AI Original model label google/gemma-4-31b-it Source version 1 Observed 2026-10-21T16:46:07.515Z Evidence SHA-256 `cc5fb3fecdb0382d492ff167a42ad84b9c5c3883373c78ed87baaa972b8b6f7c` Source locator `/tasks/overall/google~1gemma-4-31b-it` | | | | | | | | | [MortgageTax / semantic extraction](https://benchize.com/benchmarks/vals-ai%3Amortgage_tax%3Asemantic_extraction.md) | 57.08% | SE ±1.4 pp | 25 / 31 | $0 | 28.36source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_mortgage_tax.DCd-oO5Y.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 57.075 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0 \| USD \| per_test \| Lower is better \| \| `latency` \| 28.359 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 32768 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.4 pp sourcePopulation 98 stderr 1.396 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_mortgage_tax.DCd-oO5Y.json) Source Vals AI Original model label google/gemma-4-31b-it Source version 1 Observed 2026-10-21T16:45:58.500Z Evidence SHA-256 `e5c844561f97d119d66d70269e726ecfb68be79eae39acf14ead3e95422e648c` Source locator `/tasks/semantic_extraction/google~1gemma-4-31b-it` | | | | | | | | | [CaseLaw v2 / overall](https://benchize.com/benchmarks/vals-ai%3Acase_law_v2%3Aoverall.md) | 52.63% | SE ±1.34 pp | 16 / 17 | $0 | 28.72source duration | [Vals AI · 2](https://www.vals.ai/_astro/benchmark_view_case_law_v2.DLKA06_7.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 52.626 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0 \| USD \| per_test \| Lower is better \| \| `latency` \| 28.716 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 32768 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.34 pp sourcePopulation 54 stderr 1.343 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_case_law_v2.DLKA06_7.json) Source Vals AI Original model label google/gemma-4-31b-it Source version 2 Observed 2026-10-21T16:45:54.497Z Evidence SHA-256 `293f388f7e0fe563287d4cd19beaa7ef9821b32795dace34bd0197ed47a592d4` Source locator `/tasks/overall/google~1gemma-4-31b-it` | | | | | | | | | [SAGE / calculus](https://benchize.com/benchmarks/vals-ai%3Asage%3Acalculus.md) | 59.26% | SE ±6.32 pp | 25 / 36 | $0 | 88.77source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_sage.RRwIIzO5.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 59.259 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0 \| USD \| per_test \| Lower is better \| \| `latency` \| 88.767 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 30000 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±6.32 pp sourcePopulation 91 stderr 6.315 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_sage.RRwIIzO5.json) Source Vals AI Original model label google/gemma-4-31b-it Source version 1 Observed 2026-10-21T16:46:07.515Z Evidence SHA-256 `cc5fb3fecdb0382d492ff167a42ad84b9c5c3883373c78ed87baaa972b8b6f7c` Source locator `/tasks/calculus/google~1gemma-4-31b-it` | | | | | | | | | [SAGE / linear algebra](https://benchize.com/benchmarks/vals-ai%3Asage%3Alinear_algebra.md) | 72.64% | SE ±5.05 pp | 7 / 41 | $0 | 94.49source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_sage.RRwIIzO5.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 72.643 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0 \| USD \| per_test \| Lower is better \| \| `latency` \| 94.494 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 30000 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±5.05 pp sourcePopulation 91 stderr 5.046 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_sage.RRwIIzO5.json) Source Vals AI Original model label google/gemma-4-31b-it Source version 1 Observed 2026-10-21T16:46:07.515Z Evidence SHA-256 `cc5fb3fecdb0382d492ff167a42ad84b9c5c3883373c78ed87baaa972b8b6f7c` Source locator `/tasks/linear_algebra/google~1gemma-4-31b-it` | | | | | | | | | [Terminal-Bench 2.0 / hard](https://benchize.com/benchmarks/vals-ai%3Aterminal-bench-2%3Ahard.md) | 23.33% | SE ±7.85 pp | 11 / 23 | — | 2,537.38source duration | [Vals AI · 2.0](https://www.vals.ai/_astro/benchmark_view_terminal_bench_2.T48tPR4O.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 23.333 \| percent \| reported \| Higher is better \| \| `latency` \| 2537.384 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 32768 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±7.85 pp sourcePopulation 67 stderr 7.854 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_terminal_bench_2.T48tPR4O.json) Source Vals AI Original model label google/gemma-4-31b-it Source version 2.0 Observed 2026-10-21T16:46:14.732Z Evidence SHA-256 `38e640d995259778e531f048b2f2c7b3b281cb05b56be643ddc9dc53eb85adf7` Source locator `/tasks/hard/google~1gemma-4-31b-it` | | | | | | | | | [SAGE / automata](https://benchize.com/benchmarks/vals-ai%3Asage%3Aautomata.md) | 41.67% | SE ±6.32 pp | 8 / 41 | $0 | 77.76source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_sage.RRwIIzO5.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 41.666 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0 \| USD \| per_test \| Lower is better \| \| `latency` \| 77.76 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 30000 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±6.32 pp sourcePopulation 91 stderr 6.317 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_sage.RRwIIzO5.json) Source Vals AI Original model label google/gemma-4-31b-it Source version 1 Observed 2026-10-21T16:46:07.515Z Evidence SHA-256 `cc5fb3fecdb0382d492ff167a42ad84b9c5c3883373c78ed87baaa972b8b6f7c` Source locator `/tasks/automata/google~1gemma-4-31b-it` | | | | | | | | --- Cite as: Gemma 4 31BHigh — Benchize, https://benchize.com/models/gemma-4-31b-it-high (retrieved 2026-10-21)