# Gemini 3 Flash PreviewHigh > Source: Benchize (https://benchize.com) — https://benchize.com/models/gemini-3-flash-preview-high. When citing or reusing this content, credit "Benchize" and link to https://benchize.com/models/gemini-3-flash-preview-high. Google Canonical URL: https://benchize.com/models/gemini-3-flash-preview-high Last updated: Wed, 21 Oct 2026 18:52:20 GMT Reproduction and quotation must name Benchize, link to the canonical HTML URL, and retain individual source citations. Google ## Benchmarks | Benchmark | Score | ± | Rank | Cost | Latency | Source | | | --- | --- | --- | --- | --- | --- | --- | --- | | [Harvey's Legal Agent Benchmark v1 / arbitration international dispute resolution](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Aarbitration_international_dispute_resolution.md) | 0% | SE ±0 pp | 4 / 49 | $0.26 | 240.99source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.255694 \| USD \| per_test \| Lower is better \| \| `latency` \| 240.988 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 81 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label google/gemini-3-flash-preview Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/arbitration_international_dispute_resolution/google~1gemini-3-flash-preview` | | | | | | | | | [SWE-bench / overall](https://benchize.com/benchmarks/vals-ai%3Aswebench%3Aoverall.md) | 75% | SE ±1.94 pp | 22 / 39 | $0.28 | 285.22source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_swebench.Ae8ni7h0.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 75 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.280231 \| USD \| per_test \| Lower is better \| \| `latency` \| 285.223 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.94 pp sourcePopulation 88 stderr 1.938 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_swebench.Ae8ni7h0.json) Source Vals AI Original model label google/gemini-3-flash-preview Source version 1 Observed 2026-10-21T16:46:13.711Z Evidence SHA-256 `7a4e2e8485283caac500127419f68ec4bd5414c837ade114fc6afa5b2d34ccaa` Source locator `/tasks/overall/google~1gemini-3-flash-preview` | | | | | | | | | [TaxEval v2 / correctness](https://benchize.com/benchmarks/vals-ai%3Atax_eval_v2%3Acorrectness.md) | 62.31% | SE ±1.39 pp | 19 / 40 | $0.00576 | 12source duration | [Vals AI · 2](https://www.vals.ai/_astro/benchmark_view_tax_eval_v2.B3yoffr8.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 62.306 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.005764 \| USD \| per_test \| Lower is better \| \| `latency` \| 12.001 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.39 pp sourcePopulation 145 stderr 1.386 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_tax_eval_v2.B3yoffr8.json) Source Vals AI Original model label google/gemini-3-flash-preview Source version 2 Observed 2026-10-21T16:45:59.000Z Evidence SHA-256 `8b354a6ae848f5bb25a2e8b061b5f576d5b9e0c11b941dc1e6bed078e6529cd6` Source locator `/tasks/correctness/google~1gemini-3-flash-preview` | | | | | | | | | [Legal Research Bench / family](https://benchize.com/benchmarks/vals-ai%3Alegal_research%3Afamily.md) | 4.55% | SE ±4.55 pp | 39 / 50 | $0.17 | 167.28source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 4.545 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.168183 \| USD \| per_test \| Lower is better \| \| `latency` \| 167.284 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±4.55 pp sourcePopulation 82 stderr 4.545 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) Source Vals AI Original model label google/gemini-3-flash-preview Source version 1 Observed 2026-10-21T16:45:52.389Z Evidence SHA-256 `26c58dfd47d0d2dc86f33c6d056f30309e0ee63570e7cd752c593b385f9d862e` Source locator `/tasks/family/google~1gemini-3-flash-preview` | | | | | | | | | [Terminal-Bench 2.1 / easy](https://benchize.com/benchmarks/vals-ai%3Aterminal-bench-2-1%3Aeasy.md) | 75% | SE ±14.43 pp | 29 / 43 | $3.43 | 8,369.51source duration | [Vals AI · 2.1](https://www.vals.ai/_astro/benchmark_view_terminal_bench_2_1.CdyoObsA.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 75 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 3.433448 \| USD \| per_test \| Lower is better \| \| `latency` \| 8369.51 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±14.43 pp sourcePopulation 76 stderr 14.434 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_terminal_bench_2_1.CdyoObsA.json) Source Vals AI Original model label google/gemini-3-flash-preview Source version 2.1 Observed 2026-10-21T16:46:13.216Z Evidence SHA-256 `a5306805cdf86495bdcceff6b2c8534a3768b9853f5077895dab975b809b054f` Source locator `/tasks/easy/google~1gemini-3-flash-preview` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / energy natural resources](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Aenergy_natural_resources.md) | 0% | SE ±0 pp | 28 / 49 | $0.37 | 327.08source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.37291 \| USD \| per_test \| Lower is better \| \| `latency` \| 327.076 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 81 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label google/gemini-3-flash-preview Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/energy_natural_resources/google~1gemini-3-flash-preview` | | | | | | | | | [Chess Puzzles](https://benchize.com/benchmarks/epoch-ai%3Achess_puzzles.md) | 39% | SE ±4.9 pp | 30 / 63 | — | — | [epoch ai · Chess Puzzles](https://epoch.ai/data/benchmark_data.zip#file=chess_puzzles.csv) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 0.39 \| score_fraction \| Best score (across scorers) \| Higher is better \| \| `mean_score` \| 0.39 \| score_fraction \| mean_score \| Higher is better \| ### Configuration benchmarkVersion Chess Puzzles reasoningBudget Not reported Reasoning effort high sourceModelVersion gemini-3-flash-preview_high ### Reported statistics Uncertainty Standard error: ±4.9 pp standardError 0.04902071300001973 ### Source [Original source](https://epoch.ai/data/benchmark_data.zip#file=chess_puzzles.csv) Source epoch ai Original model label gemini-3-flash-preview Source version Chess Puzzles Evaluated 2026-10-13T13:42:10.000Z Observed 2026-10-21T16:30:24.199Z Evidence SHA-256 `fa0a449c370b0cceaf273d9472078cce3e5aef6be7009bddf115e6df7f930c54` Source locator `/rows/92` | | | | | | | | | [Vals Multimodal Index / corp fin v2](https://benchize.com/benchmarks/vals-ai%3Avals_multimodal_index%3Acorp_fin_v2.md) | 67.72% | SE ±1.6 pp | 8 / 20 | $0.04 | 14.83source duration | [Vals AI · 1.2](https://www.vals.ai/_astro/benchmark_view_vals_multimodal_index.virVpWMs.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 67.716 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.039017 \| USD \| per_test \| Lower is better \| \| `latency` \| 14.833 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.6 pp sourcePopulation 33 stderr 1.596 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_vals_multimodal_index.virVpWMs.json) Source Vals AI Original model label google/gemini-3-flash-preview Source version 1.2 Observed 2026-10-21T16:45:51.881Z Evidence SHA-256 `077b22575f6e075b7b78157d000ec8f94d7ead409c7dcbcf04887630ed572a2d` Source locator `/tasks/corp_fin_v2/google~1gemini-3-flash-preview` | | | | | | | | | [GPQA Diamond](https://benchize.com/benchmarks/gpqa-diamond.md) | 89.39% | SE ±2.19 pp | 32 / 125 | — | — | [epoch ai · GPQA diamond](https://epoch.ai/data/benchmark_data.zip#file=gpqa_diamond.csv) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 0.8939393939393939 \| score_fraction \| Best score (across scorers) \| Higher is better \| \| `mean_score` \| 0.8939393939393939 \| score_fraction \| mean_score \| Higher is better \| ### Configuration benchmarkVersion GPQA diamond reasoningBudget Not reported Reasoning effort high sourceModelVersion gemini-3-flash-preview_high ### Reported statistics Uncertainty Standard error: ±2.19 pp standardError 0.021938047738853106 ### Source [Original source](https://epoch.ai/data/benchmark_data.zip#file=gpqa_diamond.csv) Source epoch ai Original model label gemini-3-flash-preview Source version GPQA diamond Evaluated 2026-08-06T22:22:58.000Z Observed 2026-10-21T16:30:24.199Z Evidence SHA-256 `4b2dd8376cdd249a2c3065603575208fa726c15eabbb1622689a9abc146ba29a` Source locator `/rows/119` | | | | | | | | | [Legal Research Bench / constitutional civil rights](https://benchize.com/benchmarks/vals-ai%3Alegal_research%3Aconstitutional_civil_rights.md) | 7.9% | SE ±4.43 pp | 45 / 50 | $0.12 | 214.46source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 7.895 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.119798 \| USD \| per_test \| Lower is better \| \| `latency` \| 214.461 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±4.43 pp sourcePopulation 82 stderr 4.433 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) Source Vals AI Original model label google/gemini-3-flash-preview Source version 1 Observed 2026-10-21T16:45:52.389Z Evidence SHA-256 `26c58dfd47d0d2dc86f33c6d056f30309e0ee63570e7cd752c593b385f9d862e` Source locator `/tasks/constitutional_civil_rights/google~1gemini-3-flash-preview` | | | | | | | | | [MGSM / mgsm sw](https://benchize.com/benchmarks/vals-ai%3Amgsm%3Amgsm_sw.md) | 97.2% | SE ±1.04 pp | 1 / 8 | $0.00773 | 15.13source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_mgsm.CneCP-rm.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 97.2 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.007734 \| USD \| per_test \| Lower is better \| \| `latency` \| 15.125 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.04 pp sourcePopulation 75 stderr 1.043 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_mgsm.CneCP-rm.json) Source Vals AI Original model label google/gemini-3-flash-preview Source version 1 Observed 2026-10-21T16:46:02.976Z Evidence SHA-256 `1b87f953e52c00bf4bdd90c3add26165f644eeb7a74a8382d2a0324ff0cc1040` Source locator `/tasks/mgsm_sw/google~1gemini-3-flash-preview` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / insurance](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Ainsurance.md) | 0% | SE ±0 pp | 2 / 49 | $0.21 | 137.04source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.213394 \| USD \| per_test \| Lower is better \| \| `latency` \| 137.038 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 81 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label google/gemini-3-flash-preview Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/insurance/google~1gemini-3-flash-preview` | | | | | | | | | [Legal Research Bench / weighted pass immigration](https://benchize.com/benchmarks/vals-ai%3Alegal_research%3Aweighted_pass__immigration.md) | 71.53% | SE ±8.03 pp | 43 / 50 | $0.09 | 128.85source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 71.531 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.088481 \| USD \| per_test \| Lower is better \| \| `latency` \| 128.854 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±8.03 pp sourcePopulation 82 stderr 8.032 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) Source Vals AI Original model label google/gemini-3-flash-preview Source version 1 Observed 2026-10-21T16:45:52.389Z Evidence SHA-256 `26c58dfd47d0d2dc86f33c6d056f30309e0ee63570e7cd752c593b385f9d862e` Source locator `/tasks/weighted_pass__immigration/google~1gemini-3-flash-preview` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / tax](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Atax.md) | 0% | SE ±0 pp | 3 / 49 | $0.31 | 300.57source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.312585 \| USD \| per_test \| Lower is better \| \| `latency` \| 300.574 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 81 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label google/gemini-3-flash-preview Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/tax/google~1gemini-3-flash-preview` | | | | | | | | | [Vibe Code Bench v1.1 / overall](https://benchize.com/benchmarks/vals-ai%3Avibe-code%3Aoverall.md) | 20.2% | SE ±3.95 pp | 48 / 59 | $0.94 | 806.66source duration | [Vals AI · 1.1](https://www.vals.ai/_astro/benchmark_view_vibe_code_bench.DcEBMbCA.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 20.204 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.942067 \| USD \| per_test \| Lower is better \| \| `latency` \| 806.66 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness OpenHands maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±3.95 pp sourcePopulation 116 stderr 3.946 tokenTotals {"cache_read_tokens":279403697,"cache_write_tokens":0,"input_tokens":317204132,"output_tokens":2183311,"reasoning_tokens":357653} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_vibe_code_bench.DcEBMbCA.json) Source Vals AI Original model label google/gemini-3-flash-preview Source version 1.1 Observed 2026-10-21T16:46:08.625Z Evidence SHA-256 `0e8c23f521a0f751c5592952620e197b376cade41c78578461b7aa5740a173e7` Source locator `/tasks/overall/google~1gemini-3-flash-preview` | | | | | | | | | [MGSM / mgsm zh](https://benchize.com/benchmarks/vals-ai%3Amgsm%3Amgsm_zh.md) | 94.8% | SE ±1.4 pp | 1 / 8 | $0.00721 | 14.34source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_mgsm.CneCP-rm.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 94.8 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.007205 \| USD \| per_test \| Lower is better \| \| `latency` \| 14.336 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.4 pp sourcePopulation 75 stderr 1.404 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_mgsm.CneCP-rm.json) Source Vals AI Original model label google/gemini-3-flash-preview Source version 1 Observed 2026-10-21T16:46:02.976Z Evidence SHA-256 `1b87f953e52c00bf4bdd90c3add26165f644eeb7a74a8382d2a0324ff0cc1040` Source locator `/tasks/mgsm_zh/google~1gemini-3-flash-preview` | | | | | | | | | [Terminal-Bench 2.0 / hard](https://benchize.com/benchmarks/vals-ai%3Aterminal-bench-2%3Ahard.md) | 43.33% | SE ±9.2 pp | 7 / 23 | $0.5 | 1,353.01source duration | [Vals AI · 2.0](https://www.vals.ai/_astro/benchmark_view_terminal_bench_2.T48tPR4O.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 43.333 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.500482 \| USD \| per_test \| Lower is better \| \| `latency` \| 1353.011 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±9.2 pp sourcePopulation 67 stderr 9.202 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_terminal_bench_2.T48tPR4O.json) Source Vals AI Original model label google/gemini-3-flash-preview Source version 2.0 Observed 2026-10-21T16:46:14.732Z Evidence SHA-256 `38e640d995259778e531f048b2f2c7b3b281cb05b56be643ddc9dc53eb85adf7` Source locator `/tasks/hard/google~1gemini-3-flash-preview` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / criteria pass rate international trade sanctions](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__international_trade_sanctions.md) | 40.87% | SE ±3.1 pp | 47 / 49 | $0.23 | 292.1source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 40.873 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.228541 \| USD \| per_test \| Lower is better \| \| `latency` \| 292.1 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±3.1 pp sourcePopulation 81 stderr 3.103 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label google/gemini-3-flash-preview Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__international_trade_sanctions/google~1gemini-3-flash-preview` | | | | | | | | | [MGSM / mgsm es](https://benchize.com/benchmarks/vals-ai%3Amgsm%3Amgsm_es.md) | 96.4% | SE ±1.18 pp | 1 / 8 | $0.00627 | 12.37source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_mgsm.CneCP-rm.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 96.4 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.006273 \| USD \| per_test \| Lower is better \| \| `latency` \| 12.371 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.18 pp sourcePopulation 75 stderr 1.178 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_mgsm.CneCP-rm.json) Source Vals AI Original model label google/gemini-3-flash-preview Source version 1 Observed 2026-10-21T16:46:02.976Z Evidence SHA-256 `1b87f953e52c00bf4bdd90c3add26165f644eeb7a74a8382d2a0324ff0cc1040` Source locator `/tasks/mgsm_es/google~1gemini-3-flash-preview` | | | | | | | | | [TaxEval v2 / overall](https://benchize.com/benchmarks/vals-ai%3Atax_eval_v2%3Aoverall.md) | 73.88% | SE ±0.86 pp | 19 / 40 | $0.00576 | 12source duration | [Vals AI · 2](https://www.vals.ai/_astro/benchmark_view_tax_eval_v2.B3yoffr8.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 73.876 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.005764 \| USD \| per_test \| Lower is better \| \| `latency` \| 12.001 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.86 pp sourcePopulation 145 stderr 0.857 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_tax_eval_v2.B3yoffr8.json) Source Vals AI Original model label google/gemini-3-flash-preview Source version 2 Observed 2026-10-21T16:45:59.000Z Evidence SHA-256 `8b354a6ae848f5bb25a2e8b061b5f576d5b9e0c11b941dc1e6bed078e6529cd6` Source locator `/tasks/overall/google~1gemini-3-flash-preview` | | | | | | | | | [ARC-AGI-2](https://benchize.com/benchmarks/arc-prize%3Av2_Semi_Private.md) | 33.61% | — | 95 / 212 | 0.23usd per task | — | [arc prize · v2](https://arcprize.org/media/data/leaderboard/v2.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 0.3361 \| score_fraction \| v2_Semi_Private \| Higher is better \| \| `cost_per_task` \| 0.2314 \| usd_per_task \| v2_Semi_Private \| Lower is better \| ### Configuration evaluationSplit v2_Semi_Private harness Not reported modelGroup gemini-3-flash-preview-thinking modelType CoT Reasoning effort high sourceModelVersion gemini-3-flash-preview-thinking-high thinkingEnabled true ### Source [Original source](https://arcprize.org/media/data/leaderboard/v2.json) Source arc prize Original model label Gemini 3 Flash Preview (High) Source version v2 Observed 2026-10-21T16:34:20.750Z Evidence SHA-256 `bca68e12cc9a01954ae21b641844c1ecbabe146cb24b87f0f8e9b1db27433c29` Source locator `/evaluations/113` | | | | | | | | | [AIME 2025](https://benchize.com/benchmarks/aime-2025.md) | 95.42% | SE ±0.61 pp | 3 / 14 | $0.05 | 87.22source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_aime.C3_jItdR.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 95.417 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.048415 \| USD \| per_test \| Lower is better \| \| `latency` \| 87.215 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens Not reported provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.61 pp sourcePopulation 96 stderr 0.61 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_aime.C3_jItdR.json) Source Vals AI Original model label google/gemini-3-flash-preview Source version 1 Observed 2026-10-21T16:46:01.920Z Evidence SHA-256 `7d016210b39d24351bbb5d12be09a973decac461bb2486eb1f23724fed2b7e6c` Source locator `/tasks/aime_2025/google~1gemini-3-flash-preview` | | | | | | | | | [LegalBench / rhetoric tasks](https://benchize.com/benchmarks/vals-ai%3Alegal_bench%3Arhetoric_tasks.md) | 83.87% | SE ±0.88 pp | 2 / 44 | $0.00181 | 4.04source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_bench.D6uTgA3b.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 83.87 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.001805 \| USD \| per_test \| Lower is better \| \| `latency` \| 4.036 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.88 pp sourcePopulation 151 stderr 0.882 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_bench.D6uTgA3b.json) Source Vals AI Original model label google/gemini-3-flash-preview Source version 1 Observed 2026-10-21T16:45:53.390Z Evidence SHA-256 `c8486ba756baa5cd3955d035edff67440e8db23e69d62ad1a9e18d15b8c4bc52` Source locator `/tasks/rhetoric_tasks/google~1gemini-3-flash-preview` | | | | | | | | | [Finance Agent (v2) / comparables](https://benchize.com/benchmarks/vals-ai%3Afabv2%3Acomparables.md) | 38.56% | SE ±1.77 pp | 39 / 50 | $0.76 | 279.59source duration | [Vals AI · 2](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 38.555 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.762386 \| USD \| per_test \| Lower is better \| \| `latency` \| 279.59 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.77 pp sourcePopulation 80 stderr 1.771 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) Source Vals AI Original model label google/gemini-3-flash-preview Source version 2 Observed 2026-10-21T16:45:57.035Z Evidence SHA-256 `f731b20a26c126725d125519f2b5841aa45254656dba859a9570d7ce7e5d8191` Source locator `/tasks/comparables/google~1gemini-3-flash-preview` | | | | | | | | | [Finance Agent (v2) / financial modeling](https://benchize.com/benchmarks/vals-ai%3Afabv2%3Afinancial_modeling.md) | 14.43% | SE ±2.07 pp | 38 / 50 | $0.49 | 250.72source duration | [Vals AI · 2](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 14.434 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.486996 \| USD \| per_test \| Lower is better \| \| `latency` \| 250.719 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.07 pp sourcePopulation 80 stderr 2.066 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) Source Vals AI Original model label google/gemini-3-flash-preview Source version 2 Observed 2026-10-21T16:45:57.035Z Evidence SHA-256 `f731b20a26c126725d125519f2b5841aa45254656dba859a9570d7ce7e5d8191` Source locator `/tasks/financial_modeling/google~1gemini-3-flash-preview` | | | | | | | | | [GPQA Diamond / diamond few shot cot](https://benchize.com/benchmarks/vals-ai%3Agpqa%3Adiamond_few_shot_cot.md) | 86.36% | SE ±2.44 pp | 21 / 40 | $0.05 | 85.17source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_gpqa.ghZZBpyU.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 86.364 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.04534 \| USD \| per_test \| Lower is better \| \| `latency` \| 85.167 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.44 pp sourcePopulation 138 stderr 2.439 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_gpqa.ghZZBpyU.json) Source Vals AI Original model label google/gemini-3-flash-preview Source version 1 Observed 2026-10-21T16:46:06.040Z Evidence SHA-256 `951eb5224097733a9d09d06c14908ae2980e2e640e539f2c284877c5092d7077` Source locator `/tasks/diamond_few_shot_cot/google~1gemini-3-flash-preview` | | | | | | | | | [Legal Research Bench / criminal](https://benchize.com/benchmarks/vals-ai%3Alegal_research%3Acriminal.md) | 12.28% | SE ±4.39 pp | 42 / 50 | $0.14 | 221.17source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 12.281 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.13559 \| USD \| per_test \| Lower is better \| \| `latency` \| 221.166 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±4.39 pp sourcePopulation 82 stderr 4.386 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) Source Vals AI Original model label google/gemini-3-flash-preview Source version 1 Observed 2026-10-21T16:45:52.389Z Evidence SHA-256 `26c58dfd47d0d2dc86f33c6d056f30309e0ee63570e7cd752c593b385f9d862e` Source locator `/tasks/criminal/google~1gemini-3-flash-preview` | | | | | | | | | [CorpFin v2 / overall](https://benchize.com/benchmarks/vals-ai%3Acorp_fin_v2%3Aoverall.md) | 66.43% | SE ±0.93 pp | 9 / 39 | $0.05 | 11.28source duration | [Vals AI · 2](https://www.vals.ai/_astro/benchmark_view_corp_fin_v2.CectKwl1.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 66.434 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.047154 \| USD \| per_test \| Lower is better \| \| `latency` \| 11.279 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.93 pp sourcePopulation 134 stderr 0.93 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_corp_fin_v2.CectKwl1.json) Source Vals AI Original model label google/gemini-3-flash-preview Source version 2 Observed 2026-10-21T16:45:57.998Z Evidence SHA-256 `7cf2217ab3728d5a84bffce82aab9fc52d7f810b0cbe8c65b387df47f3d31c9b` Source locator `/tasks/overall/google~1gemini-3-flash-preview` | | | | | | | | | [OTIS Mock AIME 2024–2025](https://benchize.com/benchmarks/epoch-ai%3Aotis_mock_aime_2024_2025.md) | 95.56% | SE ±3.11 pp | 28 / 123 | — | — | [epoch ai · OTIS Mock AIME 2024-2025](https://epoch.ai/data/benchmark_data.zip#file=otis_mock_aime_2024_2025.csv) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 0.9555555555555556 \| score_fraction \| Best score (across scorers) \| Higher is better \| \| `mean_score` \| 0.9555555555555556 \| score_fraction \| mean_score \| Higher is better \| ### Configuration benchmarkVersion OTIS Mock AIME 2024-2025 reasoningBudget Not reported Reasoning effort high sourceModelVersion gemini-3-flash-preview_high ### Reported statistics Uncertainty Standard error: ±3.11 pp standardError 0.031067790907534743 ### Source [Original source](https://epoch.ai/data/benchmark_data.zip#file=otis_mock_aime_2024_2025.csv) Source epoch ai Original model label gemini-3-flash-preview Source version OTIS Mock AIME 2024-2025 Evaluated 2026-08-06T22:25:57.000Z Observed 2026-10-21T16:30:24.199Z Evidence SHA-256 `5b411612f91e155dff4b09d09b548b42f7936482ea8b3c00b510bd46e2f6bc2c` Source locator `/rows/127` | | | | | | | | | [Legal Research Bench / weighted pass criminal](https://benchize.com/benchmarks/vals-ai%3Alegal_research%3Aweighted_pass__criminal.md) | 72.14% | SE ±2.88 pp | 41 / 50 | $0.14 | 221.17source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 72.137 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.13559 \| USD \| per_test \| Lower is better \| \| `latency` \| 221.166 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.88 pp sourcePopulation 82 stderr 2.88 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) Source Vals AI Original model label google/gemini-3-flash-preview Source version 1 Observed 2026-10-21T16:45:52.389Z Evidence SHA-256 `26c58dfd47d0d2dc86f33c6d056f30309e0ee63570e7cd752c593b385f9d862e` Source locator `/tasks/weighted_pass__criminal/google~1gemini-3-flash-preview` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / criteria pass rate corporate governance](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__corporate_governance.md) | 57.88% | SE ±2.89 pp | 45 / 49 | $0.17 | 166.63source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 57.877 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.171496 \| USD \| per_test \| Lower is better \| \| `latency` \| 166.626 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.89 pp sourcePopulation 81 stderr 2.894 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label google/gemini-3-flash-preview Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__corporate_governance/google~1gemini-3-flash-preview` | | | | | | | | | [LiveCodeBench / overall](https://benchize.com/benchmarks/vals-ai%3Alcb%3Aoverall.md) | 85.59% | SE ±1 pp | 18 / 43 | $0.05 | 91.93source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_lcb.miyRUvVa.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 85.591 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.050525 \| USD \| per_test \| Lower is better \| \| `latency` \| 91.931 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1 pp sourcePopulation 143 stderr 1.004 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_lcb.miyRUvVa.json) Source Vals AI Original model label google/gemini-3-flash-preview Source version 1 Observed 2026-10-21T16:46:14.220Z Evidence SHA-256 `1a7c7dabc9d95a0182886b1edc42623987cac2095a0d407a5e334f37acdfa2f5` Source locator `/tasks/overall/google~1gemini-3-flash-preview` | | | | | | | | | [MMLU Pro / other](https://benchize.com/benchmarks/vals-ai%3Ammlu_pro%3Aother.md) | 86.91% | SE ±1.11 pp | 9 / 40 | $0.01 | 24.51source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_mmlu_pro.y416pXDi.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 86.905 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.012398 \| USD \| per_test \| Lower is better \| \| `latency` \| 24.514 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.11 pp sourcePopulation 138 stderr 1.11 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_mmlu_pro.y416pXDi.json) Source Vals AI Original model label google/gemini-3-flash-preview Source version 1 Observed 2026-10-21T16:46:06.503Z Evidence SHA-256 `dab025507f46c716dd8d145e90ec0a737d9c56219853f41e4b1cc3c5e8c222b4` Source locator `/tasks/other/google~1gemini-3-flash-preview` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / criteria pass rate energy natural resources](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__energy_natural_resources.md) | 48.19% | SE ±2.88 pp | 42 / 49 | $0.37 | 327.08source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 48.185 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.37291 \| USD \| per_test \| Lower is better \| \| `latency` \| 327.076 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.88 pp sourcePopulation 81 stderr 2.875 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label google/gemini-3-flash-preview Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__energy_natural_resources/google~1gemini-3-flash-preview` | | | | | | | | | [Legal Research Bench / administrative regulatory](https://benchize.com/benchmarks/vals-ai%3Alegal_research%3Aadministrative_regulatory.md) | 27.27% | SE ±5.52 pp | 39 / 50 | $0.13 | 166.67source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 27.273 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.129186 \| USD \| per_test \| Lower is better \| \| `latency` \| 166.672 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±5.52 pp sourcePopulation 82 stderr 5.524 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) Source Vals AI Original model label google/gemini-3-flash-preview Source version 1 Observed 2026-10-21T16:45:52.389Z Evidence SHA-256 `26c58dfd47d0d2dc86f33c6d056f30309e0ee63570e7cd752c593b385f9d862e` Source locator `/tasks/administrative_regulatory/google~1gemini-3-flash-preview` | | | | | | | | | [Legal Research Bench / civil litigation](https://benchize.com/benchmarks/vals-ai%3Alegal_research%3Acivil_litigation.md) | 11.11% | SE ±6.16 pp | 41 / 50 | $0.13 | 215.91source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 11.111 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.132078 \| USD \| per_test \| Lower is better \| \| `latency` \| 215.912 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±6.16 pp sourcePopulation 82 stderr 6.163 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) Source Vals AI Original model label google/gemini-3-flash-preview Source version 1 Observed 2026-10-21T16:45:52.389Z Evidence SHA-256 `26c58dfd47d0d2dc86f33c6d056f30309e0ee63570e7cd752c593b385f9d862e` Source locator `/tasks/civil_litigation/google~1gemini-3-flash-preview` | | | | | | | | | [LiveCodeBench / medium](https://benchize.com/benchmarks/vals-ai%3Alcb%3Amedium.md) | 87.99% | SE ±1.66 pp | 18 / 43 | $0.05 | 92.76source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_lcb.miyRUvVa.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 87.99 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.051088 \| USD \| per_test \| Lower is better \| \| `latency` \| 92.758 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.66 pp sourcePopulation 143 stderr 1.661 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_lcb.miyRUvVa.json) Source Vals AI Original model label google/gemini-3-flash-preview Source version 1 Observed 2026-10-21T16:46:14.220Z Evidence SHA-256 `1a7c7dabc9d95a0182886b1edc42623987cac2095a0d407a5e334f37acdfa2f5` Source locator `/tasks/medium/google~1gemini-3-flash-preview` | | | | | | | | | [EMB / scratch](https://benchize.com/benchmarks/vals-ai%3Aemb%3Ascratch.md) | 30.36% | SE ±3.66 pp | 44 / 47 | $0.4 | 302.05source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 30.356 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.402966 \| USD \| per_test \| Lower is better \| \| `latency` \| 302.045 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±3.66 pp sourcePopulation 76 stderr 3.656 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) Source Vals AI Original model label google/gemini-3-flash-preview Source version 1 Observed 2026-10-21T16:45:57.495Z Evidence SHA-256 `d807aee620ea47152a1fca0257a8dd12a1be4512c76cc1744c1ae9e285180918` Source locator `/tasks/scratch/google~1gemini-3-flash-preview` | | | | | | | | | [Mystery Game Puzzles](https://benchize.com/benchmarks/epoch-ai%3Amystery_game_puzzles.md) | 20% | SE ±4.02 pp | 39 / 71 | — | — | [epoch ai · Mystery Game Puzzles](https://epoch.ai/data/benchmark_data.zip#file=mystery_game_puzzles.csv) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 0.2 \| score_fraction \| Best score (across scorers) \| Higher is better \| \| `mean_score` \| 0.2 \| score_fraction \| mean_score \| Higher is better \| ### Configuration benchmarkVersion Mystery Game Puzzles reasoningBudget Not reported Reasoning effort high sourceModelVersion gemini-3-flash-preview_high ### Reported statistics Uncertainty Standard error: ±4.02 pp standardError 0.04020151261036849 ### Source [Original source](https://epoch.ai/data/benchmark_data.zip#file=mystery_game_puzzles.csv) Source epoch ai Original model label gemini-3-flash-preview Source version Mystery Game Puzzles Evaluated 2026-08-05T23:47:26.000Z Observed 2026-10-21T16:30:24.199Z Evidence SHA-256 `795b977bc82c460a5e2c1d6dd7b2569ddb21853f3829c4f6fdd16edfaf47bfcf` Source locator `/rows/110` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / emerging companies venture capital](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Aemerging_companies_venture_capital.md) | 0% | SE ±0 pp | 18 / 49 | $0.28 | 207.13source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.279031 \| USD \| per_test \| Lower is better \| \| `latency` \| 207.128 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 81 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label google/gemini-3-flash-preview Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/emerging_companies_venture_capital/google~1gemini-3-flash-preview` | | | | | | | | | [LegalBench / rule tasks](https://benchize.com/benchmarks/vals-ai%3Alegal_bench%3Arule_tasks.md) | 85.29% | SE ±1.28 pp | 11 / 44 | $0.00211 | 5.22source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_bench.D6uTgA3b.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 85.291 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.002112 \| USD \| per_test \| Lower is better \| \| `latency` \| 5.224 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.28 pp sourcePopulation 151 stderr 1.281 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_bench.D6uTgA3b.json) Source Vals AI Original model label google/gemini-3-flash-preview Source version 1 Observed 2026-10-21T16:45:53.390Z Evidence SHA-256 `c8486ba756baa5cd3955d035edff67440e8db23e69d62ad1a9e18d15b8c4bc52` Source locator `/tasks/rule_tasks/google~1gemini-3-flash-preview` | | | | | | | | | [Legal Research Bench / weighted pass](https://benchize.com/benchmarks/vals-ai%3Alegal_research%3Aweighted_pass.md) | 75.17% | SE ±1.65 pp | 41 / 50 | $0.14 | 187.7source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 75.174 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.135395 \| USD \| per_test \| Lower is better \| \| `latency` \| 187.7 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.65 pp sourcePopulation 82 stderr 1.651 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) Source Vals AI Original model label google/gemini-3-flash-preview Source version 1 Observed 2026-10-21T16:45:52.389Z Evidence SHA-256 `26c58dfd47d0d2dc86f33c6d056f30309e0ee63570e7cd752c593b385f9d862e` Source locator `/tasks/weighted_pass/google~1gemini-3-flash-preview` | | | | | | | | | [AIME / overall](https://benchize.com/benchmarks/vals-ai%3Aaime%3Aoverall.md) | 95.63% | SE ±0.52 pp | 3 / 14 | $0.05 | 84.01source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_aime.C3_jItdR.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 95.625 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.046328 \| USD \| per_test \| Lower is better \| \| `latency` \| 84.007 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens Not reported provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.52 pp sourcePopulation 96 stderr 0.516 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_aime.C3_jItdR.json) Source Vals AI Original model label google/gemini-3-flash-preview Source version 1 Observed 2026-10-21T16:46:01.920Z Evidence SHA-256 `7d016210b39d24351bbb5d12be09a973decac461bb2486eb1f23724fed2b7e6c` Source locator `/tasks/overall/google~1gemini-3-flash-preview` | | | | | | | | | [EMB / dcf](https://benchize.com/benchmarks/vals-ai%3Aemb%3Adcf.md) | 21.39% | SE ±6.31 pp | 44 / 47 | $0.32 | 294.13source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 21.392 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.323658 \| USD \| per_test \| Lower is better \| \| `latency` \| 294.126 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±6.31 pp sourcePopulation 76 stderr 6.312 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) Source Vals AI Original model label google/gemini-3-flash-preview Source version 1 Observed 2026-10-21T16:45:57.495Z Evidence SHA-256 `d807aee620ea47152a1fca0257a8dd12a1be4512c76cc1744c1ae9e285180918` Source locator `/tasks/dcf/google~1gemini-3-flash-preview` | | | | | | | | | [SAGE / calculus](https://benchize.com/benchmarks/vals-ai%3Asage%3Acalculus.md) | 70.99% | SE ±5.5 pp | 18 / 36 | $0.01 | 60.11source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_sage.RRwIIzO5.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 70.988 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.014869 \| USD \| per_test \| Lower is better \| \| `latency` \| 60.109 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 30000 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±5.5 pp sourcePopulation 91 stderr 5.5 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_sage.RRwIIzO5.json) Source Vals AI Original model label google/gemini-3-flash-preview Source version 1 Observed 2026-10-21T16:46:07.515Z Evidence SHA-256 `cc5fb3fecdb0382d492ff167a42ad84b9c5c3883373c78ed87baaa972b8b6f7c` Source locator `/tasks/calculus/google~1gemini-3-flash-preview` | | | | | | | | | [LiveCodeBench / easy](https://benchize.com/benchmarks/vals-ai%3Alcb%3Aeasy.md) | 99.07% | SE ±0.54 pp | 1 / 43 | $0.03 | 47.03source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_lcb.miyRUvVa.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 99.068 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.02567 \| USD \| per_test \| Lower is better \| \| `latency` \| 47.032 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.54 pp sourcePopulation 143 stderr 0.535 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_lcb.miyRUvVa.json) Source Vals AI Original model label google/gemini-3-flash-preview Source version 1 Observed 2026-10-21T16:46:14.220Z Evidence SHA-256 `1a7c7dabc9d95a0182886b1edc42623987cac2095a0d407a5e334f37acdfa2f5` Source locator `/tasks/easy/google~1gemini-3-flash-preview` | | | | | | | | | [ProgramBench / strict](https://benchize.com/benchmarks/vals-ai%3Aprogrambench%3Astrict.md) | 0% | SE ±0 pp | 23 / 40 | $0.44 | 418.92source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_programbench.C_J7ZU4X.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.439965 \| USD \| per_test \| Lower is better \| \| `latency` \| 418.92 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 64 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_programbench.C_J7ZU4X.json) Source Vals AI Original model label google/gemini-3-flash-preview Source version 1 Observed 2026-10-21T16:46:10.137Z Evidence SHA-256 `e9bbb4378c7a058441a21d456baf9316b96c2cf5b58d371a3c7c803f5000bd8c` Source locator `/tasks/strict/google~1gemini-3-flash-preview` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / criteria pass rate corporate ma](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__corporate_ma.md) | 41.99% | SE ±2.8 pp | 47 / 49 | $0.25 | 134.36source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 41.987 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.246824 \| USD \| per_test \| Lower is better \| \| `latency` \| 134.364 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.8 pp sourcePopulation 81 stderr 2.799 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label google/gemini-3-flash-preview Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__corporate_ma/google~1gemini-3-flash-preview` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / antitrust competition](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Aantitrust_competition.md) | 0% | SE ±0 pp | 7 / 49 | $0.24 | 236.17source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.244675 \| USD \| per_test \| Lower is better \| \| `latency` \| 236.172 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 81 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label google/gemini-3-flash-preview Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/antitrust_competition/google~1gemini-3-flash-preview` | | | | | | | | | [EMB / operating](https://benchize.com/benchmarks/vals-ai%3Aemb%3Aoperating.md) | 20.43% | SE ±7.16 pp | 44 / 47 | $0.37 | 295.09source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 20.428 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.367395 \| USD \| per_test \| Lower is better \| \| `latency` \| 295.092 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±7.16 pp sourcePopulation 76 stderr 7.156 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) Source Vals AI Original model label google/gemini-3-flash-preview Source version 1 Observed 2026-10-21T16:45:57.495Z Evidence SHA-256 `d807aee620ea47152a1fca0257a8dd12a1be4512c76cc1744c1ae9e285180918` Source locator `/tasks/operating/google~1gemini-3-flash-preview` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / criteria pass rate data privacy cybersecurity](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__data_privacy_cybersecurity.md) | 40.99% | SE ±2.93 pp | 48 / 49 | $0.2 | 127.52source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 40.989 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.195281 \| USD \| per_test \| Lower is better \| \| `latency` \| 127.518 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.93 pp sourcePopulation 81 stderr 2.929 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label google/gemini-3-flash-preview Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__data_privacy_cybersecurity/google~1gemini-3-flash-preview` | | | | | | | | | [LegalBench / interpretation tasks](https://benchize.com/benchmarks/vals-ai%3Alegal_bench%3Ainterpretation_tasks.md) | 83.91% | SE ±0.3 pp | 9 / 44 | $0.00258 | 5.67source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_bench.D6uTgA3b.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 83.908 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.002582 \| USD \| per_test \| Lower is better \| \| `latency` \| 5.669 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.3 pp sourcePopulation 151 stderr 0.302 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_bench.D6uTgA3b.json) Source Vals AI Original model label google/gemini-3-flash-preview Source version 1 Observed 2026-10-21T16:45:53.390Z Evidence SHA-256 `c8486ba756baa5cd3955d035edff67440e8db23e69d62ad1a9e18d15b8c4bc52` Source locator `/tasks/interpretation_tasks/google~1gemini-3-flash-preview` | | | | | | | | | [Finance Agent (v2) / precedents](https://benchize.com/benchmarks/vals-ai%3Afabv2%3Aprecedents.md) | 18.14% | SE ±0.79 pp | 40 / 50 | $0.96 | 453.5source duration | [Vals AI · 2](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 18.138 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.955464 \| USD \| per_test \| Lower is better \| \| `latency` \| 453.499 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.79 pp sourcePopulation 80 stderr 0.789 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) Source Vals AI Original model label google/gemini-3-flash-preview Source version 2 Observed 2026-10-21T16:45:57.035Z Evidence SHA-256 `f731b20a26c126725d125519f2b5841aa45254656dba859a9570d7ce7e5d8191` Source locator `/tasks/precedents/google~1gemini-3-flash-preview` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / criteria pass rate environmental esg](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__environmental_esg.md) | 36.92% | SE ±2.89 pp | 47 / 49 | $0.23 | 154.23source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 36.918 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.230865 \| USD \| per_test \| Lower is better \| \| `latency` \| 154.234 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.89 pp sourcePopulation 81 stderr 2.894 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label google/gemini-3-flash-preview Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__environmental_esg/google~1gemini-3-flash-preview` | | | | | | | | | [MedQA / black](https://benchize.com/benchmarks/vals-ai%3Amedqa%3Ablack.md) | 95.7% | SE ±0.45 pp | 2 / 12 | $0.01 | 27.73source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_medqa.Cco1kvog.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 95.7 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.013337 \| USD \| per_test \| Lower is better \| \| `latency` \| 27.728 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.45 pp sourcePopulation 95 stderr 0.454 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_medqa.Cco1kvog.json) Source Vals AI Original model label google/gemini-3-flash-preview Source version 1 Observed 2026-10-21T16:46:00.691Z Evidence SHA-256 `d67905978d932f64698ebe9e230d9519e5bae1c9d7176dda8379dc3874b6f90f` Source locator `/tasks/black/google~1gemini-3-flash-preview` | | | | | | | | | [Terminal-Bench 2.1 / medium](https://benchize.com/benchmarks/vals-ai%3Aterminal-bench-2-1%3Amedium.md) | 60% | SE ±2.78 pp | 35 / 43 | $0.25 | 608.69source duration | [Vals AI · 2.1](https://www.vals.ai/_astro/benchmark_view_terminal_bench_2_1.CdyoObsA.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 60 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.249705 \| USD \| per_test \| Lower is better \| \| `latency` \| 608.692 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.78 pp sourcePopulation 76 stderr 2.777 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_terminal_bench_2_1.CdyoObsA.json) Source Vals AI Original model label google/gemini-3-flash-preview Source version 2.1 Observed 2026-10-21T16:46:13.216Z Evidence SHA-256 `a5306805cdf86495bdcceff6b2c8534a3768b9853f5077895dab975b809b054f` Source locator `/tasks/medium/google~1gemini-3-flash-preview` | | | | | | | | | [Finance Agent (v2) / market analysis](https://benchize.com/benchmarks/vals-ai%3Afabv2%3Amarket_analysis.md) | 59% | SE ±3.03 pp | 32 / 50 | $0.3 | 151.69source duration | [Vals AI · 2](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 59.004 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.296649 \| USD \| per_test \| Lower is better \| \| `latency` \| 151.685 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±3.03 pp sourcePopulation 80 stderr 3.028 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) Source Vals AI Original model label google/gemini-3-flash-preview Source version 2 Observed 2026-10-21T16:45:57.035Z Evidence SHA-256 `f731b20a26c126725d125519f2b5841aa45254656dba859a9570d7ce7e5d8191` Source locator `/tasks/market_analysis/google~1gemini-3-flash-preview` | | | | | | | | | [MGSM / mgsm fr](https://benchize.com/benchmarks/vals-ai%3Amgsm%3Amgsm_fr.md) | 87.2% | SE ±2.11 pp | 2 / 8 | $0.00696 | 13.66source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_mgsm.CneCP-rm.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 87.2 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.006964 \| USD \| per_test \| Lower is better \| \| `latency` \| 13.663 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.11 pp sourcePopulation 75 stderr 2.113 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_mgsm.CneCP-rm.json) Source Vals AI Original model label google/gemini-3-flash-preview Source version 1 Observed 2026-10-21T16:46:02.976Z Evidence SHA-256 `1b87f953e52c00bf4bdd90c3add26165f644eeb7a74a8382d2a0324ff0cc1040` Source locator `/tasks/mgsm_fr/google~1gemini-3-flash-preview` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / criteria pass rate trusts estates private client](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__trusts_estates_private_client.md) | 54.2% | SE ±3.08 pp | 46 / 49 | $0.22 | 187.46source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 54.198 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.223928 \| USD \| per_test \| Lower is better \| \| `latency` \| 187.458 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±3.08 pp sourcePopulation 81 stderr 3.084 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label google/gemini-3-flash-preview Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__trusts_estates_private_client/google~1gemini-3-flash-preview` | | | | | | | | | [Legal Research Bench / immigration](https://benchize.com/benchmarks/vals-ai%3Alegal_research%3Aimmigration.md) | 27.27% | SE ±14.08 pp | 21 / 50 | $0.09 | 128.85source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 27.273 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.088481 \| USD \| per_test \| Lower is better \| \| `latency` \| 128.854 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±14.08 pp sourcePopulation 82 stderr 14.084 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) Source Vals AI Original model label google/gemini-3-flash-preview Source version 1 Observed 2026-10-21T16:45:52.389Z Evidence SHA-256 `26c58dfd47d0d2dc86f33c6d056f30309e0ee63570e7cd752c593b385f9d862e` Source locator `/tasks/immigration/google~1gemini-3-flash-preview` | | | | | | | | | [Legal Research Bench / weighted pass civil litigation](https://benchize.com/benchmarks/vals-ai%3Alegal_research%3Aweighted_pass__civil_litigation.md) | 77.01% | SE ±4.02 pp | 40 / 50 | $0.13 | 215.91source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 77.014 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.132078 \| USD \| per_test \| Lower is better \| \| `latency` \| 215.912 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±4.02 pp sourcePopulation 82 stderr 4.021 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) Source Vals AI Original model label google/gemini-3-flash-preview Source version 1 Observed 2026-10-21T16:45:52.389Z Evidence SHA-256 `26c58dfd47d0d2dc86f33c6d056f30309e0ee63570e7cd752c593b385f9d862e` Source locator `/tasks/weighted_pass__civil_litigation/google~1gemini-3-flash-preview` | | | | | | | | | [ProgramBench / partial](https://benchize.com/benchmarks/vals-ai%3Aprogrambench%3Apartial.md) | 33.83% | SE ±1.62 pp | 34 / 40 | $0.44 | 418.92source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_programbench.C_J7ZU4X.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 33.83 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.439965 \| USD \| per_test \| Lower is better \| \| `latency` \| 418.92 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.62 pp sourcePopulation 64 stderr 1.622 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_programbench.C_J7ZU4X.json) Source Vals AI Original model label google/gemini-3-flash-preview Source version 1 Observed 2026-10-21T16:46:10.137Z Evidence SHA-256 `e9bbb4378c7a058441a21d456baf9316b96c2cf5b58d371a3c7c803f5000bd8c` Source locator `/tasks/partial/google~1gemini-3-flash-preview` | | | | | | | | | [MMLU Pro / physics](https://benchize.com/benchmarks/vals-ai%3Ammlu_pro%3Aphysics.md) | 93.38% | SE ±0.69 pp | 12 / 40 | $0.02 | 42.84source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_mmlu_pro.y416pXDi.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 93.38 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.023862 \| USD \| per_test \| Lower is better \| \| `latency` \| 42.839 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.69 pp sourcePopulation 138 stderr 0.69 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_mmlu_pro.y416pXDi.json) Source Vals AI Original model label google/gemini-3-flash-preview Source version 1 Observed 2026-10-21T16:46:06.503Z Evidence SHA-256 `dab025507f46c716dd8d145e90ec0a737d9c56219853f41e4b1cc3c5e8c222b4` Source locator `/tasks/physics/google~1gemini-3-flash-preview` | | | | | | | | | [MMLU Pro / engineering](https://benchize.com/benchmarks/vals-ai%3Ammlu_pro%3Aengineering.md) | 87.31% | SE ±1.07 pp | 12 / 40 | $0.04 | 66.3source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_mmlu_pro.y416pXDi.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 87.307 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.037611 \| USD \| per_test \| Lower is better \| \| `latency` \| 66.298 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.07 pp sourcePopulation 138 stderr 1.069 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_mmlu_pro.y416pXDi.json) Source Vals AI Original model label google/gemini-3-flash-preview Source version 1 Observed 2026-10-21T16:46:06.503Z Evidence SHA-256 `dab025507f46c716dd8d145e90ec0a737d9c56219853f41e4b1cc3c5e8c222b4` Source locator `/tasks/engineering/google~1gemini-3-flash-preview` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / funds asset management](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Afunds_asset_management.md) | 0% | SE ±0 pp | 11 / 49 | $0.62 | 245.1source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.620247 \| USD \| per_test \| Lower is better \| \| `latency` \| 245.102 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 81 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label google/gemini-3-flash-preview Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/funds_asset_management/google~1gemini-3-flash-preview` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / data privacy cybersecurity](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Adata_privacy_cybersecurity.md) | 0% | SE ±0 pp | 13 / 49 | $0.2 | 127.52source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.195281 \| USD \| per_test \| Lower is better \| \| `latency` \| 127.518 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 81 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label google/gemini-3-flash-preview Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/data_privacy_cybersecurity/google~1gemini-3-flash-preview` | | | | | | | | | [Code Migration / cobol](https://benchize.com/benchmarks/vals-ai%3Acode-migration%3Acobol.md) | 0% | SE ±0 pp | 46 / 47 | $0.3 | 254.92source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_code_migration.Dttzoqk3.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.300925 \| USD \| per_test \| Lower is better \| \| `latency` \| 254.923 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 79 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_code_migration.Dttzoqk3.json) Source Vals AI Original model label google/gemini-3-flash-preview Source version 1 Observed 2026-10-21T16:46:12.712Z Evidence SHA-256 `99194597f2c76c402195be47839409046e18bd30454616da6dd40b736a088fa4` Source locator `/tasks/cobol/google~1gemini-3-flash-preview` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / environmental esg](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Aenvironmental_esg.md) | 0% | SE ±0 pp | 1 / 49 | $0.23 | 154.23source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.230865 \| USD \| per_test \| Lower is better \| \| `latency` \| 154.234 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 81 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label google/gemini-3-flash-preview Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/environmental_esg/google~1gemini-3-flash-preview` | | | | | | | | | [Legal Research Bench / overall](https://benchize.com/benchmarks/vals-ai%3Alegal_research%3Aoverall.md) | 18.27% | SE ±2.69 pp | 40 / 50 | $0.14 | 187.7source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 18.269 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.135395 \| USD \| per_test \| Lower is better \| \| `latency` \| 187.7 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.69 pp sourcePopulation 82 stderr 2.686 tokenTotals {"cache_read_tokens":50273506,"cache_write_tokens":0,"input_tokens":80643637,"output_tokens":3487795,"reasoning_tokens":2539699} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) Source Vals AI Original model label google/gemini-3-flash-preview Source version 1 Observed 2026-10-21T16:45:52.389Z Evidence SHA-256 `26c58dfd47d0d2dc86f33c6d056f30309e0ee63570e7cd752c593b385f9d862e` Source locator `/tasks/overall/google~1gemini-3-flash-preview` | | | | | | | | | [MGSM / mgsm ja](https://benchize.com/benchmarks/vals-ai%3Amgsm%3Amgsm_ja.md) | 92% | SE ±1.72 pp | 1 / 8 | $0.00706 | 14.05source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_mgsm.CneCP-rm.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 92 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.007059 \| USD \| per_test \| Lower is better \| \| `latency` \| 14.045 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.72 pp sourcePopulation 75 stderr 1.716 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_mgsm.CneCP-rm.json) Source Vals AI Original model label google/gemini-3-flash-preview Source version 1 Observed 2026-10-21T16:46:02.976Z Evidence SHA-256 `1b87f953e52c00bf4bdd90c3add26165f644eeb7a74a8382d2a0324ff0cc1040` Source locator `/tasks/mgsm_ja/google~1gemini-3-flash-preview` | | | | | | | | | [BullshitBench v2](https://benchize.com/benchmarks/bullshitbench%3Av2.md) | 10% | — | 164 / 191 | — | — | [bullshitbench · v2](https://api.github.com/repos/petergpt/bullshit-benchmark/contents/data/v2/latest/leaderboard.csv?ref=3c655234d15967738f95da696e3f938d4862dcf7) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `green_rate` \| 0.1 \| score_fraction \| all_attempts \| Higher is better \| \| `red_rate` \| 0.71 \| score_fraction \| all_attempts \| Lower is better \| \| `refusal_rate` \| 0.0 \| score_fraction \| all_attempts \| Lower is better \| \| `avg_score` \| 0.4067 \| score \| judge_grade_0_to_2 \| Higher is better \| ### Configuration Reasoning effort high suite v2 thinkingEnabled true ### Reported statistics acceptedCount 71 answeredCount 100 clearCount 10 errorCount 0 partialCount 19 rank 198 refusalCount 0 sampleSize 100 ### Source [Original source](https://api.github.com/repos/petergpt/bullshit-benchmark/contents/data/v2/latest/leaderboard.csv?ref=3c655234d15967738f95da696e3f938d4862dcf7) Source bullshitbench Original model label google/gemini-3-flash-preview Source version v2 Observed 2026-10-21T16:33:56.675Z Evidence SHA-256 `98b485e08764d1c97144bc4f7b28ced1a96d50850947c775870dfda0e5d749b5` Source locator `/rows/197` | | | | | | | | | [MMLU-Pro](https://benchize.com/benchmarks/mmlu-pro.md) | 88.59% | SE ±0.32 pp | 13 / 40 | $0.02 | 36.25source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_mmlu_pro.y416pXDi.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 88.592 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.019364 \| USD \| per_test \| Lower is better \| \| `latency` \| 36.249 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.32 pp sourcePopulation 138 stderr 0.315 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_mmlu_pro.y416pXDi.json) Source Vals AI Original model label google/gemini-3-flash-preview Source version 1 Observed 2026-10-21T16:46:06.503Z Evidence SHA-256 `dab025507f46c716dd8d145e90ec0a737d9c56219853f41e4b1cc3c5e8c222b4` Source locator `/tasks/overall/google~1gemini-3-flash-preview` | | | | | | | | | [LegalBench / conclusion tasks](https://benchize.com/benchmarks/vals-ai%3Alegal_bench%3Aconclusion_tasks.md) | 90.08% | SE ±0.61 pp | 10 / 44 | $0.00174 | 4.23source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_bench.D6uTgA3b.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 90.077 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.001736 \| USD \| per_test \| Lower is better \| \| `latency` \| 4.23 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.61 pp sourcePopulation 151 stderr 0.613 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_bench.D6uTgA3b.json) Source Vals AI Original model label google/gemini-3-flash-preview Source version 1 Observed 2026-10-21T16:45:53.390Z Evidence SHA-256 `c8486ba756baa5cd3955d035edff67440e8db23e69d62ad1a9e18d15b8c4bc52` Source locator `/tasks/conclusion_tasks/google~1gemini-3-flash-preview` | | | | | | | | | [Legal Research Bench / health](https://benchize.com/benchmarks/vals-ai%3Alegal_research%3Ahealth.md) | 30% | SE ±8.51 pp | 40 / 50 | $0.14 | 145.06source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 30 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.136684 \| USD \| per_test \| Lower is better \| \| `latency` \| 145.057 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±8.51 pp sourcePopulation 82 stderr 8.51 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) Source Vals AI Original model label google/gemini-3-flash-preview Source version 1 Observed 2026-10-21T16:45:52.389Z Evidence SHA-256 `26c58dfd47d0d2dc86f33c6d056f30309e0ee63570e7cd752c593b385f9d862e` Source locator `/tasks/health/google~1gemini-3-flash-preview` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / immigration](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Aimmigration.md) | 0% | SE ±0 pp | 6 / 49 | $0.15 | 121.82source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.154995 \| USD \| per_test \| Lower is better \| \| `latency` \| 121.822 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 81 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label google/gemini-3-flash-preview Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/immigration/google~1gemini-3-flash-preview` | | | | | | | | | [CorpFin v2 / exact pages](https://benchize.com/benchmarks/vals-ai%3Acorp_fin_v2%3Aexact_pages.md) | 68.07% | SE ±1.59 pp | 2 / 39 | $0.1 | 11.68source duration | [Vals AI · 2](https://www.vals.ai/_astro/benchmark_view_corp_fin_v2.CectKwl1.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 68.065 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.098516 \| USD \| per_test \| Lower is better \| \| `latency` \| 11.684 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.59 pp sourcePopulation 134 stderr 1.592 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_corp_fin_v2.CectKwl1.json) Source Vals AI Original model label google/gemini-3-flash-preview Source version 2 Observed 2026-10-21T16:45:57.998Z Evidence SHA-256 `7cf2217ab3728d5a84bffce82aab9fc52d7f810b0cbe8c65b387df47f3d31c9b` Source locator `/tasks/exact_pages/google~1gemini-3-flash-preview` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / international trade sanctions](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Ainternational_trade_sanctions.md) | 0% | SE ±0 pp | 4 / 49 | $0.23 | 292.1source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.228541 \| USD \| per_test \| Lower is better \| \| `latency` \| 292.1 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 81 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label google/gemini-3-flash-preview Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/international_trade_sanctions/google~1gemini-3-flash-preview` | | | | | | | | | [ProgramBench / overall](https://benchize.com/benchmarks/vals-ai%3Aprogrambench%3Aoverall.md) | 0% | SE ±0 pp | 23 / 40 | $0.44 | 418.92source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_programbench.C_J7ZU4X.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.439965 \| USD \| per_test \| Lower is better \| \| `latency` \| 418.92 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 64 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_programbench.C_J7ZU4X.json) Source Vals AI Original model label google/gemini-3-flash-preview Source version 1 Observed 2026-10-21T16:46:10.137Z Evidence SHA-256 `e9bbb4378c7a058441a21d456baf9316b96c2cf5b58d371a3c7c803f5000bd8c` Source locator `/tasks/overall/google~1gemini-3-flash-preview` | | | | | | | | | [SWE-bench / >4 hours](https://benchize.com/benchmarks/vals-ai%3Aswebench%3A%3E4%20hours.md) | 33.33% | SE ±33.33 pp | 13 / 39 | $0.35 | 366source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_swebench.Ae8ni7h0.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 33.333 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.349943 \| USD \| per_test \| Lower is better \| \| `latency` \| 365.995 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±33.33 pp sourcePopulation 88 stderr 33.333 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_swebench.Ae8ni7h0.json) Source Vals AI Original model label google/gemini-3-flash-preview Source version 1 Observed 2026-10-21T16:46:13.711Z Evidence SHA-256 `7a4e2e8485283caac500127419f68ec4bd5414c837ade114fc6afa5b2d34ccaa` Source locator `/tasks/>4 hours/google~1gemini-3-flash-preview` | | | | | | | | | [MGSM / mgsm te](https://benchize.com/benchmarks/vals-ai%3Amgsm%3Amgsm_te.md) | 94% | SE ±1.5 pp | 1 / 8 | $0.00542 | 10.51source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_mgsm.CneCP-rm.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 94 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.005416 \| USD \| per_test \| Lower is better \| \| `latency` \| 10.511 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.5 pp sourcePopulation 75 stderr 1.502 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_mgsm.CneCP-rm.json) Source Vals AI Original model label google/gemini-3-flash-preview Source version 1 Observed 2026-10-21T16:46:02.976Z Evidence SHA-256 `1b87f953e52c00bf4bdd90c3add26165f644eeb7a74a8382d2a0324ff0cc1040` Source locator `/tasks/mgsm_te/google~1gemini-3-flash-preview` | | | | | | | | | [MMLU Pro / business](https://benchize.com/benchmarks/vals-ai%3Ammlu_pro%3Abusiness.md) | 92.27% | SE ±0.95 pp | 13 / 40 | $0.02 | 31.51source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_mmlu_pro.y416pXDi.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 92.269 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.017383 \| USD \| per_test \| Lower is better \| \| `latency` \| 31.509 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.95 pp sourcePopulation 138 stderr 0.951 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_mmlu_pro.y416pXDi.json) Source Vals AI Original model label google/gemini-3-flash-preview Source version 1 Observed 2026-10-21T16:46:06.503Z Evidence SHA-256 `dab025507f46c716dd8d145e90ec0a737d9c56219853f41e4b1cc3c5e8c222b4` Source locator `/tasks/business/google~1gemini-3-flash-preview` | | | | | | | | | [Tax Agent Bench / weighted pass rule source lookup](https://benchize.com/benchmarks/vals-ai%3Atax_agent_bench%3Aweighted_pass__rule_source_lookup.md) | 29.32% | SE ±7.93 pp | 40 / 46 | $0.09 | 77.22source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 29.323 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.085031 \| USD \| per_test \| Lower is better \| \| `latency` \| 77.219 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±7.93 pp sourcePopulation 73 stderr 7.934 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) Source Vals AI Original model label google/gemini-3-flash-preview Source version 1 Observed 2026-10-21T16:45:55.958Z Evidence SHA-256 `446db182b67efb7923b3187a4040ec3a3ea492b8ad699a2b6d708d714cc063ac` Source locator `/tasks/weighted_pass__rule_source_lookup/google~1gemini-3-flash-preview` | | | | | | | | | [Tax Agent Bench / forms filings](https://benchize.com/benchmarks/vals-ai%3Atax_agent_bench%3Aforms_filings.md) | 6.67% | SE ±4.63 pp | 32 / 46 | $0.15 | 120.84source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 6.667 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.1451 \| USD \| per_test \| Lower is better \| \| `latency` \| 120.841 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±4.63 pp sourcePopulation 73 stderr 4.632 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) Source Vals AI Original model label google/gemini-3-flash-preview Source version 1 Observed 2026-10-21T16:45:55.958Z Evidence SHA-256 `446db182b67efb7923b3187a4040ec3a3ea492b8ad699a2b6d708d714cc063ac` Source locator `/tasks/forms_filings/google~1gemini-3-flash-preview` | | | | | | | | | [MortgageTax / semantic extraction](https://benchize.com/benchmarks/vals-ai%3Amortgage_tax%3Asemantic_extraction.md) | 61.69% | SE ±1.37 pp | 5 / 31 | $0.00507 | 8.14source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_mortgage_tax.DCd-oO5Y.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 61.685 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.005066 \| USD \| per_test \| Lower is better \| \| `latency` \| 8.14 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.37 pp sourcePopulation 98 stderr 1.371 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_mortgage_tax.DCd-oO5Y.json) Source Vals AI Original model label google/gemini-3-flash-preview Source version 1 Observed 2026-10-21T16:45:58.500Z Evidence SHA-256 `e5c844561f97d119d66d70269e726ecfb68be79eae39acf14ead3e95422e648c` Source locator `/tasks/semantic_extraction/google~1gemini-3-flash-preview` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / criteria pass rate antitrust competition](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__antitrust_competition.md) | 50.36% | SE ±3.02 pp | 45 / 49 | $0.24 | 236.17source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 50.362 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.244675 \| USD \| per_test \| Lower is better \| \| `latency` \| 236.172 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±3.02 pp sourcePopulation 81 stderr 3.015 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label google/gemini-3-flash-preview Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__antitrust_competition/google~1gemini-3-flash-preview` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / criteria pass rate healthcare life sciences](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__healthcare_life_sciences.md) | 59.08% | SE ±2.83 pp | 44 / 49 | $0.29 | 215.34source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 59.076 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.288454 \| USD \| per_test \| Lower is better \| \| `latency` \| 215.342 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.83 pp sourcePopulation 81 stderr 2.829 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label google/gemini-3-flash-preview Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__healthcare_life_sciences/google~1gemini-3-flash-preview` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / criteria pass rate banking finance](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__banking_finance.md) | 51.67% | SE ±3.05 pp | 47 / 49 | $0.22 | 124.38source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 51.673 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.220183 \| USD \| per_test \| Lower is better \| \| `latency` \| 124.376 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±3.05 pp sourcePopulation 81 stderr 3.053 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label google/gemini-3-flash-preview Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__banking_finance/google~1gemini-3-flash-preview` | | | | | | | | | [Tax Agent Bench / fact pattern analysis](https://benchize.com/benchmarks/vals-ai%3Atax_agent_bench%3Afact_pattern_analysis.md) | 9.3% | SE ±4.48 pp | 39 / 46 | $0.09 | 110.17source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 9.302 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.093878 \| USD \| per_test \| Lower is better \| \| `latency` \| 110.173 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±4.48 pp sourcePopulation 73 stderr 4.482 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) Source Vals AI Original model label google/gemini-3-flash-preview Source version 1 Observed 2026-10-21T16:45:55.958Z Evidence SHA-256 `446db182b67efb7923b3187a4040ec3a3ea492b8ad699a2b6d708d714cc063ac` Source locator `/tasks/fact_pattern_analysis/google~1gemini-3-flash-preview` | | | | | | | | | [MGSM / mgsm ru](https://benchize.com/benchmarks/vals-ai%3Amgsm%3Amgsm_ru.md) | 96% | SE ±1.24 pp | 1 / 8 | $0.00598 | 11.77source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_mgsm.CneCP-rm.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 96 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.005978 \| USD \| per_test \| Lower is better \| \| `latency` \| 11.772 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.24 pp sourcePopulation 75 stderr 1.239 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_mgsm.CneCP-rm.json) Source Vals AI Original model label google/gemini-3-flash-preview Source version 1 Observed 2026-10-21T16:46:02.976Z Evidence SHA-256 `1b87f953e52c00bf4bdd90c3add26165f644eeb7a74a8382d2a0324ff0cc1040` Source locator `/tasks/mgsm_ru/google~1gemini-3-flash-preview` | | | | | | | | | [Tax Agent Bench / rule source lookup](https://benchize.com/benchmarks/vals-ai%3Atax_agent_bench%3Arule_source_lookup.md) | 16.5% | SE ±6.85 pp | 39 / 46 | $0.09 | 77.22source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 16.5 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.085031 \| USD \| per_test \| Lower is better \| \| `latency` \| 77.219 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±6.85 pp sourcePopulation 73 stderr 6.853 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) Source Vals AI Original model label google/gemini-3-flash-preview Source version 1 Observed 2026-10-21T16:45:55.958Z Evidence SHA-256 `446db182b67efb7923b3187a4040ec3a3ea492b8ad699a2b6d708d714cc063ac` Source locator `/tasks/rule_source_lookup/google~1gemini-3-flash-preview` | | | | | | | | | [ProgramBench / almost](https://benchize.com/benchmarks/vals-ai%3Aprogrambench%3Aalmost.md) | 0% | SE ±0 pp | 34 / 40 | $0.44 | 418.92source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_programbench.C_J7ZU4X.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.439965 \| USD \| per_test \| Lower is better \| \| `latency` \| 418.92 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 64 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_programbench.C_J7ZU4X.json) Source Vals AI Original model label google/gemini-3-flash-preview Source version 1 Observed 2026-10-21T16:46:10.137Z Evidence SHA-256 `e9bbb4378c7a058441a21d456baf9316b96c2cf5b58d371a3c7c803f5000bd8c` Source locator `/tasks/almost/google~1gemini-3-flash-preview` | | | | | | | | | [Code Migration / code quality](https://benchize.com/benchmarks/vals-ai%3Acode-migration%3Acode_quality.md) | 2.66% | SE ±0.05 pp | 44 / 47 | $0.34 | 284.23source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_code_migration.Dttzoqk3.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 2.662 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.340087 \| USD \| per_test \| Lower is better \| \| `latency` \| 284.234 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.05 pp sourcePopulation 79 stderr 0.047 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_code_migration.Dttzoqk3.json) Source Vals AI Original model label google/gemini-3-flash-preview Source version 1 Observed 2026-10-21T16:46:12.712Z Evidence SHA-256 `99194597f2c76c402195be47839409046e18bd30454616da6dd40b736a088fa4` Source locator `/tasks/code_quality/google~1gemini-3-flash-preview` | | | | | | | | | [Tax Agent Bench / weighted pass forms filings](https://benchize.com/benchmarks/vals-ai%3Atax_agent_bench%3Aweighted_pass__forms_filings.md) | 27.52% | SE ±7.39 pp | 41 / 46 | $0.15 | 120.84source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 27.521 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.1451 \| USD \| per_test \| Lower is better \| \| `latency` \| 120.841 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±7.39 pp sourcePopulation 73 stderr 7.388 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) Source Vals AI Original model label google/gemini-3-flash-preview Source version 1 Observed 2026-10-21T16:45:55.958Z Evidence SHA-256 `446db182b67efb7923b3187a4040ec3a3ea492b8ad699a2b6d708d714cc063ac` Source locator `/tasks/weighted_pass__forms_filings/google~1gemini-3-flash-preview` | | | | | | | | | [Tax Agent Bench / weighted pass controversy precedence analysis](https://benchize.com/benchmarks/vals-ai%3Atax_agent_bench%3Aweighted_pass__controversy_precedence_analysis.md) | 30.14% | SE ±7.51 pp | 42 / 46 | $0.1 | 111.65source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 30.14 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.100435 \| USD \| per_test \| Lower is better \| \| `latency` \| 111.652 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±7.51 pp sourcePopulation 73 stderr 7.505 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) Source Vals AI Original model label google/gemini-3-flash-preview Source version 1 Observed 2026-10-21T16:45:55.958Z Evidence SHA-256 `446db182b67efb7923b3187a4040ec3a3ea492b8ad699a2b6d708d714cc063ac` Source locator `/tasks/weighted_pass__controversy_precedence_analysis/google~1gemini-3-flash-preview` | | | | | | | | | [EMB / misc](https://benchize.com/benchmarks/vals-ai%3Aemb%3Amisc.md) | 30.42% | SE ±7.94 pp | 43 / 47 | $0.38 | 257.18source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 30.42 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.378937 \| USD \| per_test \| Lower is better \| \| `latency` \| 257.181 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±7.94 pp sourcePopulation 76 stderr 7.939 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) Source Vals AI Original model label google/gemini-3-flash-preview Source version 1 Observed 2026-10-21T16:45:57.495Z Evidence SHA-256 `d807aee620ea47152a1fca0257a8dd12a1be4512c76cc1744c1ae9e285180918` Source locator `/tasks/misc/google~1gemini-3-flash-preview` | | | | | | | | | [MMLU Pro / math](https://benchize.com/benchmarks/vals-ai%3Ammlu_pro%3Amath.md) | 95.71% | SE ±0.55 pp | 14 / 40 | $0.02 | 39.1source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_mmlu_pro.y416pXDi.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 95.707 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.022316 \| USD \| per_test \| Lower is better \| \| `latency` \| 39.099 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.55 pp sourcePopulation 138 stderr 0.551 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_mmlu_pro.y416pXDi.json) Source Vals AI Original model label google/gemini-3-flash-preview Source version 1 Observed 2026-10-21T16:46:06.503Z Evidence SHA-256 `dab025507f46c716dd8d145e90ec0a737d9c56219853f41e4b1cc3c5e8c222b4` Source locator `/tasks/math/google~1gemini-3-flash-preview` | | | | | | | | | [LegalBench / overall](https://benchize.com/benchmarks/vals-ai%3Alegal_bench%3Aoverall.md) | 86.86% | SE ±0.36 pp | 9 / 44 | $0.00207 | 4.67source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_bench.D6uTgA3b.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 86.858 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.002067 \| USD \| per_test \| Lower is better \| \| `latency` \| 4.673 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.36 pp sourcePopulation 151 stderr 0.356 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_bench.D6uTgA3b.json) Source Vals AI Original model label google/gemini-3-flash-preview Source version 1 Observed 2026-10-21T16:45:53.390Z Evidence SHA-256 `c8486ba756baa5cd3955d035edff67440e8db23e69d62ad1a9e18d15b8c4bc52` Source locator `/tasks/overall/google~1gemini-3-flash-preview` | | | | | | | | | [TaxEval v2 / stepwise reasoning](https://benchize.com/benchmarks/vals-ai%3Atax_eval_v2%3Astepwise_reasoning.md) | 85.45% | SE ±1.01 pp | 16 / 40 | $0.00576 | 12source duration | [Vals AI · 2](https://www.vals.ai/_astro/benchmark_view_tax_eval_v2.B3yoffr8.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 85.446 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.005764 \| USD \| per_test \| Lower is better \| \| `latency` \| 12.001 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.01 pp sourcePopulation 145 stderr 1.008 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_tax_eval_v2.B3yoffr8.json) Source Vals AI Original model label google/gemini-3-flash-preview Source version 2 Observed 2026-10-21T16:45:59.000Z Evidence SHA-256 `8b354a6ae848f5bb25a2e8b061b5f576d5b9e0c11b941dc1e6bed078e6529cd6` Source locator `/tasks/stepwise_reasoning/google~1gemini-3-flash-preview` | | | | | | | | | [Tax Agent Bench / weighted pass fact pattern analysis](https://benchize.com/benchmarks/vals-ai%3Atax_agent_bench%3Aweighted_pass__fact_pattern_analysis.md) | 38.83% | SE ±6.64 pp | 37 / 46 | $0.09 | 110.17source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 38.825 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.093878 \| USD \| per_test \| Lower is better \| \| `latency` \| 110.173 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±6.64 pp sourcePopulation 73 stderr 6.637 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) Source Vals AI Original model label google/gemini-3-flash-preview Source version 1 Observed 2026-10-21T16:45:55.958Z Evidence SHA-256 `446db182b67efb7923b3187a4040ec3a3ea492b8ad699a2b6d708d714cc063ac` Source locator `/tasks/weighted_pass__fact_pattern_analysis/google~1gemini-3-flash-preview` | | | | | | | | | [MGSM / overall](https://benchize.com/benchmarks/vals-ai%3Amgsm%3Aoverall.md) | 93.31% | SE ±1.46 pp | 1 / 8 | $0.00712 | 14.02source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_mgsm.CneCP-rm.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 93.309 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.007116 \| USD \| per_test \| Lower is better \| \| `latency` \| 14.02 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.46 pp sourcePopulation 75 stderr 1.456 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_mgsm.CneCP-rm.json) Source Vals AI Original model label google/gemini-3-flash-preview Source version 1 Observed 2026-10-21T16:46:02.976Z Evidence SHA-256 `1b87f953e52c00bf4bdd90c3add26165f644eeb7a74a8382d2a0324ff0cc1040` Source locator `/tasks/overall/google~1gemini-3-flash-preview` | | | | | | | | [Next page](https://benchize.com/models/gemini-3-flash-preview-high.md?cursor=715f2cc5dad0e06ca1db84606129711db3b4a54d85102850c6621230e4a12cd7) --- Cite as: Gemini 3 Flash PreviewHigh — Benchize, https://benchize.com/models/gemini-3-flash-preview-high (retrieved 2026-10-21)