# Gemini 3.5 FlashHigh

> Source: Benchize (https://benchize.com) — https://benchize.com/models/gemini-3-5-flash-high. When citing or reusing this content, credit "Benchize" and link to https://benchize.com/models/gemini-3-5-flash-high.

Google

Canonical URL: https://benchize.com/models/gemini-3-5-flash-high
Last updated: Wed, 21 Oct 2026 18:05:10 GMT

Reproduction and quotation must name Benchize, link to the canonical HTML URL, and retain individual source citations.

Google

## Benchmarks

| Benchmark | Score | ± | Rank | Cost | Latency | Source |  |
| --- | --- | --- | --- | --- | --- | --- | --- |
| [ProgramBench / almost](https://benchize.com/benchmarks/vals-ai%3Aprogrambench%3Aalmost.md) | 1.5% | SE ±0.86 pp | 31 / 40 | — | 1,104.52source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_programbench.C_J7ZU4X.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 1.5 \| percent \| reported \| Higher is better \| \| `latency` \| 1104.517 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.86 pp sourcePopulation 64 stderr 0.862 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_programbench.C_J7ZU4X.json) Source Vals AI Original model label google/gemini-3.5-flash Source version 1 Observed 2026-10-21T16:46:10.137Z Evidence SHA-256 `e9bbb4378c7a058441a21d456baf9316b96c2cf5b58d371a3c7c803f5000bd8c` Source locator `/tasks/almost/google~1gemini-3.5-flash` |  |  |  |  |  |  |  |
| [Harvey's Legal Agent Benchmark v1 / criteria pass rate healthcare life sciences](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__healthcare_life_sciences.md) | 80.53% | SE ±2.28 pp | 33 / 49 | $2.41 | 575.66source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 80.528 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 2.41183 \| USD \| per_test \| Lower is better \| \| `latency` \| 575.66 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.28 pp sourcePopulation 81 stderr 2.279 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label google/gemini-3.5-flash Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__healthcare_life_sciences/google~1gemini-3.5-flash` |  |  |  |  |  |  |  |
| [Legal Research Bench / weighted pass business commercial](https://benchize.com/benchmarks/vals-ai%3Alegal_research%3Aweighted_pass__business_commercial.md) | 84.14% | SE ±2.92 pp | 29 / 50 | $1.23 | 317.65source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 84.143 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 1.232191 \| USD \| per_test \| Lower is better \| \| `latency` \| 317.652 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.92 pp sourcePopulation 82 stderr 2.921 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) Source Vals AI Original model label google/gemini-3.5-flash Source version 1 Observed 2026-10-21T16:45:52.389Z Evidence SHA-256 `26c58dfd47d0d2dc86f33c6d056f30309e0ee63570e7cd752c593b385f9d862e` Source locator `/tasks/weighted_pass__business_commercial/google~1gemini-3.5-flash` |  |  |  |  |  |  |  |
| [Arena text style control / overall](https://benchize.com/benchmarks/lmarena%3Atext_style_control%3Aoverall.md) | 1,477.47 | — | 26 / 151 | — | — | [Arena](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/text_style_control/latest) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `rating` \| 1477.4681010846125 \| rating \| reported \| Higher is better \| ### Configuration category overall config text_style_control Reasoning effort high thinkingEnabled true ### Reported statistics lower 1473.5790008726724 rank 29 upper 1481.3572012965528 variance 3.9373324469947044 votes 48479 ### Source [Original source](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/text_style_control/latest) Source Arena Original model label Gemini 3.5 Flash (High) Observed 2026-10-21T16:40:19.533Z Evidence SHA-256 `d4875c03cb7ec0c6e8943c3145fd261225e8841b937240dc62a3032c4a798d84` Source locator `/rows/28/row` |  |  |  |  |  |  |  |
| [Arena text / korean](https://benchize.com/benchmarks/lmarena%3Atext%3Akorean.md) | 1,445.15 | — | 22 / 124 | — | — | [Arena](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/text/latest) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `rating` \| 1445.1535510913502 \| rating \| reported \| Higher is better \| ### Configuration category korean config text Reasoning effort high thinkingEnabled true ### Reported statistics lower 1425.4547799143747 rank 25 upper 1464.852322268326 variance 101.01412093563167 votes 967 ### Source [Original source](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/text/latest) Source Arena Original model label Gemini 3.5 Flash (High) Observed 2026-10-21T16:40:19.533Z Evidence SHA-256 `8e027ef69b8de06c4b2a8b217bfe816b1f1536269bb257fd295b4ef1ba8246ac` Source locator `/rows/8258/row` |  |  |  |  |  |  |  |
| [Arena text / industry software and it services](https://benchize.com/benchmarks/lmarena%3Atext%3Aindustry_software_and_it_services.md) | 1,493.97 | — | 23 / 151 | — | — | [Arena](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/text/latest) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `rating` \| 1493.9672006156557 \| rating \| reported \| Higher is better \| ### Configuration category industry_software_and_it_services config text Reasoning effort high thinkingEnabled true ### Reported statistics lower 1488.6564497589957 rank 26 upper 1499.2779514723156 variance 7.342021866687058 votes 19409 ### Source [Original source](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/text/latest) Source Arena Original model label Gemini 3.5 Flash (High) Observed 2026-10-21T16:40:19.533Z Evidence SHA-256 `8e027ef69b8de06c4b2a8b217bfe816b1f1536269bb257fd295b4ef1ba8246ac` Source locator `/rows/6726/row` |  |  |  |  |  |  |  |
| [Terminal-Bench 4.0 / ml](https://benchize.com/benchmarks/vals-ai%3Aterminal-bench-4%3Aml.md) | 6.06% | SE ±3.03 pp | 29 / 36 | — | — | [Vals AI · 4.0](https://www.vals.ai/_astro/benchmark_view_terminal_bench_4.CAnGyzC_.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 6.061 \| percent \| reported \| Higher is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±3.03 pp sourcePopulation 52 stderr 3.03 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_terminal_bench_4.CAnGyzC_.json) Source Vals AI Original model label google/gemini-3.5-flash Source version 4.0 Observed 2026-10-21T16:46:10.642Z Evidence SHA-256 `98f6964580e901f1aa143db664bd92aa558415f643569492545d67c47d7f519b` Source locator `/tasks/ml/google~1gemini-3.5-flash` |  |  |  |  |  |  |  |
| [Terminal-Bench 4.0 / media](https://benchize.com/benchmarks/vals-ai%3Aterminal-bench-4%3Amedia.md) | 0% | SE ±0 pp | 24 / 36 | — | — | [Vals AI · 4.0](https://www.vals.ai/_astro/benchmark_view_terminal_bench_4.CAnGyzC_.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 52 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_terminal_bench_4.CAnGyzC_.json) Source Vals AI Original model label google/gemini-3.5-flash Source version 4.0 Observed 2026-10-21T16:46:10.642Z Evidence SHA-256 `98f6964580e901f1aa143db664bd92aa558415f643569492545d67c47d7f519b` Source locator `/tasks/media/google~1gemini-3.5-flash` |  |  |  |  |  |  |  |
| [SWE-bench / overall](https://benchize.com/benchmarks/vals-ai%3Aswebench%3Aoverall.md) | 78.8% | SE ±1.83 pp | 17 / 39 | $0.95 | 254.13source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_swebench.Ae8ni7h0.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 78.8 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.953882 \| USD \| per_test \| Lower is better \| \| `latency` \| 254.129 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.83 pp sourcePopulation 88 stderr 1.83 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_swebench.Ae8ni7h0.json) Source Vals AI Original model label google/gemini-3.5-flash Source version 1 Observed 2026-10-21T16:46:13.711Z Evidence SHA-256 `7a4e2e8485283caac500127419f68ec4bd5414c837ade114fc6afa5b2d34ccaa` Source locator `/tasks/overall/google~1gemini-3.5-flash` |  |  |  |  |  |  |  |
| [Arena webdev / image to webdev](https://benchize.com/benchmarks/lmarena%3Awebdev%3Aimage_to_webdev.md) | 1,501.19 | — | 30 / 42 | — | — | [Arena](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/webdev/latest) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `rating` \| 1501.1895278182099 \| rating \| reported \| Higher is better \| ### Configuration category image_to_webdev config webdev Reasoning effort high thinkingEnabled true ### Reported statistics lower 1487.4031418206291 rank 44 upper 1514.9759138157906 variance 49.47715119329349 votes 2024 ### Source [Original source](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/webdev/latest) Source Arena Original model label Gemini 3.5 Flash (High) Observed 2026-10-21T16:40:19.533Z Evidence SHA-256 `f82507859808110820a4ab6df15561b8864137ddefc1f6fb0c552af2e58ea738` Source locator `/rows/191/row` |  |  |  |  |  |  |  |
| [Legal Research Bench / weighted pass](https://benchize.com/benchmarks/vals-ai%3Alegal_research%3Aweighted_pass.md) | 83.93% | SE ±1.37 pp | 27 / 50 | $1.31 | 277.75source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 83.93 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 1.305281 \| USD \| per_test \| Lower is better \| \| `latency` \| 277.753 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.37 pp sourcePopulation 82 stderr 1.373 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) Source Vals AI Original model label google/gemini-3.5-flash Source version 1 Observed 2026-10-21T16:45:52.389Z Evidence SHA-256 `26c58dfd47d0d2dc86f33c6d056f30309e0ee63570e7cd752c593b385f9d862e` Source locator `/tasks/weighted_pass/google~1gemini-3.5-flash` |  |  |  |  |  |  |  |
| [Legal Research Bench / administrative regulatory](https://benchize.com/benchmarks/vals-ai%3Alegal_research%3Aadministrative_regulatory.md) | 43.94% | SE ±6.16 pp | 27 / 50 | $1.36 | 285.02source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 43.939 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 1.356228 \| USD \| per_test \| Lower is better \| \| `latency` \| 285.017 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±6.16 pp sourcePopulation 82 stderr 6.156 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) Source Vals AI Original model label google/gemini-3.5-flash Source version 1 Observed 2026-10-21T16:45:52.389Z Evidence SHA-256 `26c58dfd47d0d2dc86f33c6d056f30309e0ee63570e7cd752c593b385f9d862e` Source locator `/tasks/administrative_regulatory/google~1gemini-3.5-flash` |  |  |  |  |  |  |  |
| [Harvey's Legal Agent Benchmark v1 / overall](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Aoverall.md) | 2.5% | SE ±0.83 pp | 32 / 49 | $2.12 | 478.41source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 2.5 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 2.116049 \| USD \| per_test \| Lower is better \| \| `latency` \| 478.414 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.83 pp sourcePopulation 81 stderr 0.833 tokenTotals {"cache_read_tokens":543998500,"cache_write_tokens":null,"input_tokens":606812300,"output_tokens":8678382,"reasoning_tokens":3806922} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label google/gemini-3.5-flash Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/overall/google~1gemini-3.5-flash` |  |  |  |  |  |  |  |
| [Vibe Code Bench v1.1 / overall](https://benchize.com/benchmarks/vals-ai%3Avibe-code%3Aoverall.md) | 48.68% | SE ±4.73 pp | 39 / 59 | $2.54 | 893.07source duration | [Vals AI · 1.1](https://www.vals.ai/_astro/benchmark_view_vibe_code_bench.DcEBMbCA.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 48.683 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 2.540385 \| USD \| per_test \| Lower is better \| \| `latency` \| 893.074 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness OpenHands maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±4.73 pp sourcePopulation 116 stderr 4.727 tokenTotals {"cache_read_tokens":null,"cache_write_tokens":null,"input_tokens":64968703,"output_tokens":3285132,"reasoning_tokens":null} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_vibe_code_bench.DcEBMbCA.json) Source Vals AI Original model label google/gemini-3.5-flash Source version 1.1 Observed 2026-10-21T16:46:08.625Z Evidence SHA-256 `0e8c23f521a0f751c5592952620e197b376cade41c78578461b7aa5740a173e7` Source locator `/tasks/overall/google~1gemini-3.5-flash` |  |  |  |  |  |  |  |
| [ARC-AGI-1](https://benchize.com/benchmarks/arc-prize%3Av1_Semi_Private.md) | 92.5% | — | 47 / 213 | 0.43usd per task | — | [arc prize · v1](https://arcprize.org/media/data/leaderboard/v1.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 0.925 \| score_fraction \| v1_Semi_Private \| Higher is better \| \| `cost_per_task` \| 0.4278 \| usd_per_task \| v1_Semi_Private \| Lower is better \| ### Configuration evaluationSplit v1_Semi_Private harness Not reported modelGroup gemini-3-5-flash modelType CoT Reasoning effort high sourceModelVersion gemini-3-5-flash-high thinkingEnabled true ### Source [Original source](https://arcprize.org/media/data/leaderboard/v1.json) Source arc prize Original model label Gemini 3.5 Flash (High) Source version v1 Observed 2026-10-21T16:34:20.378Z Evidence SHA-256 `302d8267c0522bf450ec1deeac4d060e2cf2fb444ba0a4269014d9535de0bd1f` Source locator `/evaluations/153` |  |  |  |  |  |  |  |
| [CorpFin v2 / max fitting context](https://benchize.com/benchmarks/vals-ai%3Acorp_fin_v2%3Amax_fitting_context.md) | 66.67% | SE ±1.61 pp | 12 / 39 | $0.3 | 12.36source duration | [Vals AI · 2](https://www.vals.ai/_astro/benchmark_view_corp_fin_v2.CectKwl1.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 66.667 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.29559 \| USD \| per_test \| Lower is better \| \| `latency` \| 12.357 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.61 pp sourcePopulation 134 stderr 1.609 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_corp_fin_v2.CectKwl1.json) Source Vals AI Original model label google/gemini-3.5-flash Source version 2 Observed 2026-10-21T16:45:57.998Z Evidence SHA-256 `7cf2217ab3728d5a84bffce82aab9fc52d7f810b0cbe8c65b387df47f3d31c9b` Source locator `/tasks/max_fitting_context/google~1gemini-3.5-flash` |  |  |  |  |  |  |  |
| [Tax Agent Bench / weighted pass rule source lookup](https://benchize.com/benchmarks/vals-ai%3Atax_agent_bench%3Aweighted_pass__rule_source_lookup.md) | 46.79% | SE ±8.74 pp | 33 / 46 | $0.71 | 227.91source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 46.785 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.71482 \| USD \| per_test \| Lower is better \| \| `latency` \| 227.911 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±8.74 pp sourcePopulation 73 stderr 8.739 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) Source Vals AI Original model label google/gemini-3.5-flash Source version 1 Observed 2026-10-21T16:45:55.958Z Evidence SHA-256 `446db182b67efb7923b3187a4040ec3a3ea492b8ad699a2b6d708d714cc063ac` Source locator `/tasks/weighted_pass__rule_source_lookup/google~1gemini-3.5-flash` |  |  |  |  |  |  |  |
| [GPQA Diamond / diamond zero shot cot](https://benchize.com/benchmarks/vals-ai%3Agpqa%3Adiamond_zero_shot_cot.md) | 93.94% | SE ±2.14 pp | 5 / 40 | $0.06 | 28.56source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_gpqa.ghZZBpyU.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 93.939 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.061459 \| USD \| per_test \| Lower is better \| \| `latency` \| 28.557 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.14 pp sourcePopulation 138 stderr 2.142 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_gpqa.ghZZBpyU.json) Source Vals AI Original model label google/gemini-3.5-flash Source version 1 Observed 2026-10-21T16:46:06.040Z Evidence SHA-256 `951eb5224097733a9d09d06c14908ae2980e2e640e539f2c284877c5092d7077` Source locator `/tasks/diamond_zero_shot_cot/google~1gemini-3.5-flash` |  |  |  |  |  |  |  |
| [Finance Agent (v2) / all pass](https://benchize.com/benchmarks/vals-ai%3Afabv2%3Aall_pass.md) | 45.71% | SE ±0.2 pp | 13 / 50 | $2.51 | 322.28source duration | [Vals AI · 2](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 45.706 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 2.505488 \| USD \| per_test \| Lower is better \| \| `latency` \| 322.279 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.2 pp sourcePopulation 80 stderr 0.196 tokenTotals {"cache_read_tokens":707613227,"cache_write_tokens":null,"input_tokens":1227406624,"output_tokens":26782944,"reasoning_tokens":20834822} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) Source Vals AI Original model label google/gemini-3.5-flash Source version 2 Observed 2026-10-21T16:45:57.035Z Evidence SHA-256 `f731b20a26c126725d125519f2b5841aa45254656dba859a9570d7ce7e5d8191` Source locator `/tasks/all_pass/google~1gemini-3.5-flash` |  |  |  |  |  |  |  |
| [Harvey's Legal Agent Benchmark v1 / criteria pass rate litigation dispute resolution](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__litigation_dispute_resolution.md) | 80.52% | SE ±2.43 pp | 33 / 49 | $2.38 | 522.01source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 80.524 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 2.378166 \| USD \| per_test \| Lower is better \| \| `latency` \| 522.006 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.43 pp sourcePopulation 81 stderr 2.428 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label google/gemini-3.5-flash Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__litigation_dispute_resolution/google~1gemini-3.5-flash` |  |  |  |  |  |  |  |
| [Legal Research Bench / immigration](https://benchize.com/benchmarks/vals-ai%3Alegal_research%3Aimmigration.md) | 18.18% | SE ±12.2 pp | 32 / 50 | $0.88 | 219.97source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 18.182 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.884226 \| USD \| per_test \| Lower is better \| \| `latency` \| 219.971 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±12.2 pp sourcePopulation 82 stderr 12.197 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) Source Vals AI Original model label google/gemini-3.5-flash Source version 1 Observed 2026-10-21T16:45:52.389Z Evidence SHA-256 `26c58dfd47d0d2dc86f33c6d056f30309e0ee63570e7cd752c593b385f9d862e` Source locator `/tasks/immigration/google~1gemini-3.5-flash` |  |  |  |  |  |  |  |
| [Arena vision / ocr (config: vision)](https://benchize.com/benchmarks/benchmark%3Avision%3Aocr--config-vision.md) | 1,312.18 | — | 15 / 74 | — | — | [Arena](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/vision/latest) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `rating` \| 1312.182870733333 \| rating \| reported \| Higher is better \| ### Configuration category ocr config vision Reasoning effort high thinkingEnabled true ### Reported statistics lower 1305.1002356220777 rank 15 upper 1319.265505844588 variance 13.058507837946863 votes 9583 ### Source [Original source](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/vision/latest) Source Arena Original model label Gemini 3.5 Flash (High) Observed 2026-10-21T16:40:19.533Z Evidence SHA-256 `06e61305486dce3fec5dc776ac2fd8321104e84c346cf9e08fa6f8e8a66ce0d7` Source locator `/rows/1033/row` |  |  |  |  |  |  |  |
| [Harvey's Legal Agent Benchmark v1 / energy natural resources](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Aenergy_natural_resources.md) | 0% | SE ±0 pp | 28 / 49 | $2.01 | 458.45source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 2.006452 \| USD \| per_test \| Lower is better \| \| `latency` \| 458.446 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 81 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label google/gemini-3.5-flash Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/energy_natural_resources/google~1gemini-3.5-flash` |  |  |  |  |  |  |  |
| [Arena text / overall](https://benchize.com/benchmarks/lmarena%3Atext.md) | 1,481.98 | — | 18 / 151 | — | — | [Arena](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/text/latest) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `rating` \| 1481.983653833525 \| rating \| reported \| Higher is better \| ### Configuration category overall config text Reasoning effort high thinkingEnabled true ### Reported statistics lower 1478.1173211730365 rank 18 upper 1485.8499864940134 variance 3.8913675609460574 votes 48479 ### Source [Original source](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/text/latest) Source Arena Original model label Gemini 3.5 Flash (High) Observed 2026-10-21T16:40:19.533Z Evidence SHA-256 `8e027ef69b8de06c4b2a8b217bfe816b1f1536269bb257fd295b4ef1ba8246ac` Source locator `/rows/17/row` |  |  |  |  |  |  |  |
| [SkillsBench / with skills](https://benchize.com/benchmarks/vals-ai%3Askillsbench%3Awith_skills.md) | 52.74% | SE ±4.48 pp | 14 / 20 | $1.25 | 435.01source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_skillsbench.BE2TXB-M.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 52.737 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 1.253121 \| USD \| per_test \| Lower is better \| \| `latency` \| 435.011 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness OpenHands maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±4.48 pp sourcePopulation 35 stderr 4.475 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_skillsbench.BE2TXB-M.json) Source Vals AI Original model label google/gemini-3.5-flash Source version 1 Observed 2026-10-21T16:46:11.147Z Evidence SHA-256 `aaad242949d81b7c7c61c36d4895d4460e9146760c1e6b4b38d235a16043bc61` Source locator `/tasks/with_skills/google~1gemini-3.5-flash` |  |  |  |  |  |  |  |
| [MortgageTax / overall](https://benchize.com/benchmarks/vals-ai%3Amortgage_tax%3Aoverall.md) | 68.12% | SE ±0.91 pp | 13 / 31 | $0.02 | 11.43source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_mortgage_tax.DCd-oO5Y.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 68.124 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.017873 \| USD \| per_test \| Lower is better \| \| `latency` \| 11.428 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.91 pp sourcePopulation 98 stderr 0.91 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_mortgage_tax.DCd-oO5Y.json) Source Vals AI Original model label google/gemini-3.5-flash Source version 1 Observed 2026-10-21T16:45:58.500Z Evidence SHA-256 `e5c844561f97d119d66d70269e726ecfb68be79eae39acf14ead3e95422e648c` Source locator `/tasks/overall/google~1gemini-3.5-flash` |  |  |  |  |  |  |  |
| [MMLU Pro / computer science](https://benchize.com/benchmarks/vals-ai%3Ammlu_pro%3Acomputer_science.md) | 91.71% | SE ±1.36 pp | 9 / 40 | $0.02 | 10.9source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_mmlu_pro.y416pXDi.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 91.707 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.022972 \| USD \| per_test \| Lower is better \| \| `latency` \| 10.896 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.36 pp sourcePopulation 138 stderr 1.362 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_mmlu_pro.y416pXDi.json) Source Vals AI Original model label google/gemini-3.5-flash Source version 1 Observed 2026-10-21T16:46:06.503Z Evidence SHA-256 `dab025507f46c716dd8d145e90ec0a737d9c56219853f41e4b1cc3c5e8c222b4` Source locator `/tasks/computer_science/google~1gemini-3.5-flash` |  |  |  |  |  |  |  |
| [MMLU Pro / biology](https://benchize.com/benchmarks/vals-ai%3Ammlu_pro%3Abiology.md) | 93.86% | SE ±0.9 pp | 13 / 40 | $0.02 | 12.33source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_mmlu_pro.y416pXDi.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 93.863 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.015872 \| USD \| per_test \| Lower is better \| \| `latency` \| 12.325 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.9 pp sourcePopulation 138 stderr 0.896 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_mmlu_pro.y416pXDi.json) Source Vals AI Original model label google/gemini-3.5-flash Source version 1 Observed 2026-10-21T16:46:06.503Z Evidence SHA-256 `dab025507f46c716dd8d145e90ec0a737d9c56219853f41e4b1cc3c5e8c222b4` Source locator `/tasks/biology/google~1gemini-3.5-flash` |  |  |  |  |  |  |  |
| [Code Migration / cli](https://benchize.com/benchmarks/vals-ai%3Acode-migration%3Acli.md) | 17.18% | SE ±1.82 pp | 32 / 47 | $6.35 | 1,277.58source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_code_migration.Dttzoqk3.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 17.175 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 6.345007 \| USD \| per_test \| Lower is better \| \| `latency` \| 1277.58 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.82 pp sourcePopulation 79 stderr 1.823 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_code_migration.Dttzoqk3.json) Source Vals AI Original model label google/gemini-3.5-flash Source version 1 Observed 2026-10-21T16:46:12.712Z Evidence SHA-256 `99194597f2c76c402195be47839409046e18bd30454616da6dd40b736a088fa4` Source locator `/tasks/cli/google~1gemini-3.5-flash` |  |  |  |  |  |  |  |
| [Tax Agent Bench / weighted pass current temporal analysis](https://benchize.com/benchmarks/vals-ai%3Atax_agent_bench%3Aweighted_pass__current_temporal_analysis.md) | 61.65% | SE ±8.24 pp | 24 / 46 | $0.93 | 279.9source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 61.648 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.929565 \| USD \| per_test \| Lower is better \| \| `latency` \| 279.898 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±8.24 pp sourcePopulation 73 stderr 8.243 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) Source Vals AI Original model label google/gemini-3.5-flash Source version 1 Observed 2026-10-21T16:45:55.958Z Evidence SHA-256 `446db182b67efb7923b3187a4040ec3a3ea492b8ad699a2b6d708d714cc063ac` Source locator `/tasks/weighted_pass__current_temporal_analysis/google~1gemini-3.5-flash` |  |  |  |  |  |  |  |
| [EMB / overall](https://benchize.com/benchmarks/vals-ai%3Aemb%3Aoverall.md) | 63.55% | SE ±2.75 pp | 23 / 47 | $5.17 | 1,123.56source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 63.548 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 5.173115 \| USD \| per_test \| Lower is better \| \| `latency` \| 1123.555 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.75 pp sourcePopulation 76 stderr 2.752 tokenTotals {"cache_read_tokens":1520002205,"cache_write_tokens":null,"input_tokens":1617865425,"output_tokens":16984726,"reasoning_tokens":10956816} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) Source Vals AI Original model label google/gemini-3.5-flash Source version 1 Observed 2026-10-21T16:45:57.495Z Evidence SHA-256 `d807aee620ea47152a1fca0257a8dd12a1be4512c76cc1744c1ae9e285180918` Source locator `/tasks/overall/google~1gemini-3.5-flash` |  |  |  |  |  |  |  |
| [Terminal-Bench 4.0 / overall](https://benchize.com/benchmarks/vals-ai%3Aterminal-bench-4%3Aoverall.md) | 6.06% | SE ±1.52 pp | 31 / 36 | $5.98 | 4,286.82source duration | [Vals AI · 4.0](https://www.vals.ai/_astro/benchmark_view_terminal_bench_4.CAnGyzC_.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 6.061 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 5.981757 \| USD \| per_test \| Lower is better \| \| `latency` \| 4286.824 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.52 pp sourcePopulation 52 stderr 1.515 tokenTotals {"cache_read_tokens":1127424034,"cache_write_tokens":0,"input_tokens":1224247223,"output_tokens":8938622,"reasoning_tokens":5695714} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_terminal_bench_4.CAnGyzC_.json) Source Vals AI Original model label google/gemini-3.5-flash Source version 4.0 Observed 2026-10-21T16:46:10.642Z Evidence SHA-256 `98f6964580e901f1aa143db664bd92aa558415f643569492545d67c47d7f519b` Source locator `/tasks/overall/google~1gemini-3.5-flash` |  |  |  |  |  |  |  |
| [LegalBench / rule tasks](https://benchize.com/benchmarks/vals-ai%3Alegal_bench%3Arule_tasks.md) | 88% | SE ±1.73 pp | 5 / 44 | $0.01 | 35.43source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_bench.D6uTgA3b.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 87.996 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.01073 \| USD \| per_test \| Lower is better \| \| `latency` \| 35.426 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.73 pp sourcePopulation 151 stderr 1.731 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_bench.D6uTgA3b.json) Source Vals AI Original model label google/gemini-3.5-flash Source version 1 Observed 2026-10-21T16:45:53.390Z Evidence SHA-256 `c8486ba756baa5cd3955d035edff67440e8db23e69d62ad1a9e18d15b8c4bc52` Source locator `/tasks/rule_tasks/google~1gemini-3.5-flash` |  |  |  |  |  |  |  |
| [Tax Agent Bench / numbers calculations](https://benchize.com/benchmarks/vals-ai%3Atax_agent_bench%3Anumbers_calculations.md) | 46.67% | SE ±9.26 pp | 12 / 46 | $0.86 | 234.86source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 46.667 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.855352 \| USD \| per_test \| Lower is better \| \| `latency` \| 234.86 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±9.26 pp sourcePopulation 73 stderr 9.264 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) Source Vals AI Original model label google/gemini-3.5-flash Source version 1 Observed 2026-10-21T16:45:55.958Z Evidence SHA-256 `446db182b67efb7923b3187a4040ec3a3ea492b8ad699a2b6d708d714cc063ac` Source locator `/tasks/numbers_calculations/google~1gemini-3.5-flash` |  |  |  |  |  |  |  |
| [Tax Agent Bench / weighted pass forms filings](https://benchize.com/benchmarks/vals-ai%3Atax_agent_bench%3Aweighted_pass__forms_filings.md) | 45.95% | SE ±8.63 pp | 36 / 46 | $0.99 | 292.44source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 45.945 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.990018 \| USD \| per_test \| Lower is better \| \| `latency` \| 292.436 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±8.63 pp sourcePopulation 73 stderr 8.634 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) Source Vals AI Original model label google/gemini-3.5-flash Source version 1 Observed 2026-10-21T16:45:55.958Z Evidence SHA-256 `446db182b67efb7923b3187a4040ec3a3ea492b8ad699a2b6d708d714cc063ac` Source locator `/tasks/weighted_pass__forms_filings/google~1gemini-3.5-flash` |  |  |  |  |  |  |  |
| [Arena text style control / korean](https://benchize.com/benchmarks/lmarena%3Atext_style_control%3Akorean.md) | 1,435.3 | — | 27 / 124 | — | — | [Arena](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/text_style_control/latest) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `rating` \| 1435.2978999062852 \| rating \| reported \| Higher is better \| ### Configuration category korean config text_style_control Reasoning effort high thinkingEnabled true ### Reported statistics lower 1415.5297193554552 rank 36 upper 1455.0660804571153 variance 101.72722929764655 votes 967 ### Source [Original source](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/text_style_control/latest) Source Arena Original model label Gemini 3.5 Flash (High) Observed 2026-10-21T16:40:19.533Z Evidence SHA-256 `d4875c03cb7ec0c6e8943c3145fd261225e8841b937240dc62a3032c4a798d84` Source locator `/rows/8269/row` |  |  |  |  |  |  |  |
| [LiveCodeBench / medium](https://benchize.com/benchmarks/vals-ai%3Alcb%3Amedium.md) | 90.6% | SE ±1.49 pp | 5 / 43 | $0.06 | 35.47source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_lcb.miyRUvVa.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 90.601 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.063422 \| USD \| per_test \| Lower is better \| \| `latency` \| 35.468 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.49 pp sourcePopulation 143 stderr 1.491 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_lcb.miyRUvVa.json) Source Vals AI Original model label google/gemini-3.5-flash Source version 1 Observed 2026-10-21T16:46:14.220Z Evidence SHA-256 `1a7c7dabc9d95a0182886b1edc42623987cac2095a0d407a5e334f37acdfa2f5` Source locator `/tasks/medium/google~1gemini-3.5-flash` |  |  |  |  |  |  |  |
| [Public Benefits Bench v1 / neither](https://benchize.com/benchmarks/vals-ai%3Apublic-benefits-bench-v1%3Aneither.md) | 50.27% | SE ±1.3 pp | 3 / 8 | $0.06 | 54.33source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_snap_v1.BPa0Hrre.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 50.271 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.062674 \| USD \| per_test \| Lower is better \| \| `latency` \| 54.325 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.3 pp sourcePopulation 13 stderr 1.301 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_snap_v1.BPa0Hrre.json) Source Vals AI Original model label google/gemini-3.5-flash Source version 1 Observed 2026-10-21T16:46:18.363Z Evidence SHA-256 `b85e2b3cbdf03e13e2ffdb27ce4f108003a548d705a24ae26a0a115e2027475d` Source locator `/tasks/neither/google~1gemini-3.5-flash` |  |  |  |  |  |  |  |
| [Arena text style control / coding](https://benchize.com/benchmarks/lmarena%3Atext_style_control%3Acoding.md) | 1,508.09 | — | 41 / 151 | — | — | [Arena](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/text_style_control/latest) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `rating` \| 1508.0890555596661 \| rating \| reported \| Higher is better \| ### Configuration category coding config text_style_control Reasoning effort high thinkingEnabled true ### Reported statistics lower 1501.99911411351 rank 63 upper 1514.178997005822 variance 9.654505891829032 votes 13845 ### Source [Original source](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/text_style_control/latest) Source Arena Original model label Gemini 3.5 Flash (High) Observed 2026-10-21T16:40:19.533Z Evidence SHA-256 `d4875c03cb7ec0c6e8943c3145fd261225e8841b937240dc62a3032c4a798d84` Source locator `/rows/872/row` |  |  |  |  |  |  |  |
| [Tax Agent Bench / fact pattern analysis](https://benchize.com/benchmarks/vals-ai%3Atax_agent_bench%3Afact_pattern_analysis.md) | 16.28% | SE ±5.7 pp | 34 / 46 | $0.75 | 237.18source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 16.279 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.753889 \| USD \| per_test \| Lower is better \| \| `latency` \| 237.175 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±5.7 pp sourcePopulation 73 stderr 5.696 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) Source Vals AI Original model label google/gemini-3.5-flash Source version 1 Observed 2026-10-21T16:45:55.958Z Evidence SHA-256 `446db182b67efb7923b3187a4040ec3a3ea492b8ad699a2b6d708d714cc063ac` Source locator `/tasks/fact_pattern_analysis/google~1gemini-3.5-flash` |  |  |  |  |  |  |  |
| [LegalBench / issue tasks](https://benchize.com/benchmarks/vals-ai%3Alegal_bench%3Aissue_tasks.md) | 76.01% | SE ±3.49 pp | 41 / 44 | $0.00786 | 50.77source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_bench.D6uTgA3b.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 76.014 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.007857 \| USD \| per_test \| Lower is better \| \| `latency` \| 50.772 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±3.49 pp sourcePopulation 151 stderr 3.493 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_bench.D6uTgA3b.json) Source Vals AI Original model label google/gemini-3.5-flash Source version 1 Observed 2026-10-21T16:45:53.390Z Evidence SHA-256 `c8486ba756baa5cd3955d035edff67440e8db23e69d62ad1a9e18d15b8c4bc52` Source locator `/tasks/issue_tasks/google~1gemini-3.5-flash` |  |  |  |  |  |  |  |
| [Mystery Game Puzzles](https://benchize.com/benchmarks/epoch-ai%3Amystery_game_puzzles.md) | 32% | SE ±4.69 pp | 24 / 71 | — | — | [epoch ai · Mystery Game Puzzles](https://epoch.ai/data/benchmark_data.zip#file=mystery_game_puzzles.csv) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 0.32 \| score_fraction \| Best score (across scorers) \| Higher is better \| \| `mean_score` \| 0.32 \| score_fraction \| mean_score \| Higher is better \| ### Configuration benchmarkVersion Mystery Game Puzzles reasoningBudget Not reported Reasoning effort high sourceModelVersion gemini-3.5-flash_high ### Reported statistics Uncertainty Standard error: ±4.69 pp standardError 0.04688261722621506 ### Source [Original source](https://epoch.ai/data/benchmark_data.zip#file=mystery_game_puzzles.csv) Source epoch ai Original model label gemini-3.5-flash Source version Mystery Game Puzzles Evaluated 2026-07-27T18:28:52.000Z Observed 2026-10-21T16:30:24.199Z Evidence SHA-256 `795b977bc82c460a5e2c1d6dd7b2569ddb21853f3829c4f6fdd16edfaf47bfcf` Source locator `/rows/128` |  |  |  |  |  |  |  |
| [Vals Multimodal Index / sage](https://benchize.com/benchmarks/vals-ai%3Avals_multimodal_index%3Asage.md) | 49.89% | SE ±3.44 pp | 8 / 20 | $0.05 | 32.59source duration | [Vals AI · 1.2](https://www.vals.ai/_astro/benchmark_view_vals_multimodal_index.virVpWMs.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 49.885 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.052555 \| USD \| per_test \| Lower is better \| \| `latency` \| 32.592 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±3.44 pp sourcePopulation 33 stderr 3.438 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_vals_multimodal_index.virVpWMs.json) Source Vals AI Original model label google/gemini-3.5-flash Source version 1.2 Observed 2026-10-21T16:45:51.881Z Evidence SHA-256 `077b22575f6e075b7b78157d000ec8f94d7ead409c7dcbcf04887630ed572a2d` Source locator `/tasks/sage/google~1gemini-3.5-flash` |  |  |  |  |  |  |  |
| [Arena vision style control / ocr](https://benchize.com/benchmarks/lmarena%3Avision_style_control%3Aocr.md) | 1,292.36 | — | 28 / 74 | — | — | [Arena](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/vision_style_control/latest) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `rating` \| 1292.3606194925915 \| rating \| reported \| Higher is better \| ### Configuration category ocr config vision_style_control Reasoning effort high thinkingEnabled true ### Reported statistics lower 1285.2411285694295 rank 29 upper 1299.4801104157532 variance 13.194766199739103 votes 9583 ### Source [Original source](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/vision_style_control/latest) Source Arena Original model label Gemini 3.5 Flash (High) Observed 2026-10-21T16:40:19.533Z Evidence SHA-256 `38568936995f36625aae49e9ea05557d55c1b1df9b2954f5c50cee523f26fe92` Source locator `/rows/1047/row` |  |  |  |  |  |  |  |
| [Arena text / math](https://benchize.com/benchmarks/lmarena%3Atext%3Amath.md) | 1,504.21 | — | 12 / 149 | — | — | [Arena](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/text/latest) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `rating` \| 1504.2087742077758 \| rating \| reported \| Higher is better \| ### Configuration category math config text Reasoning effort high thinkingEnabled true ### Reported statistics lower 1491.8153916436527 rank 12 upper 1516.602156771899 variance 39.9837505880013 votes 2388 ### Source [Original source](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/text/latest) Source Arena Original model label Gemini 3.5 Flash (High) Observed 2026-10-21T16:40:19.533Z Evidence SHA-256 `8e027ef69b8de06c4b2a8b217bfe816b1f1536269bb257fd295b4ef1ba8246ac` Source locator `/rows/8924/row` |  |  |  |  |  |  |  |
| [Arena vision / chinese (config: vision)](https://benchize.com/benchmarks/benchmark%3Avision%3Achinese--config-vision.md) | 1,377.07 | — | 6 / 74 | — | — | [Arena](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/vision/latest) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `rating` \| 1377.065533142831 \| rating \| reported \| Higher is better \| ### Configuration category chinese config vision Reasoning effort high thinkingEnabled true ### Reported statistics lower 1354.8201684976225 rank 6 upper 1399.3108977880397 variance 128.81987580668113 votes 847 ### Source [Original source](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/vision/latest) Source Arena Original model label Gemini 3.5 Flash (High) Observed 2026-10-21T16:40:19.533Z Evidence SHA-256 `06e61305486dce3fec5dc776ac2fd8321104e84c346cf9e08fa6f8e8a66ce0d7` Source locator `/rows/206/row` |  |  |  |  |  |  |  |
| [Harvey's Legal Agent Benchmark v1 / criteria pass rate emerging companies venture capital](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__emerging_companies_venture_capital.md) | 78.5% | SE ±2.4 pp | 36 / 49 | $2.12 | 481.85source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 78.498 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 2.122257 \| USD \| per_test \| Lower is better \| \| `latency` \| 481.848 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.4 pp sourcePopulation 81 stderr 2.404 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label google/gemini-3.5-flash Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__emerging_companies_venture_capital/google~1gemini-3.5-flash` |  |  |  |  |  |  |  |
| [Arena text / industry life and physical and social science](https://benchize.com/benchmarks/lmarena%3Atext%3Aindustry_life_and_physical_and_social_science.md) | 1,494.15 | — | 20 / 151 | — | — | [Arena](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/text/latest) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `rating` \| 1494.1491178152087 \| rating \| reported \| Higher is better \| ### Configuration category industry_life_and_physical_and_social_science config text Reasoning effort high thinkingEnabled true ### Reported statistics lower 1486.9519984398703 rank 23 upper 1501.3462371905468 variance 13.48407720104356 votes 8006 ### Source [Original source](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/text/latest) Source Arena Original model label Gemini 3.5 Flash (High) Observed 2026-10-21T16:40:19.533Z Evidence SHA-256 `8e027ef69b8de06c4b2a8b217bfe816b1f1536269bb257fd295b4ef1ba8246ac` Source locator `/rows/5520/row` |  |  |  |  |  |  |  |
| [LegalBench / interpretation tasks](https://benchize.com/benchmarks/vals-ai%3Alegal_bench%3Ainterpretation_tasks.md) | 88.68% | SE ±0.29 pp | 1 / 44 | $0.0062 | 30.37source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_bench.D6uTgA3b.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 88.675 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.006199 \| USD \| per_test \| Lower is better \| \| `latency` \| 30.374 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.29 pp sourcePopulation 151 stderr 0.289 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_bench.D6uTgA3b.json) Source Vals AI Original model label google/gemini-3.5-flash Source version 1 Observed 2026-10-21T16:45:53.390Z Evidence SHA-256 `c8486ba756baa5cd3955d035edff67440e8db23e69d62ad1a9e18d15b8c4bc52` Source locator `/tasks/interpretation_tasks/google~1gemini-3.5-flash` |  |  |  |  |  |  |  |
| [Legal Research Bench / weighted pass health](https://benchize.com/benchmarks/vals-ai%3Alegal_research%3Aweighted_pass__health.md) | 91.47% | SE ±1.98 pp | 24 / 50 | $1.4 | 274.58source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 91.474 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 1.402227 \| USD \| per_test \| Lower is better \| \| `latency` \| 274.578 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.98 pp sourcePopulation 82 stderr 1.976 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) Source Vals AI Original model label google/gemini-3.5-flash Source version 1 Observed 2026-10-21T16:45:52.389Z Evidence SHA-256 `26c58dfd47d0d2dc86f33c6d056f30309e0ee63570e7cd752c593b385f9d862e` Source locator `/tasks/weighted_pass__health/google~1gemini-3.5-flash` |  |  |  |  |  |  |  |
| [Public Benefits Bench v1 / both](https://benchize.com/benchmarks/vals-ai%3Apublic-benefits-bench-v1%3Aboth.md) | 57.98% | SE ±1.28 pp | 5 / 8 | $0.78 | 273.99source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_snap_v1.BPa0Hrre.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 57.984 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.775836 \| USD \| per_test \| Lower is better \| \| `latency` \| 273.985 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.28 pp sourcePopulation 13 stderr 1.284 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_snap_v1.BPa0Hrre.json) Source Vals AI Original model label google/gemini-3.5-flash Source version 1 Observed 2026-10-21T16:46:18.363Z Evidence SHA-256 `b85e2b3cbdf03e13e2ffdb27ce4f108003a548d705a24ae26a0a115e2027475d` Source locator `/tasks/both/google~1gemini-3.5-flash` |  |  |  |  |  |  |  |
| [Finance Agent (v2) / comparables](https://benchize.com/benchmarks/vals-ai%3Afabv2%3Acomparables.md) | 48.78% | SE ±0.31 pp | 6 / 50 | $3.12 | 380.72source duration | [Vals AI · 2](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 48.782 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 3.1206 \| USD \| per_test \| Lower is better \| \| `latency` \| 380.721 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.31 pp sourcePopulation 80 stderr 0.311 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) Source Vals AI Original model label google/gemini-3.5-flash Source version 2 Observed 2026-10-21T16:45:57.035Z Evidence SHA-256 `f731b20a26c126725d125519f2b5841aa45254656dba859a9570d7ce7e5d8191` Source locator `/tasks/comparables/google~1gemini-3.5-flash` |  |  |  |  |  |  |  |
| [Arena text style control / industry mathematical](https://benchize.com/benchmarks/lmarena%3Atext_style_control%3Aindustry_mathematical.md) | 1,485.46 | — | 37 / 148 | — | — | [Arena](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/text_style_control/latest) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `rating` \| 1485.463177952919 \| rating \| reported \| Higher is better \| ### Configuration category industry_mathematical config text_style_control Reasoning effort high thinkingEnabled true ### Reported statistics lower 1473.8478571238084 rank 45 upper 1497.0784987820293 variance 35.12094864491818 votes 2880 ### Source [Original source](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/text_style_control/latest) Source Arena Original model label Gemini 3.5 Flash (High) Observed 2026-10-21T16:40:19.533Z Evidence SHA-256 `d4875c03cb7ec0c6e8943c3145fd261225e8841b937240dc62a3032c4a798d84` Source locator `/rows/5957/row` |  |  |  |  |  |  |  |
| [MMLU Pro / physics](https://benchize.com/benchmarks/vals-ai%3Ammlu_pro%3Aphysics.md) | 94.07% | SE ±0.66 pp | 7 / 40 | $0.03 | 58.13source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_mmlu_pro.y416pXDi.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 94.072 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.029412 \| USD \| per_test \| Lower is better \| \| `latency` \| 58.125 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.66 pp sourcePopulation 138 stderr 0.663 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_mmlu_pro.y416pXDi.json) Source Vals AI Original model label google/gemini-3.5-flash Source version 1 Observed 2026-10-21T16:46:06.503Z Evidence SHA-256 `dab025507f46c716dd8d145e90ec0a737d9c56219853f41e4b1cc3c5e8c222b4` Source locator `/tasks/physics/google~1gemini-3.5-flash` |  |  |  |  |  |  |  |
| [ARC-AGI-2](https://benchize.com/benchmarks/arc-prize%3Av2_Semi_Private.md) | 72.08% | — | 48 / 212 | 0.85usd per task | — | [arc prize · v2](https://arcprize.org/media/data/leaderboard/v2.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 0.7208 \| score_fraction \| v2_Semi_Private \| Higher is better \| \| `cost_per_task` \| 0.8503 \| usd_per_task \| v2_Semi_Private \| Lower is better \| ### Configuration evaluationSplit v2_Semi_Private harness Not reported modelGroup gemini-3-5-flash modelType CoT Reasoning effort high sourceModelVersion gemini-3-5-flash-high thinkingEnabled true ### Source [Original source](https://arcprize.org/media/data/leaderboard/v2.json) Source arc prize Original model label Gemini 3.5 Flash (High) Source version v2 Observed 2026-10-21T16:34:20.750Z Evidence SHA-256 `bca68e12cc9a01954ae21b641844c1ecbabe146cb24b87f0f8e9b1db27433c29` Source locator `/evaluations/155` |  |  |  |  |  |  |  |
| [Arena text / chinese](https://benchize.com/benchmarks/lmarena%3Atext%3Achinese.md) | 1,524.95 | — | 22 / 146 | — | — | [Arena](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/text/latest) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `rating` \| 1524.9478295985198 \| rating \| reported \| Higher is better \| ### Configuration category chinese config text Reasoning effort high thinkingEnabled true ### Reported statistics lower 1514.4668288762214 rank 27 upper 1535.428830320818 variance 28.596265447137498 votes 3620 ### Source [Original source](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/text/latest) Source Arena Original model label Gemini 3.5 Flash (High) Observed 2026-10-21T16:40:19.533Z Evidence SHA-256 `8e027ef69b8de06c4b2a8b217bfe816b1f1536269bb257fd295b4ef1ba8246ac` Source locator `/rows/443/row` |  |  |  |  |  |  |  |
| [EMB / comps](https://benchize.com/benchmarks/vals-ai%3Aemb%3Acomps.md) | 69.79% | SE ±4.04 pp | 17 / 47 | $8.1 | 1,282.56source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 69.788 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 8.099855 \| USD \| per_test \| Lower is better \| \| `latency` \| 1282.564 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±4.04 pp sourcePopulation 76 stderr 4.035 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) Source Vals AI Original model label google/gemini-3.5-flash Source version 1 Observed 2026-10-21T16:45:57.495Z Evidence SHA-256 `d807aee620ea47152a1fca0257a8dd12a1be4512c76cc1744c1ae9e285180918` Source locator `/tasks/comps/google~1gemini-3.5-flash` |  |  |  |  |  |  |  |
| [Harvey's Legal Agent Benchmark v1 / funds asset management](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Afunds_asset_management.md) | 0% | SE ±0 pp | 11 / 49 | $2.3 | 522.94source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 2.30202 \| USD \| per_test \| Lower is better \| \| `latency` \| 522.94 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 81 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label google/gemini-3.5-flash Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/funds_asset_management/google~1gemini-3.5-flash` |  |  |  |  |  |  |  |
| [Harvey's Legal Agent Benchmark v1 / healthcare life sciences](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Ahealthcare_life_sciences.md) | 0% | SE ±0 pp | 24 / 49 | $2.41 | 575.66source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 2.41183 \| USD \| per_test \| Lower is better \| \| `latency` \| 575.66 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 81 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label google/gemini-3.5-flash Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/healthcare_life_sciences/google~1gemini-3.5-flash` |  |  |  |  |  |  |  |
| [Terminal-Bench 2.0 / medium](https://benchize.com/benchmarks/vals-ai%3Aterminal-bench-2%3Amedium.md) | 72.73% | SE ±2.78 pp | 3 / 23 | $2.95 | 660.3source duration | [Vals AI · 2.0](https://www.vals.ai/_astro/benchmark_view_terminal_bench_2.T48tPR4O.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 72.727 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 2.947382 \| USD \| per_test \| Lower is better \| \| `latency` \| 660.295 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.78 pp sourcePopulation 67 stderr 2.777 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_terminal_bench_2.T48tPR4O.json) Source Vals AI Original model label google/gemini-3.5-flash Source version 2.0 Observed 2026-10-21T16:46:14.732Z Evidence SHA-256 `38e640d995259778e531f048b2f2c7b3b281cb05b56be643ddc9dc53eb85adf7` Source locator `/tasks/medium/google~1gemini-3.5-flash` |  |  |  |  |  |  |  |
| [Arena text / industry entertainment and sports and media](https://benchize.com/benchmarks/lmarena%3Atext%3Aindustry_entertainment_and_sports_and_media.md) | 1,458.68 | — | 20 / 151 | — | — | [Arena](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/text/latest) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `rating` \| 1458.677682563511 \| rating \| reported \| Higher is better \| ### Configuration category industry_entertainment_and_sports_and_media config text Reasoning effort high thinkingEnabled true ### Reported statistics lower 1452.1609066643587 rank 20 upper 1465.1944584626633 variance 11.055270953574459 votes 11925 ### Source [Original source](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/text/latest) Source Arena Original model label Gemini 3.5 Flash (High) Observed 2026-10-21T16:40:19.533Z Evidence SHA-256 `8e027ef69b8de06c4b2a8b217bfe816b1f1536269bb257fd295b4ef1ba8246ac` Source locator `/rows/4712/row` |  |  |  |  |  |  |  |
| [Legal Research Bench / business commercial](https://benchize.com/benchmarks/vals-ai%3Alegal_research%3Abusiness_commercial.md) | 27.91% | SE ±6.92 pp | 32 / 50 | $1.23 | 317.65source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 27.907 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 1.232191 \| USD \| per_test \| Lower is better \| \| `latency` \| 317.652 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±6.92 pp sourcePopulation 82 stderr 6.921 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) Source Vals AI Original model label google/gemini-3.5-flash Source version 1 Observed 2026-10-21T16:45:52.389Z Evidence SHA-256 `26c58dfd47d0d2dc86f33c6d056f30309e0ee63570e7cd752c593b385f9d862e` Source locator `/tasks/business_commercial/google~1gemini-3.5-flash` |  |  |  |  |  |  |  |
| [Harvey's Legal Agent Benchmark v1 / criteria pass rate data privacy cybersecurity](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__data_privacy_cybersecurity.md) | 84.45% | SE ±2.16 pp | 35 / 49 | $1.79 | 460.39source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 84.452 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 1.788615 \| USD \| per_test \| Lower is better \| \| `latency` \| 460.386 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.16 pp sourcePopulation 81 stderr 2.158 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label google/gemini-3.5-flash Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__data_privacy_cybersecurity/google~1gemini-3.5-flash` |  |  |  |  |  |  |  |
| [SAGE / automata](https://benchize.com/benchmarks/vals-ai%3Asage%3Aautomata.md) | 38.49% | SE ±6.37 pp | 15 / 41 | $0.05 | 27.74source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_sage.RRwIIzO5.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 38.492 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.051282 \| USD \| per_test \| Lower is better \| \| `latency` \| 27.741 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 30000 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±6.37 pp sourcePopulation 91 stderr 6.367 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_sage.RRwIIzO5.json) Source Vals AI Original model label google/gemini-3.5-flash Source version 1 Observed 2026-10-21T16:46:07.515Z Evidence SHA-256 `cc5fb3fecdb0382d492ff167a42ad84b9c5c3883373c78ed87baaa972b8b6f7c` Source locator `/tasks/automata/google~1gemini-3.5-flash` |  |  |  |  |  |  |  |
| [Tax Agent Bench / overall](https://benchize.com/benchmarks/vals-ai%3Atax_agent_bench%3Aoverall.md) | 21.65% | SE ±2.96 pp | 32 / 46 | $0.86 | 233source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 21.652 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.859679 \| USD \| per_test \| Lower is better \| \| `latency` \| 233.001 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.96 pp sourcePopulation 73 stderr 2.963 tokenTotals {"cache_read_tokens":288602367,"cache_write_tokens":null,"input_tokens":335113206,"output_tokens":5873488,"reasoning_tokens":4405887} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) Source Vals AI Original model label google/gemini-3.5-flash Source version 1 Observed 2026-10-21T16:45:55.958Z Evidence SHA-256 `446db182b67efb7923b3187a4040ec3a3ea492b8ad699a2b6d708d714cc063ac` Source locator `/tasks/overall/google~1gemini-3.5-flash` |  |  |  |  |  |  |  |
| [Finance Agent (v2) / market analysis](https://benchize.com/benchmarks/vals-ai%3Afabv2%3Amarket_analysis.md) | 73.97% | SE ±1.91 pp | 5 / 50 | $1.49 | 231.63source duration | [Vals AI · 2](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 73.97 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 1.486992 \| USD \| per_test \| Lower is better \| \| `latency` \| 231.63 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.91 pp sourcePopulation 80 stderr 1.911 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) Source Vals AI Original model label google/gemini-3.5-flash Source version 2 Observed 2026-10-21T16:45:57.035Z Evidence SHA-256 `f731b20a26c126725d125519f2b5841aa45254656dba859a9570d7ce7e5d8191` Source locator `/tasks/market_analysis/google~1gemini-3.5-flash` |  |  |  |  |  |  |  |
| [Harvey's Legal Agent Benchmark v1 / criteria pass rate insurance](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__insurance.md) | 63.74% | SE ±2.92 pp | 41 / 49 | $1.77 | 401.88source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 63.736 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 1.769687 \| USD \| per_test \| Lower is better \| \| `latency` \| 401.878 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.92 pp sourcePopulation 81 stderr 2.915 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label google/gemini-3.5-flash Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__insurance/google~1gemini-3.5-flash` |  |  |  |  |  |  |  |
| [MedScribe / overall](https://benchize.com/benchmarks/vals-ai%3Amedscribe%3Aoverall.md) | 76.57% | SE ±1.92 pp | 34 / 49 | $0.17 | 57.08source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_medscribe.BYf5a3C-.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 76.574 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.166341 \| USD \| per_test \| Lower is better \| \| `latency` \| 57.075 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 30000 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.92 pp sourcePopulation 109 stderr 1.923 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_medscribe.BYf5a3C-.json) Source Vals AI Original model label google/gemini-3.5-flash Source version 1 Observed 2026-10-21T16:46:00.004Z Evidence SHA-256 `271fc122ddc2b0602bd6c7947f65a9a7f0d8cab94f6dc27d62a9380a9317f9c0` Source locator `/tasks/overall/google~1gemini-3.5-flash` |  |  |  |  |  |  |  |
| [MMLU Pro / business](https://benchize.com/benchmarks/vals-ai%3Ammlu_pro%3Abusiness.md) | 93.28% | SE ±0.89 pp | 6 / 40 | $0.02 | 17.42source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_mmlu_pro.y416pXDi.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 93.283 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.02368 \| USD \| per_test \| Lower is better \| \| `latency` \| 17.416 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.89 pp sourcePopulation 138 stderr 0.891 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_mmlu_pro.y416pXDi.json) Source Vals AI Original model label google/gemini-3.5-flash Source version 1 Observed 2026-10-21T16:46:06.503Z Evidence SHA-256 `dab025507f46c716dd8d145e90ec0a737d9c56219853f41e4b1cc3c5e8c222b4` Source locator `/tasks/business/google~1gemini-3.5-flash` |  |  |  |  |  |  |  |
| [Chess Puzzles](https://benchize.com/benchmarks/epoch-ai%3Achess_puzzles.md) | 45% | SE ±5 pp | 22 / 63 | — | — | [epoch ai · Chess Puzzles](https://epoch.ai/data/benchmark_data.zip#file=chess_puzzles.csv) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 0.45 \| score_fraction \| Best score (across scorers) \| Higher is better \| \| `mean_score` \| 0.45 \| score_fraction \| mean_score \| Higher is better \| ### Configuration benchmarkVersion Chess Puzzles reasoningBudget Not reported Reasoning effort high sourceModelVersion gemini-3.5-flash_high ### Reported statistics Uncertainty Standard error: ±5 pp standardError 0.05 ### Source [Original source](https://epoch.ai/data/benchmark_data.zip#file=chess_puzzles.csv) Source epoch ai Original model label gemini-3.5-flash Source version Chess Puzzles Evaluated 2026-10-13T13:33:53.000Z Observed 2026-10-21T16:30:24.199Z Evidence SHA-256 `fa0a449c370b0cceaf273d9472078cce3e5aef6be7009bddf115e6df7f930c54` Source locator `/rows/112` |  |  |  |  |  |  |  |
| [Arena text / expert](https://benchize.com/benchmarks/lmarena%3Atext%3Aexpert.md) | 1,494.82 | — | 28 / 148 | — | — | [Arena](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/text/latest) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `rating` \| 1494.8215657172702 \| rating \| reported \| Higher is better \| ### Configuration category expert config text Reasoning effort high thinkingEnabled true ### Reported statistics lower 1486.3326364161846 rank 32 upper 1503.3104950183558 variance 18.758998610275114 votes 5578 ### Source [Original source](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/text/latest) Source Arena Original model label Gemini 3.5 Flash (High) Observed 2026-10-21T16:40:19.533Z Evidence SHA-256 `8e027ef69b8de06c4b2a8b217bfe816b1f1536269bb257fd295b4ef1ba8246ac` Source locator `/rows/2502/row` |  |  |  |  |  |  |  |
| [Earthborne Rangers (EBR-bench)](https://benchize.com/benchmarks/epoch-ai%3Aebr_bench.md) | 4.76% | — | 13 / 13 | — | — | [epoch ai · EBR-bench](https://epoch.ai/data/benchmark_data.zip#file=ebr_bench.csv) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 0.047619047619047616 \| score_fraction \| Best score (across scorers) \| Higher is better \| \| `mean_score` \| 0.047619047619047616 \| score_fraction \| mean_score \| Higher is better \| ### Configuration benchmarkVersion EBR-bench reasoningBudget Not reported Reasoning effort high sourceModelVersion gemini-3.5-flash_high ### Source [Original source](https://epoch.ai/data/benchmark_data.zip#file=ebr_bench.csv) Source epoch ai Original model label gemini-3.5-flash Source version EBR-bench Evaluated 2026-06-25T01:46:45.000Z Observed 2026-10-21T16:30:24.199Z Evidence SHA-256 `15ec96ff8e424623e6a7452956d849312f595f2b5cb4ebe70ec47831f9087b44` Source locator `/rows/21` |  |  |  |  |  |  |  |
| [ProgramBench / partial](https://benchize.com/benchmarks/vals-ai%3Aprogrambench%3Apartial.md) | 56.36% | SE ±1.94 pp | 30 / 40 | — | 1,104.52source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_programbench.C_J7ZU4X.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 56.356 \| percent \| reported \| Higher is better \| \| `latency` \| 1104.517 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.94 pp sourcePopulation 64 stderr 1.942 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_programbench.C_J7ZU4X.json) Source Vals AI Original model label google/gemini-3.5-flash Source version 1 Observed 2026-10-21T16:46:10.137Z Evidence SHA-256 `e9bbb4378c7a058441a21d456baf9316b96c2cf5b58d371a3c7c803f5000bd8c` Source locator `/tasks/partial/google~1gemini-3.5-flash` |  |  |  |  |  |  |  |
| [Finance Agent (v2) / adjustments](https://benchize.com/benchmarks/vals-ai%3Afabv2%3Aadjustments.md) | 54.87% | SE ±1.14 pp | 5 / 50 | $2.3 | 317.21source duration | [Vals AI · 2](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 54.869 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 2.304186 \| USD \| per_test \| Lower is better \| \| `latency` \| 317.205 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.14 pp sourcePopulation 80 stderr 1.14 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) Source Vals AI Original model label google/gemini-3.5-flash Source version 2 Observed 2026-10-21T16:45:57.035Z Evidence SHA-256 `f731b20a26c126725d125519f2b5841aa45254656dba859a9570d7ce7e5d8191` Source locator `/tasks/adjustments/google~1gemini-3.5-flash` |  |  |  |  |  |  |  |
| [Harvey's Legal Agent Benchmark v1 / arbitration international dispute resolution](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Aarbitration_international_dispute_resolution.md) | 0% | SE ±0 pp | 4 / 49 | $2.45 | 573.84source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 2.445749 \| USD \| per_test \| Lower is better \| \| `latency` \| 573.844 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 81 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label google/gemini-3.5-flash Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/arbitration_international_dispute_resolution/google~1gemini-3.5-flash` |  |  |  |  |  |  |  |
| [Arena vision / humor (config: vision)](https://benchize.com/benchmarks/benchmark%3Avision%3Ahumor--config-vision.md) | 1,321.39 | — | 4 / 63 | — | — | [Arena](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/vision/latest) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `rating` \| 1321.3855700046079 \| rating \| reported \| Higher is better \| ### Configuration category humor config vision Reasoning effort high thinkingEnabled true ### Reported statistics lower 1295.9675682281559 rank 5 upper 1346.80357178106 variance 168.18475596491888 votes 577 ### Source [Original source](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/vision/latest) Source Arena Original model label Gemini 3.5 Flash (High) Observed 2026-10-21T16:40:19.533Z Evidence SHA-256 `06e61305486dce3fec5dc776ac2fd8321104e84c346cf9e08fa6f8e8a66ce0d7` Source locator `/rows/930/row` |  |  |  |  |  |  |  |
| [Harvey's Legal Agent Benchmark v1 / insurance](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Ainsurance.md) | 0% | SE ±0 pp | 2 / 49 | $1.77 | 401.88source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 1.769687 \| USD \| per_test \| Lower is better \| \| `latency` \| 401.878 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 81 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label google/gemini-3.5-flash Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/insurance/google~1gemini-3.5-flash` |  |  |  |  |  |  |  |
| [Legal Research Bench / civil litigation](https://benchize.com/benchmarks/vals-ai%3Alegal_research%3Acivil_litigation.md) | 22.22% | SE ±8.15 pp | 36 / 50 | $1.23 | 255.11source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 22.222 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 1.230964 \| USD \| per_test \| Lower is better \| \| `latency` \| 255.111 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±8.15 pp sourcePopulation 82 stderr 8.153 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) Source Vals AI Original model label google/gemini-3.5-flash Source version 1 Observed 2026-10-21T16:45:52.389Z Evidence SHA-256 `26c58dfd47d0d2dc86f33c6d056f30309e0ee63570e7cd752c593b385f9d862e` Source locator `/tasks/civil_litigation/google~1gemini-3.5-flash` |  |  |  |  |  |  |  |
| [Terminal-Bench 2.1 / medium](https://benchize.com/benchmarks/vals-ai%3Aterminal-bench-2-1%3Amedium.md) | 80% | SE ±1.05 pp | 16 / 43 | $1.4 | 603.94source duration | [Vals AI · 2.1](https://www.vals.ai/_astro/benchmark_view_terminal_bench_2_1.CdyoObsA.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 80 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 1.404871 \| USD \| per_test \| Lower is better \| \| `latency` \| 603.942 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.05 pp sourcePopulation 76 stderr 1.05 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_terminal_bench_2_1.CdyoObsA.json) Source Vals AI Original model label google/gemini-3.5-flash Source version 2.1 Observed 2026-10-21T16:46:13.216Z Evidence SHA-256 `a5306805cdf86495bdcceff6b2c8534a3768b9853f5077895dab975b809b054f` Source locator `/tasks/medium/google~1gemini-3.5-flash` |  |  |  |  |  |  |  |
| [Arena text style control / hard prompts english](https://benchize.com/benchmarks/lmarena%3Atext_style_control%3Ahard_prompts_english.md) | 1,490.27 | — | 39 / 151 | — | — | [Arena](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/text_style_control/latest) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `rating` \| 1490.2737753434021 \| rating \| reported \| Higher is better \| ### Configuration category hard_prompts_english config text_style_control Reasoning effort high thinkingEnabled true ### Reported statistics lower 1484.2065404709651 rank 53 upper 1496.3410102158387 variance 9.582645738386795 votes 13780 ### Source [Original source](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/text_style_control/latest) Source Arena Original model label Gemini 3.5 Flash (High) Observed 2026-10-21T16:40:19.533Z Evidence SHA-256 `d4875c03cb7ec0c6e8943c3145fd261225e8841b937240dc62a3032c4a798d84` Source locator `/rows/3920/row` |  |  |  |  |  |  |  |
| [Harvey's Legal Agent Benchmark v1 / criteria pass rate immigration](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__immigration.md) | 76.73% | SE ±2.55 pp | 38 / 49 | $1.56 | 419.57source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 76.727 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 1.560451 \| USD \| per_test \| Lower is better \| \| `latency` \| 419.566 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.55 pp sourcePopulation 81 stderr 2.553 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label google/gemini-3.5-flash Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__immigration/google~1gemini-3.5-flash` |  |  |  |  |  |  |  |
| [Harvey's Legal Agent Benchmark v1 / antitrust competition](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Aantitrust_competition.md) | 0% | SE ±0 pp | 7 / 49 | $2.16 | 444.68source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 2.159387 \| USD \| per_test \| Lower is better \| \| `latency` \| 444.684 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 81 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label google/gemini-3.5-flash Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/antitrust_competition/google~1gemini-3.5-flash` |  |  |  |  |  |  |  |
| [MMLU-Pro](https://benchize.com/benchmarks/mmlu-pro.md) | 89.52% | SE ±0.31 pp | 9 / 40 | $0.02 | 26.26source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_mmlu_pro.y416pXDi.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 89.515 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.02498 \| USD \| per_test \| Lower is better \| \| `latency` \| 26.262 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.31 pp sourcePopulation 138 stderr 0.306 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_mmlu_pro.y416pXDi.json) Source Vals AI Original model label google/gemini-3.5-flash Source version 1 Observed 2026-10-21T16:46:06.503Z Evidence SHA-256 `dab025507f46c716dd8d145e90ec0a737d9c56219853f41e4b1cc3c5e8c222b4` Source locator `/tasks/overall/google~1gemini-3.5-flash` |  |  |  |  |  |  |  |
| [Legal Research Bench / weighted pass criminal](https://benchize.com/benchmarks/vals-ai%3Alegal_research%3Aweighted_pass__criminal.md) | 82.44% | SE ±2.63 pp | 25 / 50 | $1.36 | 256.08source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 82.44 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 1.359179 \| USD \| per_test \| Lower is better \| \| `latency` \| 256.075 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.63 pp sourcePopulation 82 stderr 2.626 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) Source Vals AI Original model label google/gemini-3.5-flash Source version 1 Observed 2026-10-21T16:45:52.389Z Evidence SHA-256 `26c58dfd47d0d2dc86f33c6d056f30309e0ee63570e7cd752c593b385f9d862e` Source locator `/tasks/weighted_pass__criminal/google~1gemini-3.5-flash` |  |  |  |  |  |  |  |
| [Arena text style control / creative writing](https://benchize.com/benchmarks/lmarena%3Atext_style_control%3Acreative_writing.md) | 1,464.3 | — | 22 / 151 | — | — | [Arena](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/text_style_control/latest) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `rating` \| 1464.2968066193812 \| rating \| reported \| Higher is better \| ### Configuration category creative_writing config text_style_control Reasoning effort high thinkingEnabled true ### Reported statistics lower 1457.0964747762819 rank 23 upper 1471.4971384624805 variance 13.496117248858711 votes 9473 ### Source [Original source](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/text_style_control/latest) Source Arena Original model label Gemini 3.5 Flash (High) Observed 2026-10-21T16:40:19.533Z Evidence SHA-256 `d4875c03cb7ec0c6e8943c3145fd261225e8841b937240dc62a3032c4a798d84` Source locator `/rows/1244/row` |  |  |  |  |  |  |  |
| [LegalBench / overall](https://benchize.com/benchmarks/vals-ai%3Alegal_bench%3Aoverall.md) | 83.6% | SE ±0.85 pp | 22 / 44 | $0.00755 | 34.58source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_bench.D6uTgA3b.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 83.602 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.007553 \| USD \| per_test \| Lower is better \| \| `latency` \| 34.576 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.85 pp sourcePopulation 151 stderr 0.849 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_bench.D6uTgA3b.json) Source Vals AI Original model label google/gemini-3.5-flash Source version 1 Observed 2026-10-21T16:45:53.390Z Evidence SHA-256 `c8486ba756baa5cd3955d035edff67440e8db23e69d62ad1a9e18d15b8c4bc52` Source locator `/tasks/overall/google~1gemini-3.5-flash` |  |  |  |  |  |  |  |
| [Harvey's Legal Agent Benchmark v1 / corporate ma](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acorporate_ma.md) | 0% | SE ±0 pp | 21 / 49 | $1.88 | 421.04source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 1.877373 \| USD \| per_test \| Lower is better \| \| `latency` \| 421.036 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 81 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label google/gemini-3.5-flash Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/corporate_ma/google~1gemini-3.5-flash` |  |  |  |  |  |  |  |
| [Tax Agent Bench / current temporal analysis](https://benchize.com/benchmarks/vals-ai%3Atax_agent_bench%3Acurrent_temporal_analysis.md) | 26.44% | SE ±8.15 pp | 34 / 46 | $0.93 | 279.9source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 26.444 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.929565 \| USD \| per_test \| Lower is better \| \| `latency` \| 279.898 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±8.15 pp sourcePopulation 73 stderr 8.146 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) Source Vals AI Original model label google/gemini-3.5-flash Source version 1 Observed 2026-10-21T16:45:55.958Z Evidence SHA-256 `446db182b67efb7923b3187a4040ec3a3ea492b8ad699a2b6d708d714cc063ac` Source locator `/tasks/current_temporal_analysis/google~1gemini-3.5-flash` |  |  |  |  |  |  |  |
| [FrontierMath (2025-02-28 private)](https://benchize.com/benchmarks/epoch-ai%3Afrontiermath.md) | 38.97% | SE ±2.87 pp | 4 / 18 | — | — | [epoch ai · FrontierMath-2025-02-28-Private](https://epoch.ai/data/benchmark_data.zip#file=frontiermath.csv) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 0.3896551724137931 \| score_fraction \| Best score (across scorers) \| Higher is better \| \| `mean_score` \| 0.3896551724137931 \| score_fraction \| mean_score \| Higher is better \| ### Configuration benchmarkVersion FrontierMath-2025-02-28-Private reasoningBudget Not reported Reasoning effort high sourceModelVersion gemini-3.5-flash_high ### Reported statistics Uncertainty Standard error: ±2.87 pp standardError 0.028686590614000477 ### Source [Original source](https://epoch.ai/data/benchmark_data.zip#file=frontiermath.csv) Source epoch ai Original model label gemini-3.5-flash Source version FrontierMath-2025-02-28-Private Evaluated 2026-05-22T16:42:34.000Z Observed 2026-10-21T16:30:24.199Z Evidence SHA-256 `f5c92f0f0b9103abb02308fd428c0806e7c21b91e8a14d43edb8af77ce941598` Source locator `/rows/2` |  |  |  |  |  |  |  |
| [Finance Agent (v2) / general qualitative analysis](https://benchize.com/benchmarks/vals-ai%3Afabv2%3Ageneral_qualitative_analysis.md) | 56.83% | SE ±0.42 pp | 36 / 50 | $2.62 | 287.44source duration | [Vals AI · 2](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 56.832 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 2.622948 \| USD \| per_test \| Lower is better \| \| `latency` \| 287.438 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.42 pp sourcePopulation 80 stderr 0.418 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) Source Vals AI Original model label google/gemini-3.5-flash Source version 2 Observed 2026-10-21T16:45:57.035Z Evidence SHA-256 `f731b20a26c126725d125519f2b5841aa45254656dba859a9570d7ce7e5d8191` Source locator `/tasks/general_qualitative_analysis/google~1gemini-3.5-flash` |  |  |  |  |  |  |  |
| [EMB / dataroom](https://benchize.com/benchmarks/vals-ai%3Aemb%3Adataroom.md) | 80.92% | SE ±5.38 pp | 21 / 47 | $4.45 | 1,239.06source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 80.921 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 4.451663 \| USD \| per_test \| Lower is better \| \| `latency` \| 1239.062 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±5.38 pp sourcePopulation 76 stderr 5.377 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) Source Vals AI Original model label google/gemini-3.5-flash Source version 1 Observed 2026-10-21T16:45:57.495Z Evidence SHA-256 `d807aee620ea47152a1fca0257a8dd12a1be4512c76cc1744c1ae9e285180918` Source locator `/tasks/dataroom/google~1gemini-3.5-flash` |  |  |  |  |  |  |  |
| [Harvey's Legal Agent Benchmark v1 / criteria pass rate corporate ma](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__corporate_ma.md) | 86.54% | SE ±1.94 pp | 36 / 49 | $1.88 | 421.04source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 86.538 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 1.877373 \| USD \| per_test \| Lower is better \| \| `latency` \| 421.036 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.94 pp sourcePopulation 81 stderr 1.935 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label google/gemini-3.5-flash Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__corporate_ma/google~1gemini-3.5-flash` |  |  |  |  |  |  |  |
| [Arena text style control / polish](https://benchize.com/benchmarks/lmarena%3Atext_style_control%3Apolish.md) | 1,505.41 | — | 4 / 117 | — | — | [Arena](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/text_style_control/latest) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `rating` \| 1505.409734851644 \| rating \| reported \| Higher is better \| ### Configuration category polish config text_style_control Reasoning effort high thinkingEnabled true ### Reported statistics lower 1483.751668405633 rank 4 upper 1527.0678012976553 variance 122.10773668923001 votes 797 ### Source [Original source](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/text_style_control/latest) Source Arena Original model label Gemini 3.5 Flash (High) Observed 2026-10-21T16:40:19.533Z Evidence SHA-256 `d4875c03cb7ec0c6e8943c3145fd261225e8841b937240dc62a3032c4a798d84` Source locator `/rows/10151/row` |  |  |  |  |  |  |  |
| [Finance Agent (v2) / disclosure analysis](https://benchize.com/benchmarks/vals-ai%3Afabv2%3Adisclosure_analysis.md) | 63.95% | SE ±2.29 pp | 20 / 50 | $2.59 | 307.82source duration | [Vals AI · 2](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 63.948 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 2.588355 \| USD \| per_test \| Lower is better \| \| `latency` \| 307.82 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.29 pp sourcePopulation 80 stderr 2.287 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) Source Vals AI Original model label google/gemini-3.5-flash Source version 2 Observed 2026-10-21T16:45:57.035Z Evidence SHA-256 `f731b20a26c126725d125519f2b5841aa45254656dba859a9570d7ce7e5d8191` Source locator `/tasks/disclosure_analysis/google~1gemini-3.5-flash` |  |  |  |  |  |  |  |
| [Arena text / hard prompts english](https://benchize.com/benchmarks/lmarena%3Atext%3Ahard_prompts_english.md) | 1,482.61 | — | 27 / 151 | — | — | [Arena](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/text/latest) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `rating` \| 1482.6075541719047 \| rating \| reported \| Higher is better \| ### Configuration category hard_prompts_english config text Reasoning effort high thinkingEnabled true ### Reported statistics lower 1476.599028682172 rank 31 upper 1488.6160796616373 variance 9.398090737372938 votes 13780 ### Source [Original source](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/text/latest) Source Arena Original model label Gemini 3.5 Flash (High) Observed 2026-10-21T16:40:19.533Z Evidence SHA-256 `8e027ef69b8de06c4b2a8b217bfe816b1f1536269bb257fd295b4ef1ba8246ac` Source locator `/rows/3898/row` |  |  |  |  |  |  |  |
| [Arena webdev / webdev-react](https://benchize.com/benchmarks/lmarena%3Awebdev%3Awebdev-react.md) | 1,495.02 | — | 43 / 81 | — | — | [Arena](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/webdev/latest) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `rating` \| 1495.0161705716014 \| rating \| reported \| Higher is better \| ### Configuration category webdev-react config webdev Reasoning effort high thinkingEnabled true ### Reported statistics lower 1486.633061222341 rank 62 upper 1503.3992799208618 variance 18.2942276988823 votes 6781 ### Source [Original source](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/webdev/latest) Source Arena Original model label Gemini 3.5 Flash (High) Observed 2026-10-21T16:40:19.533Z Evidence SHA-256 `f82507859808110820a4ab6df15561b8864137ddefc1f6fb0c552af2e58ea738` Source locator `/rows/568/row` |  |  |  |  |  |  |  |
| [MortgageTax / semantic extraction](https://benchize.com/benchmarks/vals-ai%3Amortgage_tax%3Asemantic_extraction.md) | 58.74% | SE ±1.39 pp | 19 / 31 | $0.02 | 10.79source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_mortgage_tax.DCd-oO5Y.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 58.744 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.01655 \| USD \| per_test \| Lower is better \| \| `latency` \| 10.793 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.39 pp sourcePopulation 98 stderr 1.388 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_mortgage_tax.DCd-oO5Y.json) Source Vals AI Original model label google/gemini-3.5-flash Source version 1 Observed 2026-10-21T16:45:58.500Z Evidence SHA-256 `e5c844561f97d119d66d70269e726ecfb68be79eae39acf14ead3e95422e648c` Source locator `/tasks/semantic_extraction/google~1gemini-3.5-flash` |  |  |  |  |  |  |  |
| [Finance Agent (v2) / financial modeling](https://benchize.com/benchmarks/vals-ai%3Afabv2%3Afinancial_modeling.md) | 29.37% | SE ±0.94 pp | 7 / 50 | $2.97 | 424.76source duration | [Vals AI · 2](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 29.372 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 2.97355 \| USD \| per_test \| Lower is better \| \| `latency` \| 424.764 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.94 pp sourcePopulation 80 stderr 0.936 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) Source Vals AI Original model label google/gemini-3.5-flash Source version 2 Observed 2026-10-21T16:45:57.035Z Evidence SHA-256 `f731b20a26c126725d125519f2b5841aa45254656dba859a9570d7ce7e5d8191` Source locator `/tasks/financial_modeling/google~1gemini-3.5-flash` |  |  |  |  |  |  |  |
| [Public Benefits Bench v1 / multi turn](https://benchize.com/benchmarks/vals-ai%3Apublic-benefits-bench-v1%3Amulti_turn.md) | 53.99% | SE ±1.3 pp | 3 / 8 | $0.13 | 129.71source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_snap_v1.BPa0Hrre.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 53.992 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.130219 \| USD \| per_test \| Lower is better \| \| `latency` \| 129.707 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.3 pp sourcePopulation 13 stderr 1.296 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_snap_v1.BPa0Hrre.json) Source Vals AI Original model label google/gemini-3.5-flash Source version 1 Observed 2026-10-21T16:46:18.363Z Evidence SHA-256 `b85e2b3cbdf03e13e2ffdb27ce4f108003a548d705a24ae26a0a115e2027475d` Source locator `/tasks/multi_turn/google~1gemini-3.5-flash` |  |  |  |  |  |  |  |
| [Arena text / coding](https://benchize.com/benchmarks/lmarena%3Atext%3Acoding.md) | 1,493.07 | — | 27 / 151 | — | — | [Arena](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/text/latest) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `rating` \| 1493.069564292129 \| rating \| reported \| Higher is better \| ### Configuration category coding config text Reasoning effort high thinkingEnabled true ### Reported statistics lower 1487.0394768913445 rank 33 upper 1499.099651692913 variance 9.465662853188174 votes 13845 ### Source [Original source](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/text/latest) Source Arena Original model label Gemini 3.5 Flash (High) Observed 2026-10-21T16:40:19.533Z Evidence SHA-256 `8e027ef69b8de06c4b2a8b217bfe816b1f1536269bb257fd295b4ef1ba8246ac` Source locator `/rows/842/row` |  |  |  |  |  |  |  |

[Next page](https://benchize.com/models/gemini-3-5-flash-high.md?cursor=57f1eacf5102f260e7323463d420e13954588fbdbd37c875c773be38d6f07a71)

---
Cite as: Gemini 3.5 FlashHigh — Benchize, https://benchize.com/models/gemini-3-5-flash-high (retrieved 2026-10-21)