# Gemini 3.1 Flash Lite PreviewHigh > Source: Benchize (https://benchize.com) — https://benchize.com/models/gemini-3-1-flash-lite-preview-high. When citing or reusing this content, credit "Benchize" and link to https://benchize.com/models/gemini-3-1-flash-lite-preview-high. Google Canonical URL: https://benchize.com/models/gemini-3-1-flash-lite-preview-high Last updated: Thu, 22 Oct 2026 09:10:07 GMT Reproduction and quotation must name Benchize, link to the canonical HTML URL, and retain individual source citations. Google ## Benchmarks | Benchmark | Score | ± | Rank | Cost | Latency | Source | | | --- | --- | --- | --- | --- | --- | --- | --- | | [ProgramBench / overall](https://benchize.com/benchmarks/vals-ai%3Aprogrambench%3Aoverall.md) | 0% | SE ±0 pp | 23 / 40 | $0.06 | 176.25source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_programbench.C_J7ZU4X.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.063906 \| USD \| per_test \| Lower is better \| \| `latency` \| 176.246 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 64 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_programbench.C_J7ZU4X.json) Source Vals AI Original model label google/gemini-3.1-flash-lite-preview Source version 1 Observed 2026-10-22T02:10:24.817Z Evidence SHA-256 `e9bbb4378c7a058441a21d456baf9316b96c2cf5b58d371a3c7c803f5000bd8c` Source locator `/tasks/overall/google~1gemini-3.1-flash-lite-preview` | | | | | | | | | [Legal Research Bench / weighted pass business commercial](https://benchize.com/benchmarks/vals-ai%3Alegal_research%3Aweighted_pass__business_commercial.md) | 54.63% | SE ±3.65 pp | 50 / 50 | $0.01 | 23.44source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 54.626 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.014473 \| USD \| per_test \| Lower is better \| \| `latency` \| 23.436 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±3.65 pp sourcePopulation 82 stderr 3.647 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) Source Vals AI Original model label google/gemini-3.1-flash-lite-preview Source version 1 Observed 2026-10-22T02:10:07.934Z Evidence SHA-256 `26c58dfd47d0d2dc86f33c6d056f30309e0ee63570e7cd752c593b385f9d862e` Source locator `/tasks/weighted_pass__business_commercial/google~1gemini-3.1-flash-lite-preview` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / criteria pass rate intellectual property](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__intellectual_property.md) | 34.63% | SE ±2.83 pp | 49 / 49 | $0.09 | 65.61source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 34.629 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.092629 \| USD \| per_test \| Lower is better \| \| `latency` \| 65.61 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.83 pp sourcePopulation 81 stderr 2.833 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label google/gemini-3.1-flash-lite-preview Source version 1 Observed 2026-10-22T02:10:09.422Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__intellectual_property/google~1gemini-3.1-flash-lite-preview` | | | | | | | | | [Tax Agent Bench / controversy precedence analysis](https://benchize.com/benchmarks/vals-ai%3Atax_agent_bench%3Acontroversy_precedence_analysis.md) | 0% | SE ±0 pp | 43 / 46 | $0.01 | 22.45source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.014665 \| USD \| per_test \| Lower is better \| \| `latency` \| 22.448 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 73 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) Source Vals AI Original model label google/gemini-3.1-flash-lite-preview Source version 1 Observed 2026-10-22T02:10:11.407Z Evidence SHA-256 `446db182b67efb7923b3187a4040ec3a3ea492b8ad699a2b6d708d714cc063ac` Source locator `/tasks/controversy_precedence_analysis/google~1gemini-3.1-flash-lite-preview` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / criteria pass rate international trade sanctions](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__international_trade_sanctions.md) | 32.14% | SE ±2.95 pp | 48 / 49 | $0.05 | 47.57source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 32.143 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.051418 \| USD \| per_test \| Lower is better \| \| `latency` \| 47.572 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.95 pp sourcePopulation 81 stderr 2.948 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label google/gemini-3.1-flash-lite-preview Source version 1 Observed 2026-10-22T02:10:09.422Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__international_trade_sanctions/google~1gemini-3.1-flash-lite-preview` | | | | | | | | | [Legal Research Bench / weighted pass civil litigation](https://benchize.com/benchmarks/vals-ai%3Alegal_research%3Aweighted_pass__civil_litigation.md) | 58.85% | SE ±5.22 pp | 49 / 50 | $0.01 | 19.79source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 58.849 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.013273 \| USD \| per_test \| Lower is better \| \| `latency` \| 19.793 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±5.22 pp sourcePopulation 82 stderr 5.219 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) Source Vals AI Original model label google/gemini-3.1-flash-lite-preview Source version 1 Observed 2026-10-22T02:10:07.934Z Evidence SHA-256 `26c58dfd47d0d2dc86f33c6d056f30309e0ee63570e7cd752c593b385f9d862e` Source locator `/tasks/weighted_pass__civil_litigation/google~1gemini-3.1-flash-lite-preview` | | | | | | | | | [Finance Agent (v2) / disclosure analysis](https://benchize.com/benchmarks/vals-ai%3Afabv2%3Adisclosure_analysis.md) | 37.38% | SE ±2.13 pp | 45 / 50 | $0.16 | 54.43source duration | [Vals AI · 2](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 37.38 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.162369 \| USD \| per_test \| Lower is better \| \| `latency` \| 54.432 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.13 pp sourcePopulation 80 stderr 2.128 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) Source Vals AI Original model label google/gemini-3.1-flash-lite-preview Source version 2 Observed 2026-10-22T02:10:12.404Z Evidence SHA-256 `f731b20a26c126725d125519f2b5841aa45254656dba859a9570d7ce7e5d8191` Source locator `/tasks/disclosure_analysis/google~1gemini-3.1-flash-lite-preview` | | | | | | | | | [GPQA Diamond / diamond few shot cot](https://benchize.com/benchmarks/vals-ai%3Agpqa%3Adiamond_few_shot_cot.md) | 79.29% | SE ±2.88 pp | 29 / 40 | $0.00233 | 16.09source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_gpqa.ghZZBpyU.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 79.293 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.002329 \| USD \| per_test \| Lower is better \| \| `latency` \| 16.085 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.88 pp sourcePopulation 138 stderr 2.88 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_gpqa.ghZZBpyU.json) Source Vals AI Original model label google/gemini-3.1-flash-lite-preview Source version 1 Observed 2026-10-22T02:10:20.661Z Evidence SHA-256 `951eb5224097733a9d09d06c14908ae2980e2e640e539f2c284877c5092d7077` Source locator `/tasks/diamond_few_shot_cot/google~1gemini-3.1-flash-lite-preview` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / bankruptcy restructuring](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Abankruptcy_restructuring.md) | 0% | SE ±0 pp | 3 / 49 | $0.11 | 53.97source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.111354 \| USD \| per_test \| Lower is better \| \| `latency` \| 53.972 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 81 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label google/gemini-3.1-flash-lite-preview Source version 1 Observed 2026-10-22T02:10:09.422Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/bankruptcy_restructuring/google~1gemini-3.1-flash-lite-preview` | | | | | | | | | [LegalBench / conclusion tasks](https://benchize.com/benchmarks/vals-ai%3Alegal_bench%3Aconclusion_tasks.md) | 88.47% | SE ±0.71 pp | 24 / 44 | $0.000289 | 2.9source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_bench.D6uTgA3b.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 88.472 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.000289 \| USD \| per_test \| Lower is better \| \| `latency` \| 2.904 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.71 pp sourcePopulation 151 stderr 0.714 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_bench.D6uTgA3b.json) Source Vals AI Original model label google/gemini-3.1-flash-lite-preview Source version 1 Observed 2026-10-22T02:10:08.909Z Evidence SHA-256 `c8486ba756baa5cd3955d035edff67440e8db23e69d62ad1a9e18d15b8c4bc52` Source locator `/tasks/conclusion_tasks/google~1gemini-3.1-flash-lite-preview` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / white collar defense investigations](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Awhite_collar_defense_investigations.md) | 0% | SE ±0 pp | 1 / 49 | $0.07 | 47.61source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.072372 \| USD \| per_test \| Lower is better \| \| `latency` \| 47.606 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 81 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label google/gemini-3.1-flash-lite-preview Source version 1 Observed 2026-10-22T02:10:09.422Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/white_collar_defense_investigations/google~1gemini-3.1-flash-lite-preview` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / criteria pass rate tax](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__tax.md) | 33.45% | SE ±2.73 pp | 47 / 49 | $0.09 | 61.97source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 33.445 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.08909 \| USD \| per_test \| Lower is better \| \| `latency` \| 61.968 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.73 pp sourcePopulation 81 stderr 2.733 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label google/gemini-3.1-flash-lite-preview Source version 1 Observed 2026-10-22T02:10:09.422Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__tax/google~1gemini-3.1-flash-lite-preview` | | | | | | | | | [CorpFin v2 / max fitting context](https://benchize.com/benchmarks/vals-ai%3Acorp_fin_v2%3Amax_fitting_context.md) | 60.61% | SE ±1.67 pp | 26 / 39 | $0.05 | 4source duration | [Vals AI · 2](https://www.vals.ai/_astro/benchmark_view_corp_fin_v2.CectKwl1.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 60.606 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.048652 \| USD \| per_test \| Lower is better \| \| `latency` \| 4.003 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.67 pp sourcePopulation 134 stderr 1.668 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_corp_fin_v2.CectKwl1.json) Source Vals AI Original model label google/gemini-3.1-flash-lite-preview Source version 2 Observed 2026-10-22T02:10:13.429Z Evidence SHA-256 `7cf2217ab3728d5a84bffce82aab9fc52d7f810b0cbe8c65b387df47f3d31c9b` Source locator `/tasks/max_fitting_context/google~1gemini-3.1-flash-lite-preview` | | | | | | | | | [Finance Agent (v2) / general qualitative analysis](https://benchize.com/benchmarks/vals-ai%3Afabv2%3Ageneral_qualitative_analysis.md) | 36.18% | SE ±0.8 pp | 47 / 50 | $0.13 | 42.88source duration | [Vals AI · 2](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 36.184 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.128348 \| USD \| per_test \| Lower is better \| \| `latency` \| 42.875 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.8 pp sourcePopulation 80 stderr 0.795 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) Source Vals AI Original model label google/gemini-3.1-flash-lite-preview Source version 2 Observed 2026-10-22T02:10:12.404Z Evidence SHA-256 `f731b20a26c126725d125519f2b5841aa45254656dba859a9570d7ce7e5d8191` Source locator `/tasks/general_qualitative_analysis/google~1gemini-3.1-flash-lite-preview` | | | | | | | | | [LegalBench / rule tasks](https://benchize.com/benchmarks/vals-ai%3Alegal_bench%3Arule_tasks.md) | 82.17% | SE ±1.58 pp | 16 / 44 | $0.000287 | 3.14source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_bench.D6uTgA3b.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 82.168 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.000287 \| USD \| per_test \| Lower is better \| \| `latency` \| 3.143 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.58 pp sourcePopulation 151 stderr 1.576 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_bench.D6uTgA3b.json) Source Vals AI Original model label google/gemini-3.1-flash-lite-preview Source version 1 Observed 2026-10-22T02:10:08.909Z Evidence SHA-256 `c8486ba756baa5cd3955d035edff67440e8db23e69d62ad1a9e18d15b8c4bc52` Source locator `/tasks/rule_tasks/google~1gemini-3.1-flash-lite-preview` | | | | | | | | | [Legal Research Bench / weighted pass health](https://benchize.com/benchmarks/vals-ai%3Alegal_research%3Aweighted_pass__health.md) | 56.58% | SE ±4.94 pp | 50 / 50 | $0.01 | 19.51source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 56.579 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.013992 \| USD \| per_test \| Lower is better \| \| `latency` \| 19.508 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±4.94 pp sourcePopulation 82 stderr 4.937 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) Source Vals AI Original model label google/gemini-3.1-flash-lite-preview Source version 1 Observed 2026-10-22T02:10:07.934Z Evidence SHA-256 `26c58dfd47d0d2dc86f33c6d056f30309e0ee63570e7cd752c593b385f9d862e` Source locator `/tasks/weighted_pass__health/google~1gemini-3.1-flash-lite-preview` | | | | | | | | | [Vals Multimodal Index / corp fin v2](https://benchize.com/benchmarks/vals-ai%3Avals_multimodal_index%3Acorp_fin_v2.md) | 60.84% | SE ±1.67 pp | 18 / 20 | $0.02 | 4.71source duration | [Vals AI · 1.2](https://www.vals.ai/_astro/benchmark_view_vals_multimodal_index.virVpWMs.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 60.839 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.018404 \| USD \| per_test \| Lower is better \| \| `latency` \| 4.708 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.67 pp sourcePopulation 33 stderr 1.666 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_vals_multimodal_index.virVpWMs.json) Source Vals AI Original model label google/gemini-3.1-flash-lite-preview Source version 1.2 Observed 2026-10-22T02:10:07.406Z Evidence SHA-256 `077b22575f6e075b7b78157d000ec8f94d7ead409c7dcbcf04887630ed572a2d` Source locator `/tasks/corp_fin_v2/google~1gemini-3.1-flash-lite-preview` | | | | | | | | | [Tax Agent Bench / weighted pass forms filings](https://benchize.com/benchmarks/vals-ai%3Atax_agent_bench%3Aweighted_pass__forms_filings.md) | 14.08% | SE ±5.27 pp | 45 / 46 | $0.02 | 24.41source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 14.077 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.017174 \| USD \| per_test \| Lower is better \| \| `latency` \| 24.412 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±5.27 pp sourcePopulation 73 stderr 5.269 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) Source Vals AI Original model label google/gemini-3.1-flash-lite-preview Source version 1 Observed 2026-10-22T02:10:11.407Z Evidence SHA-256 `446db182b67efb7923b3187a4040ec3a3ea492b8ad699a2b6d708d714cc063ac` Source locator `/tasks/weighted_pass__forms_filings/google~1gemini-3.1-flash-lite-preview` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / criteria pass rate trusts estates private client](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__trusts_estates_private_client.md) | 35.5% | SE ±2.96 pp | 49 / 49 | $0.04 | 45.06source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 35.496 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.043684 \| USD \| per_test \| Lower is better \| \| `latency` \| 45.062 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.96 pp sourcePopulation 81 stderr 2.962 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label google/gemini-3.1-flash-lite-preview Source version 1 Observed 2026-10-22T02:10:09.422Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__trusts_estates_private_client/google~1gemini-3.1-flash-lite-preview` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / criteria pass rate structured finance securitization](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__structured_finance_securitization.md) | 26.03% | SE ±2.48 pp | 49 / 49 | $0.07 | 55.1source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 26.032 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.066982 \| USD \| per_test \| Lower is better \| \| `latency` \| 55.102 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.48 pp sourcePopulation 81 stderr 2.476 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label google/gemini-3.1-flash-lite-preview Source version 1 Observed 2026-10-22T02:10:09.422Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__structured_finance_securitization/google~1gemini-3.1-flash-lite-preview` | | | | | | | | | [Legal Research Bench / weighted pass](https://benchize.com/benchmarks/vals-ai%3Alegal_research%3Aweighted_pass.md) | 54.42% | SE ±1.96 pp | 50 / 50 | $0.01 | 20.4source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 54.415 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.013607 \| USD \| per_test \| Lower is better \| \| `latency` \| 20.403 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.96 pp sourcePopulation 82 stderr 1.956 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) Source Vals AI Original model label google/gemini-3.1-flash-lite-preview Source version 1 Observed 2026-10-22T02:10:07.934Z Evidence SHA-256 `26c58dfd47d0d2dc86f33c6d056f30309e0ee63570e7cd752c593b385f9d862e` Source locator `/tasks/weighted_pass/google~1gemini-3.1-flash-lite-preview` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / criteria pass rate banking finance](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__banking_finance.md) | 36.06% | SE ±2.93 pp | 49 / 49 | $0.06 | 49.89source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 36.059 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.057659 \| USD \| per_test \| Lower is better \| \| `latency` \| 49.89 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.93 pp sourcePopulation 81 stderr 2.933 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label google/gemini-3.1-flash-lite-preview Source version 1 Observed 2026-10-22T02:10:09.422Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__banking_finance/google~1gemini-3.1-flash-lite-preview` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / overall](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Aoverall.md) | 0% | SE ±0 pp | 42 / 49 | $0.08 | 60.55source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.084048 \| USD \| per_test \| Lower is better \| \| `latency` \| 60.554 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 81 stderr 0 tokenTotals {"cache_read_tokens":138155856,"cache_write_tokens":null,"input_tokens":160066368,"output_tokens":769494,"reasoning_tokens":360545} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label google/gemini-3.1-flash-lite-preview Source version 1 Observed 2026-10-22T02:10:09.422Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/overall/google~1gemini-3.1-flash-lite-preview` | | | | | | | | | [Tax Agent Bench / weighted pass controversy precedence analysis](https://benchize.com/benchmarks/vals-ai%3Atax_agent_bench%3Aweighted_pass__controversy_precedence_analysis.md) | 11.49% | SE ±4.49 pp | 45 / 46 | $0.01 | 22.45source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 11.494 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.014665 \| USD \| per_test \| Lower is better \| \| `latency` \| 22.448 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±4.49 pp sourcePopulation 73 stderr 4.488 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) Source Vals AI Original model label google/gemini-3.1-flash-lite-preview Source version 1 Observed 2026-10-22T02:10:11.407Z Evidence SHA-256 `446db182b67efb7923b3187a4040ec3a3ea492b8ad699a2b6d708d714cc063ac` Source locator `/tasks/weighted_pass__controversy_precedence_analysis/google~1gemini-3.1-flash-lite-preview` | | | | | | | | | [SAGE / overall](https://benchize.com/benchmarks/vals-ai%3Asage%3Aoverall.md) | 49.54% | SE ±3.48 pp | 12 / 41 | $0.00738 | 16.26source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_sage.RRwIIzO5.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 49.54 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.007378 \| USD \| per_test \| Lower is better \| \| `latency` \| 16.257 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 30000 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±3.48 pp sourcePopulation 91 stderr 3.484 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_sage.RRwIIzO5.json) Source Vals AI Original model label google/gemini-3.1-flash-lite-preview Source version 1 Observed 2026-10-22T02:10:22.165Z Evidence SHA-256 `cc5fb3fecdb0382d492ff167a42ad84b9c5c3883373c78ed87baaa972b8b6f7c` Source locator `/tasks/overall/google~1gemini-3.1-flash-lite-preview` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / banking finance](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Abanking_finance.md) | 0% | SE ±0 pp | 5 / 49 | $0.06 | 49.89source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.057659 \| USD \| per_test \| Lower is better \| \| `latency` \| 49.89 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 81 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label google/gemini-3.1-flash-lite-preview Source version 1 Observed 2026-10-22T02:10:09.422Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/banking_finance/google~1gemini-3.1-flash-lite-preview` | | | | | | | | | [Code Migration / overall](https://benchize.com/benchmarks/vals-ai%3Acode-migration%3Aoverall.md) | 4.61% | SE ±1.07 pp | 46 / 47 | $0.05 | 92.63source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_code_migration.Dttzoqk3.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 4.609 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.053028 \| USD \| per_test \| Lower is better \| \| `latency` \| 92.63 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.07 pp sourcePopulation 79 stderr 1.069 tokenTotals {"cache_read_tokens":61387242,"cache_write_tokens":null,"input_tokens":73960601,"output_tokens":1433780,"reasoning_tokens":895097} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_code_migration.Dttzoqk3.json) Source Vals AI Original model label google/gemini-3.1-flash-lite-preview Source version 1 Observed 2026-10-22T02:10:27.307Z Evidence SHA-256 `99194597f2c76c402195be47839409046e18bd30454616da6dd40b736a088fa4` Source locator `/tasks/overall/google~1gemini-3.1-flash-lite-preview` | | | | | | | | | [Vals Multimodal Index / vibe code bench](https://benchize.com/benchmarks/vals-ai%3Avals_multimodal_index%3Avibe_code_bench.md) | 0% | SE ±0 pp | 20 / 20 | $0.51 | 572.71source duration | [Vals AI · 1.2](https://www.vals.ai/_astro/benchmark_view_vals_multimodal_index.virVpWMs.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.51 \| USD \| per_test \| Lower is better \| \| `latency` \| 572.711 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 33 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_vals_multimodal_index.virVpWMs.json) Source Vals AI Original model label google/gemini-3.1-flash-lite-preview Source version 1.2 Observed 2026-10-22T02:10:07.406Z Evidence SHA-256 `077b22575f6e075b7b78157d000ec8f94d7ead409c7dcbcf04887630ed572a2d` Source locator `/tasks/vibe_code_bench/google~1gemini-3.1-flash-lite-preview` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / criteria pass rate](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate.md) | 29.61% | SE ±0.55 pp | 49 / 49 | $0.08 | 60.55source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 29.614 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.084048 \| USD \| per_test \| Lower is better \| \| `latency` \| 60.554 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.55 pp sourcePopulation 81 stderr 0.549 tokenTotals {"cache_read_tokens":138155856,"cache_write_tokens":null,"input_tokens":160066368,"output_tokens":769494,"reasoning_tokens":360545} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label google/gemini-3.1-flash-lite-preview Source version 1 Observed 2026-10-22T02:10:09.422Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate/google~1gemini-3.1-flash-lite-preview` | | | | | | | | | [Finance Agent (v2) / earnings analysis](https://benchize.com/benchmarks/vals-ai%3Afabv2%3Aearnings_analysis.md) | 31.26% | SE ±0.37 pp | 46 / 50 | $0.09 | 58source duration | [Vals AI · 2](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 31.261 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.08654 \| USD \| per_test \| Lower is better \| \| `latency` \| 57.997 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.37 pp sourcePopulation 80 stderr 0.373 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) Source Vals AI Original model label google/gemini-3.1-flash-lite-preview Source version 2 Observed 2026-10-22T02:10:12.404Z Evidence SHA-256 `f731b20a26c126725d125519f2b5841aa45254656dba859a9570d7ce7e5d8191` Source locator `/tasks/earnings_analysis/google~1gemini-3.1-flash-lite-preview` | | | | | | | | | [CaseLaw v2 / overall](https://benchize.com/benchmarks/vals-ai%3Acase_law_v2%3Aoverall.md) | 54.98% | SE ±0.74 pp | 14 / 17 | $0.00931 | 4.64source duration | [Vals AI · 2](https://www.vals.ai/_astro/benchmark_view_case_law_v2.DLKA06_7.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 54.984 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.009312 \| USD \| per_test \| Lower is better \| \| `latency` \| 4.641 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.74 pp sourcePopulation 54 stderr 0.742 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_case_law_v2.DLKA06_7.json) Source Vals AI Original model label google/gemini-3.1-flash-lite-preview Source version 2 Observed 2026-10-22T02:10:09.895Z Evidence SHA-256 `293f388f7e0fe563287d4cd19beaa7ef9821b32795dace34bd0197ed47a592d4` Source locator `/tasks/overall/google~1gemini-3.1-flash-lite-preview` | | | | | | | | | [AIME 2025](https://benchize.com/benchmarks/aime-2025.md) | 78.33% | SE ±2.18 pp | 7 / 14 | $0.00744 | 42.97source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_aime.C3_jItdR.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 78.333 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.007442 \| USD \| per_test \| Lower is better \| \| `latency` \| 42.968 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens Not reported provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.18 pp sourcePopulation 96 stderr 2.182 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_aime.C3_jItdR.json) Source Vals AI Original model label google/gemini-3.1-flash-lite-preview Source version 1 Observed 2026-10-22T02:10:17.157Z Evidence SHA-256 `7d016210b39d24351bbb5d12be09a973decac461bb2486eb1f23724fed2b7e6c` Source locator `/tasks/aime_2025/google~1gemini-3.1-flash-lite-preview` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / capital markets](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acapital_markets.md) | 0% | SE ±0 pp | 16 / 49 | $0.04 | 35.43source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.043334 \| USD \| per_test \| Lower is better \| \| `latency` \| 35.432 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 81 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label google/gemini-3.1-flash-lite-preview Source version 1 Observed 2026-10-22T02:10:09.422Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/capital_markets/google~1gemini-3.1-flash-lite-preview` | | | | | | | | | [EMB / overall](https://benchize.com/benchmarks/vals-ai%3Aemb%3Aoverall.md) | 8.63% | SE ±1.09 pp | 47 / 47 | $0.07 | 67.01source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 8.629 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.069424 \| USD \| per_test \| Lower is better \| \| `latency` \| 67.007 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.09 pp sourcePopulation 76 stderr 1.088 tokenTotals {"cache_read_tokens":62784855,"cache_write_tokens":null,"input_tokens":77708032,"output_tokens":1187254,"reasoning_tokens":588837} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) Source Vals AI Original model label google/gemini-3.1-flash-lite-preview Source version 1 Observed 2026-10-22T02:10:12.907Z Evidence SHA-256 `d807aee620ea47152a1fca0257a8dd12a1be4512c76cc1744c1ae9e285180918` Source locator `/tasks/overall/google~1gemini-3.1-flash-lite-preview` | | | | | | | | | [Vals Multimodal Index / sage](https://benchize.com/benchmarks/vals-ai%3Avals_multimodal_index%3Asage.md) | 49.54% | SE ±3.48 pp | 9 / 20 | $0.00773 | 16.26source duration | [Vals AI · 1.2](https://www.vals.ai/_astro/benchmark_view_vals_multimodal_index.virVpWMs.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 49.54 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.00773 \| USD \| per_test \| Lower is better \| \| `latency` \| 16.257 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±3.48 pp sourcePopulation 33 stderr 3.484 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_vals_multimodal_index.virVpWMs.json) Source Vals AI Original model label google/gemini-3.1-flash-lite-preview Source version 1.2 Observed 2026-10-22T02:10:07.406Z Evidence SHA-256 `077b22575f6e075b7b78157d000ec8f94d7ead409c7dcbcf04887630ed572a2d` Source locator `/tasks/sage/google~1gemini-3.1-flash-lite-preview` | | | | | | | | | [EMB / comps](https://benchize.com/benchmarks/vals-ai%3Aemb%3Acomps.md) | 10.48% | SE ±3.48 pp | 47 / 47 | $0.18 | 101.86source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 10.48 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.182456 \| USD \| per_test \| Lower is better \| \| `latency` \| 101.863 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±3.48 pp sourcePopulation 76 stderr 3.475 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) Source Vals AI Original model label google/gemini-3.1-flash-lite-preview Source version 1 Observed 2026-10-22T02:10:12.907Z Evidence SHA-256 `d807aee620ea47152a1fca0257a8dd12a1be4512c76cc1744c1ae9e285180918` Source locator `/tasks/comps/google~1gemini-3.1-flash-lite-preview` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / criteria pass rate funds asset management](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__funds_asset_management.md) | 22.45% | SE ±2.26 pp | 49 / 49 | $0.14 | 81.95source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 22.449 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.142138 \| USD \| per_test \| Lower is better \| \| `latency` \| 81.95 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.26 pp sourcePopulation 81 stderr 2.256 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label google/gemini-3.1-flash-lite-preview Source version 1 Observed 2026-10-22T02:10:09.422Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__funds_asset_management/google~1gemini-3.1-flash-lite-preview` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / criteria pass rate insurance](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__insurance.md) | 23.81% | SE ±2.58 pp | 49 / 49 | $0.09 | 73.88source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 23.81 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.087874 \| USD \| per_test \| Lower is better \| \| `latency` \| 73.878 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.58 pp sourcePopulation 81 stderr 2.583 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label google/gemini-3.1-flash-lite-preview Source version 1 Observed 2026-10-22T02:10:09.422Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__insurance/google~1gemini-3.1-flash-lite-preview` | | | | | | | | | [EMB / dataroom](https://benchize.com/benchmarks/vals-ai%3Aemb%3Adataroom.md) | 15.18% | SE ±5.15 pp | 46 / 47 | $0.05 | 84.27source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 15.178 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.050422 \| USD \| per_test \| Lower is better \| \| `latency` \| 84.269 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±5.15 pp sourcePopulation 76 stderr 5.151 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) Source Vals AI Original model label google/gemini-3.1-flash-lite-preview Source version 1 Observed 2026-10-22T02:10:12.907Z Evidence SHA-256 `d807aee620ea47152a1fca0257a8dd12a1be4512c76cc1744c1ae9e285180918` Source locator `/tasks/dataroom/google~1gemini-3.1-flash-lite-preview` | | | | | | | | | [AIME / overall](https://benchize.com/benchmarks/vals-ai%3Aaime%3Aoverall.md) | 83.33% | SE ±1.18 pp | 7 / 14 | $0.00697 | 40.56source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_aime.C3_jItdR.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 83.333 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.006966 \| USD \| per_test \| Lower is better \| \| `latency` \| 40.564 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens Not reported provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.18 pp sourcePopulation 96 stderr 1.178 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_aime.C3_jItdR.json) Source Vals AI Original model label google/gemini-3.1-flash-lite-preview Source version 1 Observed 2026-10-22T02:10:17.157Z Evidence SHA-256 `7d016210b39d24351bbb5d12be09a973decac461bb2486eb1f23724fed2b7e6c` Source locator `/tasks/overall/google~1gemini-3.1-flash-lite-preview` | | | | | | | | | [Terminal-Bench 2.1 / easy](https://benchize.com/benchmarks/vals-ai%3Aterminal-bench-2-1%3Aeasy.md) | 66.67% | SE ±8.33 pp | 37 / 43 | $1.89 | 6,193.12source duration | [Vals AI · 2.1](https://www.vals.ai/_astro/benchmark_view_terminal_bench_2_1.CdyoObsA.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 66.667 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 1.888892 \| USD \| per_test \| Lower is better \| \| `latency` \| 6193.118 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±8.33 pp sourcePopulation 76 stderr 8.333 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_terminal_bench_2_1.CdyoObsA.json) Source Vals AI Original model label google/gemini-3.1-flash-lite-preview Source version 2.1 Observed 2026-10-22T02:10:27.811Z Evidence SHA-256 `a5306805cdf86495bdcceff6b2c8534a3768b9853f5077895dab975b809b054f` Source locator `/tasks/easy/google~1gemini-3.1-flash-lite-preview` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / international trade sanctions](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Ainternational_trade_sanctions.md) | 0% | SE ±0 pp | 4 / 49 | $0.05 | 47.57source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.051418 \| USD \| per_test \| Lower is better \| \| `latency` \| 47.572 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 81 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label google/gemini-3.1-flash-lite-preview Source version 1 Observed 2026-10-22T02:10:09.422Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/international_trade_sanctions/google~1gemini-3.1-flash-lite-preview` | | | | | | | | | [TaxEval v2 / correctness](https://benchize.com/benchmarks/vals-ai%3Atax_eval_v2%3Acorrectness.md) | 60.43% | SE ±1.4 pp | 30 / 40 | $0.00096 | 7.64source duration | [Vals AI · 2](https://www.vals.ai/_astro/benchmark_view_tax_eval_v2.B3yoffr8.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 60.425 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.00096 \| USD \| per_test \| Lower is better \| \| `latency` \| 7.644 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.4 pp sourcePopulation 145 stderr 1.398 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_tax_eval_v2.B3yoffr8.json) Source Vals AI Original model label google/gemini-3.1-flash-lite-preview Source version 2 Observed 2026-10-22T02:10:14.414Z Evidence SHA-256 `8b354a6ae848f5bb25a2e8b061b5f576d5b9e0c11b941dc1e6bed078e6529cd6` Source locator `/tasks/correctness/google~1gemini-3.1-flash-lite-preview` | | | | | | | | | [Finance Agent (v2) / market analysis](https://benchize.com/benchmarks/vals-ai%3Afabv2%3Amarket_analysis.md) | 34.19% | SE ±1.17 pp | 46 / 50 | $0.09 | 39.44source duration | [Vals AI · 2](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 34.189 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.094503 \| USD \| per_test \| Lower is better \| \| `latency` \| 39.442 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.17 pp sourcePopulation 80 stderr 1.174 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) Source Vals AI Original model label google/gemini-3.1-flash-lite-preview Source version 2 Observed 2026-10-22T02:10:12.404Z Evidence SHA-256 `f731b20a26c126725d125519f2b5841aa45254656dba859a9570d7ce7e5d8191` Source locator `/tasks/market_analysis/google~1gemini-3.1-flash-lite-preview` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / litigation dispute resolution](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Alitigation_dispute_resolution.md) | 0% | SE ±0 pp | 6 / 49 | $0.08 | 57.23source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.081949 \| USD \| per_test \| Lower is better \| \| `latency` \| 57.228 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 81 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label google/gemini-3.1-flash-lite-preview Source version 1 Observed 2026-10-22T02:10:09.422Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/litigation_dispute_resolution/google~1gemini-3.1-flash-lite-preview` | | | | | | | | | [MedScribe / overall](https://benchize.com/benchmarks/vals-ai%3Amedscribe%3Aoverall.md) | 63.9% | SE ±1.82 pp | 46 / 49 | $0.0022 | 16.79source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_medscribe.BYf5a3C-.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 63.902 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.002195 \| USD \| per_test \| Lower is better \| \| `latency` \| 16.793 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 30000 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.82 pp sourcePopulation 109 stderr 1.823 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_medscribe.BYf5a3C-.json) Source Vals AI Original model label google/gemini-3.1-flash-lite-preview Source version 1 Observed 2026-10-22T02:10:15.398Z Evidence SHA-256 `271fc122ddc2b0602bd6c7947f65a9a7f0d8cab94f6dc27d62a9380a9317f9c0` Source locator `/tasks/overall/google~1gemini-3.1-flash-lite-preview` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / data privacy cybersecurity](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Adata_privacy_cybersecurity.md) | 0% | SE ±0 pp | 13 / 49 | $0.06 | 78.59source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.057493 \| USD \| per_test \| Lower is better \| \| `latency` \| 78.586 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 81 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label google/gemini-3.1-flash-lite-preview Source version 1 Observed 2026-10-22T02:10:09.422Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/data_privacy_cybersecurity/google~1gemini-3.1-flash-lite-preview` | | | | | | | | | [Finance Agent (v2) / precedents](https://benchize.com/benchmarks/vals-ai%3Afabv2%3Aprecedents.md) | 11.81% | SE ±0.74 pp | 46 / 50 | $0.14 | 60.88source duration | [Vals AI · 2](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 11.808 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.143796 \| USD \| per_test \| Lower is better \| \| `latency` \| 60.876 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.74 pp sourcePopulation 80 stderr 0.741 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) Source Vals AI Original model label google/gemini-3.1-flash-lite-preview Source version 2 Observed 2026-10-22T02:10:12.404Z Evidence SHA-256 `f731b20a26c126725d125519f2b5841aa45254656dba859a9570d7ce7e5d8191` Source locator `/tasks/precedents/google~1gemini-3.1-flash-lite-preview` | | | | | | | | | [SWE-bench / 1-4 hours](https://benchize.com/benchmarks/vals-ai%3Aswebench%3A1-4%20hours.md) | 21.43% | SE ±6.41 pp | 36 / 39 | $0.13 | 156.31source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_swebench.Ae8ni7h0.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 21.429 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.132103 \| USD \| per_test \| Lower is better \| \| `latency` \| 156.311 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±6.41 pp sourcePopulation 88 stderr 6.408 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_swebench.Ae8ni7h0.json) Source Vals AI Original model label google/gemini-3.1-flash-lite-preview Source version 1 Observed 2026-10-22T02:10:28.317Z Evidence SHA-256 `7a4e2e8485283caac500127419f68ec4bd5414c837ade114fc6afa5b2d34ccaa` Source locator `/tasks/1-4 hours/google~1gemini-3.1-flash-lite-preview` | | | | | | | | | [Tax Agent Bench / rule source lookup](https://benchize.com/benchmarks/vals-ai%3Atax_agent_bench%3Arule_source_lookup.md) | 3.33% | SE ±3.33 pp | 45 / 46 | $0.01 | 18.57source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 3.333 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.010765 \| USD \| per_test \| Lower is better \| \| `latency` \| 18.573 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±3.33 pp sourcePopulation 73 stderr 3.333 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) Source Vals AI Original model label google/gemini-3.1-flash-lite-preview Source version 1 Observed 2026-10-22T02:10:11.407Z Evidence SHA-256 `446db182b67efb7923b3187a4040ec3a3ea492b8ad699a2b6d708d714cc063ac` Source locator `/tasks/rule_source_lookup/google~1gemini-3.1-flash-lite-preview` | | | | | | | | | [MortgageTax / overall](https://benchize.com/benchmarks/vals-ai%3Amortgage_tax%3Aoverall.md) | 68.04% | SE ±0.91 pp | 14 / 31 | $0.00284 | 5.37source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_mortgage_tax.DCd-oO5Y.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 68.044 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.00284 \| USD \| per_test \| Lower is better \| \| `latency` \| 5.368 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.91 pp sourcePopulation 98 stderr 0.912 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_mortgage_tax.DCd-oO5Y.json) Source Vals AI Original model label google/gemini-3.1-flash-lite-preview Source version 1 Observed 2026-10-22T02:10:13.924Z Evidence SHA-256 `e5c844561f97d119d66d70269e726ecfb68be79eae39acf14ead3e95422e648c` Source locator `/tasks/overall/google~1gemini-3.1-flash-lite-preview` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / criteria pass rate energy natural resources](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__energy_natural_resources.md) | 38.28% | SE ±2.8 pp | 43 / 49 | $0.21 | 87.67source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 38.284 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.205085 \| USD \| per_test \| Lower is better \| \| `latency` \| 87.666 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.8 pp sourcePopulation 81 stderr 2.797 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label google/gemini-3.1-flash-lite-preview Source version 1 Observed 2026-10-22T02:10:09.422Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__energy_natural_resources/google~1gemini-3.1-flash-lite-preview` | | | | | | | | | [Legal Research Bench / civil litigation](https://benchize.com/benchmarks/vals-ai%3Alegal_research%3Acivil_litigation.md) | 3.7% | SE ±3.7 pp | 47 / 50 | $0.01 | 19.79source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 3.704 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.013273 \| USD \| per_test \| Lower is better \| \| `latency` \| 19.793 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±3.7 pp sourcePopulation 82 stderr 3.704 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) Source Vals AI Original model label google/gemini-3.1-flash-lite-preview Source version 1 Observed 2026-10-22T02:10:07.934Z Evidence SHA-256 `26c58dfd47d0d2dc86f33c6d056f30309e0ee63570e7cd752c593b385f9d862e` Source locator `/tasks/civil_litigation/google~1gemini-3.1-flash-lite-preview` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / tax](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Atax.md) | 0% | SE ±0 pp | 3 / 49 | $0.09 | 61.97source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.08909 \| USD \| per_test \| Lower is better \| \| `latency` \| 61.968 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 81 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label google/gemini-3.1-flash-lite-preview Source version 1 Observed 2026-10-22T02:10:09.422Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/tax/google~1gemini-3.1-flash-lite-preview` | | | | | | | | | [MortgageTax / numerical extraction](https://benchize.com/benchmarks/vals-ai%3Amortgage_tax%3Anumerical_extraction.md) | 77.03% | SE ±1.19 pp | 9 / 31 | $0.00323 | 6.32source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_mortgage_tax.DCd-oO5Y.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 77.027 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.003233 \| USD \| per_test \| Lower is better \| \| `latency` \| 6.323 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.19 pp sourcePopulation 98 stderr 1.186 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_mortgage_tax.DCd-oO5Y.json) Source Vals AI Original model label google/gemini-3.1-flash-lite-preview Source version 1 Observed 2026-10-22T02:10:13.924Z Evidence SHA-256 `e5c844561f97d119d66d70269e726ecfb68be79eae39acf14ead3e95422e648c` Source locator `/tasks/numerical_extraction/google~1gemini-3.1-flash-lite-preview` | | | | | | | | | [Code Migration / cli](https://benchize.com/benchmarks/vals-ai%3Acode-migration%3Acli.md) | 5.73% | SE ±1.36 pp | 45 / 47 | $0.05 | 87.66source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_code_migration.Dttzoqk3.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 5.729 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.052306 \| USD \| per_test \| Lower is better \| \| `latency` \| 87.657 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.36 pp sourcePopulation 79 stderr 1.363 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_code_migration.Dttzoqk3.json) Source Vals AI Original model label google/gemini-3.1-flash-lite-preview Source version 1 Observed 2026-10-22T02:10:27.307Z Evidence SHA-256 `99194597f2c76c402195be47839409046e18bd30454616da6dd40b736a088fa4` Source locator `/tasks/cli/google~1gemini-3.1-flash-lite-preview` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / antitrust competition](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Aantitrust_competition.md) | 0% | SE ±0 pp | 7 / 49 | $0.07 | 62.2source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.065573 \| USD \| per_test \| Lower is better \| \| `latency` \| 62.2 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 81 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label google/gemini-3.1-flash-lite-preview Source version 1 Observed 2026-10-22T02:10:09.422Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/antitrust_competition/google~1gemini-3.1-flash-lite-preview` | | | | | | | | | [Finance Agent (v2) / financial modeling](https://benchize.com/benchmarks/vals-ai%3Afabv2%3Afinancial_modeling.md) | 4.11% | SE ±1.21 pp | 45 / 50 | $0.14 | 60.91source duration | [Vals AI · 2](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 4.112 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.141152 \| USD \| per_test \| Lower is better \| \| `latency` \| 60.912 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.21 pp sourcePopulation 80 stderr 1.206 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) Source Vals AI Original model label google/gemini-3.1-flash-lite-preview Source version 2 Observed 2026-10-22T02:10:12.404Z Evidence SHA-256 `f731b20a26c126725d125519f2b5841aa45254656dba859a9570d7ce7e5d8191` Source locator `/tasks/financial_modeling/google~1gemini-3.1-flash-lite-preview` | | | | | | | | | [ProgramBench / almost](https://benchize.com/benchmarks/vals-ai%3Aprogrambench%3Aalmost.md) | 0% | SE ±0 pp | 34 / 40 | $0.06 | 176.25source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_programbench.C_J7ZU4X.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.063906 \| USD \| per_test \| Lower is better \| \| `latency` \| 176.246 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 64 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_programbench.C_J7ZU4X.json) Source Vals AI Original model label google/gemini-3.1-flash-lite-preview Source version 1 Observed 2026-10-22T02:10:24.817Z Evidence SHA-256 `e9bbb4378c7a058441a21d456baf9316b96c2cf5b58d371a3c7c803f5000bd8c` Source locator `/tasks/almost/google~1gemini-3.1-flash-lite-preview` | | | | | | | | | [EMB / dcf](https://benchize.com/benchmarks/vals-ai%3Aemb%3Adcf.md) | 5.1% | SE ±1.01 pp | 45 / 47 | $0.05 | 64.7source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 5.098 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.046823 \| USD \| per_test \| Lower is better \| \| `latency` \| 64.7 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.01 pp sourcePopulation 76 stderr 1.006 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) Source Vals AI Original model label google/gemini-3.1-flash-lite-preview Source version 1 Observed 2026-10-22T02:10:12.907Z Evidence SHA-256 `d807aee620ea47152a1fca0257a8dd12a1be4512c76cc1744c1ae9e285180918` Source locator `/tasks/dcf/google~1gemini-3.1-flash-lite-preview` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / corporate ma](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acorporate_ma.md) | 0% | SE ±0 pp | 21 / 49 | $0.09 | 68.16source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.091726 \| USD \| per_test \| Lower is better \| \| `latency` \| 68.164 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 81 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label google/gemini-3.1-flash-lite-preview Source version 1 Observed 2026-10-22T02:10:09.422Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/corporate_ma/google~1gemini-3.1-flash-lite-preview` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / criteria pass rate real estate](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__real_estate.md) | 29.45% | SE ±2.47 pp | 49 / 49 | $0.21 | 82.09source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 29.446 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.20807 \| USD \| per_test \| Lower is better \| \| `latency` \| 82.088 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.47 pp sourcePopulation 81 stderr 2.465 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label google/gemini-3.1-flash-lite-preview Source version 1 Observed 2026-10-22T02:10:09.422Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__real_estate/google~1gemini-3.1-flash-lite-preview` | | | | | | | | | [Vals Multimodal Index / finance agent](https://benchize.com/benchmarks/vals-ai%3Avals_multimodal_index%3Afinance_agent.md) | 30.07% | SE ±0.73 pp | 20 / 20 | $0.13 | 53.58source duration | [Vals AI · 1.2](https://www.vals.ai/_astro/benchmark_view_vals_multimodal_index.virVpWMs.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 30.068 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.133469 \| USD \| per_test \| Lower is better \| \| `latency` \| 53.577 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.73 pp sourcePopulation 33 stderr 0.727 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_vals_multimodal_index.virVpWMs.json) Source Vals AI Original model label google/gemini-3.1-flash-lite-preview Source version 1.2 Observed 2026-10-22T02:10:07.406Z Evidence SHA-256 `077b22575f6e075b7b78157d000ec8f94d7ead409c7dcbcf04887630ed572a2d` Source locator `/tasks/finance_agent/google~1gemini-3.1-flash-lite-preview` | | | | | | | | | [EMB / template](https://benchize.com/benchmarks/vals-ai%3Aemb%3Atemplate.md) | 8.14% | SE ±1.8 pp | 45 / 47 | $0.07 | 70.99source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 8.137 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.074608 \| USD \| per_test \| Lower is better \| \| `latency` \| 70.993 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.8 pp sourcePopulation 76 stderr 1.799 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) Source Vals AI Original model label google/gemini-3.1-flash-lite-preview Source version 1 Observed 2026-10-22T02:10:12.907Z Evidence SHA-256 `d807aee620ea47152a1fca0257a8dd12a1be4512c76cc1744c1ae9e285180918` Source locator `/tasks/template/google~1gemini-3.1-flash-lite-preview` | | | | | | | | | [SWE-bench / <15 min fix](https://benchize.com/benchmarks/vals-ai%3Aswebench%3A%3C15%20min%20fix.md) | 76.8% | SE ±3.04 pp | 34 / 39 | $0.09 | 112.63source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_swebench.Ae8ni7h0.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 76.804 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.087251 \| USD \| per_test \| Lower is better \| \| `latency` \| 112.63 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±3.04 pp sourcePopulation 88 stderr 3.038 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_swebench.Ae8ni7h0.json) Source Vals AI Original model label google/gemini-3.1-flash-lite-preview Source version 1 Observed 2026-10-22T02:10:28.317Z Evidence SHA-256 `7a4e2e8485283caac500127419f68ec4bd5414c837ade114fc6afa5b2d34ccaa` Source locator `/tasks/<15 min fix/google~1gemini-3.1-flash-lite-preview` | | | | | | | | | [CorpFin v2 / overall](https://benchize.com/benchmarks/vals-ai%3Acorp_fin_v2%3Aoverall.md) | 59.36% | SE ±0.97 pp | 29 / 39 | $0.02 | 4.03source duration | [Vals AI · 2](https://www.vals.ai/_astro/benchmark_view_corp_fin_v2.CectKwl1.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 59.363 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.022927 \| USD \| per_test \| Lower is better \| \| `latency` \| 4.03 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.97 pp sourcePopulation 134 stderr 0.967 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_corp_fin_v2.CectKwl1.json) Source Vals AI Original model label google/gemini-3.1-flash-lite-preview Source version 2 Observed 2026-10-22T02:10:13.429Z Evidence SHA-256 `7cf2217ab3728d5a84bffce82aab9fc52d7f810b0cbe8c65b387df47f3d31c9b` Source locator `/tasks/overall/google~1gemini-3.1-flash-lite-preview` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / criteria pass rate corporate ma](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__corporate_ma.md) | 32.37% | SE ±2.65 pp | 49 / 49 | $0.09 | 68.16source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 32.372 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.091726 \| USD \| per_test \| Lower is better \| \| `latency` \| 68.164 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.65 pp sourcePopulation 81 stderr 2.653 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label google/gemini-3.1-flash-lite-preview Source version 1 Observed 2026-10-22T02:10:09.422Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__corporate_ma/google~1gemini-3.1-flash-lite-preview` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / intellectual property](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Aintellectual_property.md) | 0% | SE ±0 pp | 6 / 49 | $0.09 | 65.61source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.092629 \| USD \| per_test \| Lower is better \| \| `latency` \| 65.61 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 81 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label google/gemini-3.1-flash-lite-preview Source version 1 Observed 2026-10-22T02:10:09.422Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/intellectual_property/google~1gemini-3.1-flash-lite-preview` | | | | | | | | | [SWE-bench / overall](https://benchize.com/benchmarks/vals-ai%3Aswebench%3Aoverall.md) | 62.8% | SE ±2.16 pp | 33 / 39 | $0.11 | 132.94source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_swebench.Ae8ni7h0.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 62.8 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.105389 \| USD \| per_test \| Lower is better \| \| `latency` \| 132.936 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.16 pp sourcePopulation 88 stderr 2.164 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_swebench.Ae8ni7h0.json) Source Vals AI Original model label google/gemini-3.1-flash-lite-preview Source version 1 Observed 2026-10-22T02:10:28.317Z Evidence SHA-256 `7a4e2e8485283caac500127419f68ec4bd5414c837ade114fc6afa5b2d34ccaa` Source locator `/tasks/overall/google~1gemini-3.1-flash-lite-preview` | | | | | | | | | [Terminal-Bench 2.0 / easy](https://benchize.com/benchmarks/vals-ai%3Aterminal-bench-2%3Aeasy.md) | 50% | SE ±28.87 pp | 17 / 23 | $0.46 | 14,860.31source duration | [Vals AI · 2.0](https://www.vals.ai/_astro/benchmark_view_terminal_bench_2.T48tPR4O.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 50 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.46031 \| USD \| per_test \| Lower is better \| \| `latency` \| 14860.307 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±28.87 pp sourcePopulation 67 stderr 28.868 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_terminal_bench_2.T48tPR4O.json) Source Vals AI Original model label google/gemini-3.1-flash-lite-preview Source version 2.0 Observed 2026-10-22T02:10:29.313Z Evidence SHA-256 `38e640d995259778e531f048b2f2c7b3b281cb05b56be643ddc9dc53eb85adf7` Source locator `/tasks/easy/google~1gemini-3.1-flash-lite-preview` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / healthcare life sciences](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Ahealthcare_life_sciences.md) | 0% | SE ±0 pp | 24 / 49 | $0.05 | 46.9source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.049241 \| USD \| per_test \| Lower is better \| \| `latency` \| 46.904 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 81 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label google/gemini-3.1-flash-lite-preview Source version 1 Observed 2026-10-22T02:10:09.422Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/healthcare_life_sciences/google~1gemini-3.1-flash-lite-preview` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / criteria pass rate corporate governance](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__corporate_governance.md) | 29.8% | SE ±2.68 pp | 49 / 49 | $0.05 | 45.13source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 29.795 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.053496 \| USD \| per_test \| Lower is better \| \| `latency` \| 45.132 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.68 pp sourcePopulation 81 stderr 2.681 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label google/gemini-3.1-flash-lite-preview Source version 1 Observed 2026-10-22T02:10:09.422Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__corporate_governance/google~1gemini-3.1-flash-lite-preview` | | | | | | | | | [EMB / lbo](https://benchize.com/benchmarks/vals-ai%3Aemb%3Albo.md) | 8.81% | SE ±2.18 pp | 45 / 47 | $0.04 | 49.22source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 8.805 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.037241 \| USD \| per_test \| Lower is better \| \| `latency` \| 49.222 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.18 pp sourcePopulation 76 stderr 2.184 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) Source Vals AI Original model label google/gemini-3.1-flash-lite-preview Source version 1 Observed 2026-10-22T02:10:12.907Z Evidence SHA-256 `d807aee620ea47152a1fca0257a8dd12a1be4512c76cc1744c1ae9e285180918` Source locator `/tasks/lbo/google~1gemini-3.1-flash-lite-preview` | | | | | | | | | [Vals Multimodal Index / mortgage tax](https://benchize.com/benchmarks/vals-ai%3Avals_multimodal_index%3Amortgage_tax.md) | 68.04% | SE ±0.91 pp | 11 / 20 | $0.00284 | 5.37source duration | [Vals AI · 1.2](https://www.vals.ai/_astro/benchmark_view_vals_multimodal_index.virVpWMs.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 68.044 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.00284 \| USD \| per_test \| Lower is better \| \| `latency` \| 5.368 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.91 pp sourcePopulation 33 stderr 0.912 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_vals_multimodal_index.virVpWMs.json) Source Vals AI Original model label google/gemini-3.1-flash-lite-preview Source version 1.2 Observed 2026-10-22T02:10:07.406Z Evidence SHA-256 `077b22575f6e075b7b78157d000ec8f94d7ead409c7dcbcf04887630ed572a2d` Source locator `/tasks/mortgage_tax/google~1gemini-3.1-flash-lite-preview` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / emerging companies venture capital](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Aemerging_companies_venture_capital.md) | 0% | SE ±0 pp | 18 / 49 | $0.09 | 61.06source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.085625 \| USD \| per_test \| Lower is better \| \| `latency` \| 61.06 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 81 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label google/gemini-3.1-flash-lite-preview Source version 1 Observed 2026-10-22T02:10:09.422Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/emerging_companies_venture_capital/google~1gemini-3.1-flash-lite-preview` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / criteria pass rate emerging companies venture capital](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__emerging_companies_venture_capital.md) | 35.84% | SE ±2.81 pp | 48 / 49 | $0.09 | 61.06source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 35.836 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.085625 \| USD \| per_test \| Lower is better \| \| `latency` \| 61.06 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.81 pp sourcePopulation 81 stderr 2.806 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label google/gemini-3.1-flash-lite-preview Source version 1 Observed 2026-10-22T02:10:09.422Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__emerging_companies_venture_capital/google~1gemini-3.1-flash-lite-preview` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / criteria pass rate antitrust competition](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__antitrust_competition.md) | 23.91% | SE ±2.57 pp | 49 / 49 | $0.07 | 62.2source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 23.913 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.065573 \| USD \| per_test \| Lower is better \| \| `latency` \| 62.2 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.57 pp sourcePopulation 81 stderr 2.572 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label google/gemini-3.1-flash-lite-preview Source version 1 Observed 2026-10-22T02:10:09.422Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__antitrust_competition/google~1gemini-3.1-flash-lite-preview` | | | | | | | | | [SAGE / linear algebra](https://benchize.com/benchmarks/vals-ai%3Asage%3Alinear_algebra.md) | 58.94% | SE ±5.55 pp | 32 / 41 | $0.00762 | 15.66source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_sage.RRwIIzO5.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 58.936 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.007622 \| USD \| per_test \| Lower is better \| \| `latency` \| 15.659 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 30000 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±5.55 pp sourcePopulation 91 stderr 5.549 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_sage.RRwIIzO5.json) Source Vals AI Original model label google/gemini-3.1-flash-lite-preview Source version 1 Observed 2026-10-22T02:10:22.165Z Evidence SHA-256 `cc5fb3fecdb0382d492ff167a42ad84b9c5c3883373c78ed87baaa972b8b6f7c` Source locator `/tasks/linear_algebra/google~1gemini-3.1-flash-lite-preview` | | | | | | | | | [Terminal-Bench 2.0 / medium](https://benchize.com/benchmarks/vals-ai%3Aterminal-bench-2%3Amedium.md) | 34.55% | SE ±6.47 pp | 18 / 23 | $0.03 | 1,080.75source duration | [Vals AI · 2.0](https://www.vals.ai/_astro/benchmark_view_terminal_bench_2.T48tPR4O.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 34.545 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.033477 \| USD \| per_test \| Lower is better \| \| `latency` \| 1080.75 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±6.47 pp sourcePopulation 67 stderr 6.471 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_terminal_bench_2.T48tPR4O.json) Source Vals AI Original model label google/gemini-3.1-flash-lite-preview Source version 2.0 Observed 2026-10-22T02:10:29.313Z Evidence SHA-256 `38e640d995259778e531f048b2f2c7b3b281cb05b56be643ddc9dc53eb85adf7` Source locator `/tasks/medium/google~1gemini-3.1-flash-lite-preview` | | | | | | | | | [Legal Research Bench / health](https://benchize.com/benchmarks/vals-ai%3Alegal_research%3Ahealth.md) | 6.67% | SE ±4.63 pp | 45 / 50 | $0.01 | 19.51source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 6.667 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.013992 \| USD \| per_test \| Lower is better \| \| `latency` \| 19.508 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±4.63 pp sourcePopulation 82 stderr 4.632 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) Source Vals AI Original model label google/gemini-3.1-flash-lite-preview Source version 1 Observed 2026-10-22T02:10:07.934Z Evidence SHA-256 `26c58dfd47d0d2dc86f33c6d056f30309e0ee63570e7cd752c593b385f9d862e` Source locator `/tasks/health/google~1gemini-3.1-flash-lite-preview` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / criteria pass rate white collar defense investigations](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__white_collar_defense_investigations.md) | 27.95% | SE ±2.61 pp | 49 / 49 | $0.07 | 47.61source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 27.946 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.072372 \| USD \| per_test \| Lower is better \| \| `latency` \| 47.606 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.61 pp sourcePopulation 81 stderr 2.608 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label google/gemini-3.1-flash-lite-preview Source version 1 Observed 2026-10-22T02:10:09.422Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__white_collar_defense_investigations/google~1gemini-3.1-flash-lite-preview` | | | | | | | | | [Code Migration / code quality](https://benchize.com/benchmarks/vals-ai%3Acode-migration%3Acode_quality.md) | 2.66% | SE ±0.04 pp | 43 / 47 | $0.05 | 92.63source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_code_migration.Dttzoqk3.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 2.664 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.053028 \| USD \| per_test \| Lower is better \| \| `latency` \| 92.63 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.04 pp sourcePopulation 79 stderr 0.038 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_code_migration.Dttzoqk3.json) Source Vals AI Original model label google/gemini-3.1-flash-lite-preview Source version 1 Observed 2026-10-22T02:10:27.307Z Evidence SHA-256 `99194597f2c76c402195be47839409046e18bd30454616da6dd40b736a088fa4` Source locator `/tasks/code_quality/google~1gemini-3.1-flash-lite-preview` | | | | | | | | | [Finance Agent (v2) / all pass](https://benchize.com/benchmarks/vals-ai%3Afabv2%3Aall_pass.md) | 19.3% | SE ±0.53 pp | 45 / 50 | $0.14 | 54.78source duration | [Vals AI · 2](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 19.303 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.142482 \| USD \| per_test \| Lower is better \| \| `latency` \| 54.778 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.53 pp sourcePopulation 80 stderr 0.526 tokenTotals {"cache_read_tokens":88769598,"cache_write_tokens":null,"input_tokens":309249776,"output_tokens":4518540,"reasoning_tokens":3033245} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) Source Vals AI Original model label google/gemini-3.1-flash-lite-preview Source version 2 Observed 2026-10-22T02:10:12.404Z Evidence SHA-256 `f731b20a26c126725d125519f2b5841aa45254656dba859a9570d7ce7e5d8191` Source locator `/tasks/all_pass/google~1gemini-3.1-flash-lite-preview` | | | | | | | | | [Legal Research Bench / family](https://benchize.com/benchmarks/vals-ai%3Alegal_research%3Afamily.md) | 0% | SE ±0 pp | 44 / 50 | $0.01 | 18.54source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.013412 \| USD \| per_test \| Lower is better \| \| `latency` \| 18.535 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 82 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) Source Vals AI Original model label google/gemini-3.1-flash-lite-preview Source version 1 Observed 2026-10-22T02:10:07.934Z Evidence SHA-256 `26c58dfd47d0d2dc86f33c6d056f30309e0ee63570e7cd752c593b385f9d862e` Source locator `/tasks/family/google~1gemini-3.1-flash-lite-preview` | | | | | | | | | [Tax Agent Bench / fact pattern analysis](https://benchize.com/benchmarks/vals-ai%3Atax_agent_bench%3Afact_pattern_analysis.md) | 4.65% | SE ±3.25 pp | 42 / 46 | $0.02 | 23.47source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 4.651 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.015379 \| USD \| per_test \| Lower is better \| \| `latency` \| 23.467 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±3.25 pp sourcePopulation 73 stderr 3.249 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) Source Vals AI Original model label google/gemini-3.1-flash-lite-preview Source version 1 Observed 2026-10-22T02:10:11.407Z Evidence SHA-256 `446db182b67efb7923b3187a4040ec3a3ea492b8ad699a2b6d708d714cc063ac` Source locator `/tasks/fact_pattern_analysis/google~1gemini-3.1-flash-lite-preview` | | | | | | | | | [LegalBench / rhetoric tasks](https://benchize.com/benchmarks/vals-ai%3Alegal_bench%3Arhetoric_tasks.md) | 80.51% | SE ±0.83 pp | 23 / 44 | $0.000262 | 2.35source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_bench.D6uTgA3b.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 80.514 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.000262 \| USD \| per_test \| Lower is better \| \| `latency` \| 2.351 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.83 pp sourcePopulation 151 stderr 0.833 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_bench.D6uTgA3b.json) Source Vals AI Original model label google/gemini-3.1-flash-lite-preview Source version 1 Observed 2026-10-22T02:10:08.909Z Evidence SHA-256 `c8486ba756baa5cd3955d035edff67440e8db23e69d62ad1a9e18d15b8c4bc52` Source locator `/tasks/rhetoric_tasks/google~1gemini-3.1-flash-lite-preview` | | | | | | | | | [SAGE / calculus](https://benchize.com/benchmarks/vals-ai%3Asage%3Acalculus.md) | 58.33% | SE ±6.1 pp | 27 / 36 | $0.00219 | 19.83source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_sage.RRwIIzO5.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 58.333 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.002188 \| USD \| per_test \| Lower is better \| \| `latency` \| 19.833 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 30000 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±6.1 pp sourcePopulation 91 stderr 6.102 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_sage.RRwIIzO5.json) Source Vals AI Original model label google/gemini-3.1-flash-lite-preview Source version 1 Observed 2026-10-22T02:10:22.165Z Evidence SHA-256 `cc5fb3fecdb0382d492ff167a42ad84b9c5c3883373c78ed87baaa972b8b6f7c` Source locator `/tasks/calculus/google~1gemini-3.1-flash-lite-preview` | | | | | | | | | [Vibe Code Bench v1.1 / overall](https://benchize.com/benchmarks/vals-ai%3Avibe-code%3Aoverall.md) | 0% | SE ±0 pp | 58 / 59 | $0.51 | 572.71source duration | [Vals AI · 1.1](https://www.vals.ai/_astro/benchmark_view_vibe_code_bench.DcEBMbCA.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.51 \| USD \| per_test \| Lower is better \| \| `latency` \| 572.711 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness OpenHands maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 116 stderr 0 tokenTotals {"cache_read_tokens":null,"cache_write_tokens":null,"input_tokens":53939702,"output_tokens":1042512,"reasoning_tokens":null} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_vibe_code_bench.DcEBMbCA.json) Source Vals AI Original model label google/gemini-3.1-flash-lite-preview Source version 1.1 Observed 2026-10-22T02:10:23.531Z Evidence SHA-256 `0e8c23f521a0f751c5592952620e197b376cade41c78578461b7aa5740a173e7` Source locator `/tasks/overall/google~1gemini-3.1-flash-lite-preview` | | | | | | | | | [Finance Agent (v2) / overall](https://benchize.com/benchmarks/vals-ai%3Afabv2%3Aoverall.md) | 29.99% | SE ±0.64 pp | 45 / 50 | $0.14 | 54.78source duration | [Vals AI · 2](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 29.988 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.142482 \| USD \| per_test \| Lower is better \| \| `latency` \| 54.778 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.64 pp sourcePopulation 80 stderr 0.643 tokenTotals {"cache_read_tokens":88769598,"cache_write_tokens":null,"input_tokens":309249776,"output_tokens":4518540,"reasoning_tokens":3033245} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) Source Vals AI Original model label google/gemini-3.1-flash-lite-preview Source version 2 Observed 2026-10-22T02:10:12.404Z Evidence SHA-256 `f731b20a26c126725d125519f2b5841aa45254656dba859a9570d7ce7e5d8191` Source locator `/tasks/overall/google~1gemini-3.1-flash-lite-preview` | | | | | | | | | [Terminal-Bench 2.1 / hard](https://benchize.com/benchmarks/vals-ai%3Aterminal-bench-2-1%3Ahard.md) | 15.56% | SE ±1.11 pp | 38 / 43 | $0.25 | 825.75source duration | [Vals AI · 2.1](https://www.vals.ai/_astro/benchmark_view_terminal_bench_2_1.CdyoObsA.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 15.556 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.251852 \| USD \| per_test \| Lower is better \| \| `latency` \| 825.749 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.11 pp sourcePopulation 76 stderr 1.111 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_terminal_bench_2_1.CdyoObsA.json) Source Vals AI Original model label google/gemini-3.1-flash-lite-preview Source version 2.1 Observed 2026-10-22T02:10:27.811Z Evidence SHA-256 `a5306805cdf86495bdcceff6b2c8534a3768b9853f5077895dab975b809b054f` Source locator `/tasks/hard/google~1gemini-3.1-flash-lite-preview` | | | | | | | | | [Tax Agent Bench / numbers calculations](https://benchize.com/benchmarks/vals-ai%3Atax_agent_bench%3Anumbers_calculations.md) | 10% | SE ±5.57 pp | 44 / 46 | $0.01 | 20.93source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 10 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.013865 \| USD \| per_test \| Lower is better \| \| `latency` \| 20.929 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±5.57 pp sourcePopulation 73 stderr 5.571 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) Source Vals AI Original model label google/gemini-3.1-flash-lite-preview Source version 1 Observed 2026-10-22T02:10:11.407Z Evidence SHA-256 `446db182b67efb7923b3187a4040ec3a3ea492b8ad699a2b6d708d714cc063ac` Source locator `/tasks/numbers_calculations/google~1gemini-3.1-flash-lite-preview` | | | | | | | | | [Legal Research Bench / weighted pass administrative regulatory](https://benchize.com/benchmarks/vals-ai%3Alegal_research%3Aweighted_pass__administrative_regulatory.md) | 52.83% | SE ±3.45 pp | 50 / 50 | $0.01 | 18.89source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 52.826 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.013517 \| USD \| per_test \| Lower is better \| \| `latency` \| 18.892 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±3.45 pp sourcePopulation 82 stderr 3.448 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) Source Vals AI Original model label google/gemini-3.1-flash-lite-preview Source version 1 Observed 2026-10-22T02:10:07.934Z Evidence SHA-256 `26c58dfd47d0d2dc86f33c6d056f30309e0ee63570e7cd752c593b385f9d862e` Source locator `/tasks/weighted_pass__administrative_regulatory/google~1gemini-3.1-flash-lite-preview` | | | | | | | | | [Vals Multimodal Index / overall](https://benchize.com/benchmarks/vals-ai%3Avals_multimodal_index%3Aoverall.md) | 41.35% | SE ±0.66 pp | 19 / 20 | $0.12 | 152.43source duration | [Vals AI · 1.2](https://www.vals.ai/_astro/benchmark_view_vals_multimodal_index.virVpWMs.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 41.352 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.123381 \| USD \| per_test \| Lower is better \| \| `latency` \| 152.434 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.66 pp sourcePopulation 33 stderr 0.661 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_vals_multimodal_index.virVpWMs.json) Source Vals AI Original model label google/gemini-3.1-flash-lite-preview Source version 1.2 Observed 2026-10-22T02:10:07.406Z Evidence SHA-256 `077b22575f6e075b7b78157d000ec8f94d7ead409c7dcbcf04887630ed572a2d` Source locator `/tasks/overall/google~1gemini-3.1-flash-lite-preview` | | | | | | | | | [Vals Multimodal Index / terminal bench 2 1](https://benchize.com/benchmarks/vals-ai%3Avals_multimodal_index%3Aterminal_bench_2_1.md) | 34.08% | SE ±0.99 pp | 20 / 20 | $0.08 | 278.34source duration | [Vals AI · 1.2](https://www.vals.ai/_astro/benchmark_view_vals_multimodal_index.virVpWMs.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 34.082 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.084894 \| USD \| per_test \| Lower is better \| \| `latency` \| 278.342 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.99 pp sourcePopulation 33 stderr 0.991 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_vals_multimodal_index.virVpWMs.json) Source Vals AI Original model label google/gemini-3.1-flash-lite-preview Source version 1.2 Observed 2026-10-22T02:10:07.406Z Evidence SHA-256 `077b22575f6e075b7b78157d000ec8f94d7ead409c7dcbcf04887630ed572a2d` Source locator `/tasks/terminal_bench_2_1/google~1gemini-3.1-flash-lite-preview` | | | | | | | | | [EMB / ma](https://benchize.com/benchmarks/vals-ai%3Aemb%3Ama.md) | 6.75% | SE ±1.42 pp | 46 / 47 | $0.03 | 55.63source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 6.751 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.034741 \| USD \| per_test \| Lower is better \| \| `latency` \| 55.632 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.42 pp sourcePopulation 76 stderr 1.416 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) Source Vals AI Original model label google/gemini-3.1-flash-lite-preview Source version 1 Observed 2026-10-22T02:10:12.907Z Evidence SHA-256 `d807aee620ea47152a1fca0257a8dd12a1be4512c76cc1744c1ae9e285180918` Source locator `/tasks/ma/google~1gemini-3.1-flash-lite-preview` | | | | | | | | | [LegalBench / issue tasks](https://benchize.com/benchmarks/vals-ai%3Alegal_bench%3Aissue_tasks.md) | 89.37% | SE ±0.57 pp | 15 / 44 | $0.000329 | 2.39source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_bench.D6uTgA3b.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 89.371 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.000329 \| USD \| per_test \| Lower is better \| \| `latency` \| 2.392 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.57 pp sourcePopulation 151 stderr 0.57 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_bench.D6uTgA3b.json) Source Vals AI Original model label google/gemini-3.1-flash-lite-preview Source version 1 Observed 2026-10-22T02:10:08.909Z Evidence SHA-256 `c8486ba756baa5cd3955d035edff67440e8db23e69d62ad1a9e18d15b8c4bc52` Source locator `/tasks/issue_tasks/google~1gemini-3.1-flash-lite-preview` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / corporate governance](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acorporate_governance.md) | 0% | SE ±0 pp | 11 / 49 | $0.05 | 45.13source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.053496 \| USD \| per_test \| Lower is better \| \| `latency` \| 45.132 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 81 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label google/gemini-3.1-flash-lite-preview Source version 1 Observed 2026-10-22T02:10:09.422Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/corporate_governance/google~1gemini-3.1-flash-lite-preview` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / insurance](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Ainsurance.md) | 0% | SE ±0 pp | 2 / 49 | $0.09 | 73.88source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.087874 \| USD \| per_test \| Lower is better \| \| `latency` \| 73.878 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 81 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label google/gemini-3.1-flash-lite-preview Source version 1 Observed 2026-10-22T02:10:09.422Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/insurance/google~1gemini-3.1-flash-lite-preview` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / employment labor](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Aemployment_labor.md) | 0% | SE ±0 pp | 1 / 49 | $0.05 | 45.71source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.053854 \| USD \| per_test \| Lower is better \| \| `latency` \| 45.712 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 81 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label google/gemini-3.1-flash-lite-preview Source version 1 Observed 2026-10-22T02:10:09.422Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/employment_labor/google~1gemini-3.1-flash-lite-preview` | | | | | | | | | [Tax Agent Bench / weighted pass](https://benchize.com/benchmarks/vals-ai%3Atax_agent_bench%3Aweighted_pass.md) | 20.09% | SE ±2.46 pp | 45 / 46 | $0.01 | 18.74source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 20.085 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.014308 \| USD \| per_test \| Lower is better \| \| `latency` \| 18.744 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.46 pp sourcePopulation 73 stderr 2.457 tokenTotals {"cache_read_tokens":1281568,"cache_write_tokens":null,"input_tokens":6757967,"output_tokens":906826,"reasoning_tokens":487639} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) Source Vals AI Original model label google/gemini-3.1-flash-lite-preview Source version 1 Observed 2026-10-22T02:10:11.407Z Evidence SHA-256 `446db182b67efb7923b3187a4040ec3a3ea492b8ad699a2b6d708d714cc063ac` Source locator `/tasks/weighted_pass/google~1gemini-3.1-flash-lite-preview` | | | | | | | | [Next page](https://benchize.com/models/gemini-3-1-flash-lite-preview-high.md?cursor=9a01e76f30b9709af3c34f75a5de75bd55f6d52714f8b12cff97b0b05c47e24e) --- Cite as: Gemini 3.1 Flash Lite PreviewHigh — Benchize, https://benchize.com/models/gemini-3-1-flash-lite-preview-high (retrieved 2026-10-22)