# Harvey's Legal Agent Benchmark v1 / overall > Source: Benchize (https://benchize.com) — https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Aoverall. When citing or reusing this content, credit "Benchize" and link to https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Aoverall. Tests an agent's ability to complete legal work using documents, spreadsheets, presentations, and file-system tools Canonical URL: https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Aoverall Last updated: Wed, 21 Oct 2026 18:52:20 GMT Reproduction and quotation must name Benchize, link to the canonical HTML URL, and retain individual source citations. Tests an agent's ability to complete legal work using documents, spreadsheets, presentations, and file-system tools Professional work · Models tested: 49 · Score publishers: Vals AI ## Model results | # | Model | Score | ± | Cost | Latency | | | --- | --- | --- | --- | --- | --- | --- | | 1 | [Muse Spark 1.2](https://benchize.com/models/muse-spark-1-2-xhigh.md)Extra high | 25.42% | SE ±3.67 pp | $2.09 | 1,456.3source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 25.417 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 2.08555 \| USD \| per_test \| Lower is better \| \| `latency` \| 1456.298 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 131072 provider Meta reasoning Not reported Reasoning effort xhigh Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±3.67 pp sourcePopulation 81 stderr 3.667 tokenTotals {"cache_read_tokens":512433248,"cache_write_tokens":null,"input_tokens":608754402,"output_tokens":10507621,"reasoning_tokens":826686} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label meta/muse_spark_1_2 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/overall/meta~1muse_spark_1_2` | | | | | | | | 2 | [Muse Spark 1.3](https://benchize.com/models/muse-spark-1-3-xhigh.md)Extra high | 22.92% | SE ±3.42 pp | $2.76 | 1,024.26source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 22.917 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 2.75654 \| USD \| per_test \| Lower is better \| \| `latency` \| 1024.261 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 131072 provider Meta reasoning Not reported Reasoning effort xhigh Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±3.42 pp sourcePopulation 81 stderr 3.416 tokenTotals {"cache_read_tokens":690282338,"cache_write_tokens":null,"input_tokens":835346220,"output_tokens":8208587,"reasoning_tokens":1656318} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label meta/muse_spark_1_3 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/overall/meta~1muse_spark_1_3` | | | | | | | | 3 | [Muse Spark 1.1](https://benchize.com/models/muse-spark-1-1-xhigh.md)Extra high | 20% | SE ±2.94 pp | $0.8 | 740.83source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 20 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.800257 \| USD \| per_test \| Lower is better \| \| `latency` \| 740.833 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider Meta reasoning Not reported Reasoning effort xhigh Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.94 pp sourcePopulation 81 stderr 2.943 tokenTotals {"cache_read_tokens":263652672,"cache_write_tokens":null,"input_tokens":283802188,"output_tokens":5292532,"reasoning_tokens":497732} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label meta/muse_spark_1_1 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/overall/meta~1muse_spark_1_1` | | | | | | | | 4 | [Gemini 4 Argon](https://benchize.com/models/gemini-4-argon-high.md)High | 19.58% | SE ±3.31 pp | $10.55 | 2,634.92source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 19.583 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 10.54836 \| USD \| per_test \| Lower is better \| \| `latency` \| 2634.915 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 262144 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±3.31 pp sourcePopulation 81 stderr 3.31 tokenTotals {"cache_read_tokens":636080391,"cache_write_tokens":null,"input_tokens":826582568,"output_tokens":18828918,"reasoning_tokens":5044847} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label google/gemini-4-argon Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/overall/google~1gemini-4-argon` | | | | | | | | 5 | [Muse Wonder](https://benchize.com/models/muse-wonder-max.md)Max | 17.08% | SE ±3.1 pp | $7.08 | 1,370.23source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 17.083 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 7.081987 \| USD \| per_test \| Lower is better \| \| `latency` \| 1370.229 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 131072 provider Meta reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±3.1 pp sourcePopulation 81 stderr 3.103 tokenTotals {"cache_read_tokens":1101383344,"cache_write_tokens":0,"input_tokens":1199011074,"output_tokens":11598443,"reasoning_tokens":2010938} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label meta/muse-wonder Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/overall/meta~1muse-wonder` | | | | | | | | 6 | [Muse Spark Open](https://benchize.com/models/muse-spark-open-xhigh.md)Extra high | 16.67% | SE ±3.09 pp | $6.83 | 8,090.07source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 16.667 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 6.834878 \| USD \| per_test \| Lower is better \| \| `latency` \| 8090.065 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 131072 provider Together AI reasoning Not reported Reasoning effort xhigh Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±3.09 pp sourcePopulation 81 stderr 3.094 tokenTotals {"cache_read_tokens":595452080,"cache_write_tokens":null,"input_tokens":1146205590,"output_tokens":9982506,"reasoning_tokens":5173489} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label meta/muse-spark-open Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/overall/meta~1muse-spark-open` | | | | | | | | 7 | [MiMo-V2.6-Flash](https://benchize.com/models/mimo-v2-6-flash.md) | 11.25% | SE ±2.54 pp | $0.09 | 1,031.58source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 11.25 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.085646 \| USD \| per_test \| Lower is better \| \| `latency` \| 1031.581 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider Xiaomi reasoning Not reported Reasoning effort unknown Temperature 1 topP 0.95 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.54 pp sourcePopulation 81 stderr 2.543 tokenTotals {"cache_read_tokens":715360896,"cache_write_tokens":null,"input_tokens":751303984,"output_tokens":11580281,"reasoning_tokens":5709544} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label xiaomi/mimo-v2.6-flash Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/overall/xiaomi~1mimo-v2.6-flash` | | | | | | | | 7 | [Claude Fable 5](https://benchize.com/models/claude-fable-5-max.md)Max | 11.25% | SE ±2.15 pp | $19.23 | 1,613.04source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 11.25 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 19.225253 \| USD \| per_test \| Lower is better \| \| `latency` \| 1613.035 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.15 pp sourcePopulation 81 stderr 2.148 tokenTotals {"cache_read_tokens":880023896,"cache_write_tokens":59236069,"input_tokens":939267211,"output_tokens":13345157,"reasoning_tokens":null} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label anthropic/claude-fable-5 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/overall/anthropic~1claude-fable-5` | | | | | | | | 9 | [MiMo-V2.6-Pro](https://benchize.com/models/mimo-v2-6-pro.md) | 10.83% | SE ±2.45 pp | $0.22 | 1,553.05source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 10.833 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.216614 \| USD \| per_test \| Lower is better \| \| `latency` \| 1553.048 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider Xiaomi reasoning Not reported Reasoning effort unknown Temperature 1 topP 0.95 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.45 pp sourcePopulation 81 stderr 2.453 tokenTotals {"cache_read_tokens":600189312,"cache_write_tokens":null,"input_tokens":632411237,"output_tokens":11283302,"reasoning_tokens":5261809} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label xiaomi/mimo-v2.6-pro Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/overall/xiaomi~1mimo-v2.6-pro` | | | | | | | | 10 | [Gemini 3.8 Flash](https://benchize.com/models/gemini-3-8-flash-high.md)High | 10% | SE ±2.42 pp | $3.66 | 1,760.87source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 10 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 3.656207 \| USD \| per_test \| Lower is better \| \| `latency` \| 1760.871 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.42 pp sourcePopulation 81 stderr 2.415 tokenTotals {"cache_read_tokens":1066501383,"cache_write_tokens":null,"input_tokens":1171742866,"output_tokens":16120993,"reasoning_tokens":5514918} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label google/gemini-3.8-flash Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/overall/google~1gemini-3.8-flash` | | | | | | | | 11 | [Claude Opus 4.8](https://benchize.com/models/claude-opus-4-8-max.md)Max | 9.58% | SE ±2.15 pp | $10.22 | 1,447.79source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 9.583 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 10.222944 \| USD \| per_test \| Lower is better \| \| `latency` \| 1447.794 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.15 pp sourcePopulation 81 stderr 2.148 tokenTotals {"cache_read_tokens":1089433052,"cache_write_tokens":53824786,"input_tokens":1143265902,"output_tokens":13823660,"reasoning_tokens":null} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label anthropic/claude-opus-4-8 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/overall/anthropic~1claude-opus-4-8` | | | | | | | | 12 | [Step 5 Preview](https://benchize.com/models/step-5-preview.md) | 9.17% | SE ±2.44 pp | $1 | 1,741.94source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 9.167 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.997491 \| USD \| per_test \| Lower is better \| \| `latency` \| 1741.944 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 1024000 provider Stepfun reasoning Not reported Reasoning effort unknown Temperature 1 topP 0.95 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.44 pp sourcePopulation 81 stderr 2.439 tokenTotals {"cache_read_tokens":1056014080,"cache_write_tokens":null,"input_tokens":1090091574,"output_tokens":12155839,"reasoning_tokens":0} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label stepfun/step-5-preview Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/overall/stepfun~1step-5-preview` | | | | | | | | 12 | [DeepSeek V4.1 Flash 1017](https://benchize.com/models/deepseek-v4-1-flash-1017-max.md)Max | 9.17% | SE ±2.51 pp | $0.54 | 1,298.99source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 9.167 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.543702 \| USD \| per_test \| Lower is better \| \| `latency` \| 1298.986 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.51 pp sourcePopulation 81 stderr 2.509 tokenTotals {"cache_read_tokens":4425635584,"cache_write_tokens":null,"input_tokens":4452935434,"output_tokens":25417039,"reasoning_tokens":10306539} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash-1017 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/overall/deepseek~1deepseek-v4.1-flash-1017` | | | | | | | | 14 | [Gemini 3.7 Flash](https://benchize.com/models/gemini-3-7-flash-high.md)High | 8.75% | SE ±2.15 pp | $2.56 | 452.1source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 8.75 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 2.556743 \| USD \| per_test \| Lower is better \| \| `latency` \| 452.101 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.15 pp sourcePopulation 81 stderr 2.148 tokenTotals {"cache_read_tokens":753832806,"cache_write_tokens":null,"input_tokens":811642014,"output_tokens":14269397,"reasoning_tokens":3601488} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label google/gemini-3.7-flash Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/overall/google~1gemini-3.7-flash` | | | | | | | | 15 | [DeepSeek V4 Flash 0731](https://benchize.com/models/deepseek-v4-flash-0731-high.md)High | 8.33% | SE ±2.15 pp | $0.06 | 1,014.03source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 8.333 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.059087 \| USD \| per_test \| Lower is better \| \| `latency` \| 1014.028 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.15 pp sourcePopulation 81 stderr 2.148 tokenTotals {"cache_read_tokens":723421312,"cache_write_tokens":null,"input_tokens":744084543,"output_tokens":7335131,"reasoning_tokens":2960038} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label deepseek/deepseek-v4-flash-0731 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/overall/deepseek~1deepseek-v4-flash-0731` | | | | | | | | 16 | [DeepSeek V4 Pro 0813](https://benchize.com/models/deepseek-v4-pro-0813-max.md)Max | 7.5% | SE ±2.15 pp | $0.17 | 1,313.32source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 7.5 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.166206 \| USD \| per_test \| Lower is better \| \| `latency` \| 1313.321 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.15 pp sourcePopulation 81 stderr 2.148 tokenTotals {"cache_read_tokens":785343488,"cache_write_tokens":null,"input_tokens":805739429,"output_tokens":9454727,"reasoning_tokens":4381477} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label deepseek/deepseek-v4-pro-0813 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/overall/deepseek~1deepseek-v4-pro-0813` | | | | | | | | 17 | [Claude Fable 5.1](https://benchize.com/models/claude-fable-5-1-max.md)Max | 6.67% | SE ±1.17 pp | $46.21 | 6,749.45source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 6.667 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 46.211522 \| USD \| per_test \| Lower is better \| \| `latency` \| 6749.446 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.17 pp sourcePopulation 81 stderr 1.174 tokenTotals {"cache_read_tokens":1833706187,"cache_write_tokens":306775910,"input_tokens":2140493271,"output_tokens":25042909,"reasoning_tokens":9339413} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label anthropic/claude-fable-5-1 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/overall/anthropic~1claude-fable-5-1` | | | | | | | | 17 | [Claude Opus 4.7](https://benchize.com/models/claude-opus-4-7-max.md)Max | 6.67% | SE ±2 pp | $10.81 | 1,483.7source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 6.667 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 10.80884 \| USD \| per_test \| Lower is better \| \| `latency` \| 1483.696 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2 pp sourcePopulation 81 stderr 1.998 tokenTotals {"cache_read_tokens":1081804814,"cache_write_tokens":68683898,"input_tokens":1150494190,"output_tokens":13074265,"reasoning_tokens":1317583} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label anthropic/claude-opus-4-7 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/overall/anthropic~1claude-opus-4-7` | | | | | | | | 17 | [Claude Opus 5](https://benchize.com/models/claude-opus-5-max.md)Max | 6.67% | SE ±1.65 pp | $23.67 | 3,338.27source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 6.667 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 23.669537 \| USD \| per_test \| Lower is better \| \| `latency` \| 3338.269 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.65 pp sourcePopulation 81 stderr 1.646 tokenTotals {"cache_read_tokens":2623329853,"cache_write_tokens":134339560,"input_tokens":2757683085,"output_tokens":27559556,"reasoning_tokens":6673656} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label anthropic/claude-opus-5 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/overall/anthropic~1claude-opus-5` | | | | | | | | 17 | [DeepSeek V4.1 Flash](https://benchize.com/models/deepseek-v4-1-flash-max.md)Max | 6.67% | SE ±1.83 pp | $0.17 | 579.44source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 6.667 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.173471 \| USD \| per_test \| Lower is better \| \| `latency` \| 579.44 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.83 pp sourcePopulation 81 stderr 1.832 tokenTotals {"cache_read_tokens":699334016,"cache_write_tokens":null,"input_tokens":721200533,"output_tokens":8383826,"reasoning_tokens":4048611} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/overall/deepseek~1deepseek-v4.1-flash` | | | | | | | | 21 | [GPT-6.1 Sol](https://benchize.com/models/gpt-6-1-sol-max.md)Max | 5.42% | SE ±1.76 pp | $3.93 | 2,594.18source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 5.417 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 3.933528 \| USD \| per_test \| Lower is better \| \| `latency` \| 2594.18 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.76 pp sourcePopulation 81 stderr 1.755 tokenTotals {"cache_read_tokens":1274733716,"cache_write_tokens":36236558,"input_tokens":1310994009,"output_tokens":13514557,"reasoning_tokens":5869694} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label openai/gpt-6.1-sol Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/overall/openai~1gpt-6.1-sol` | | | | | | | | 21 | [GPT-6 Astra](https://benchize.com/models/gpt-6-astra-max.md)Max | 5.42% | SE ±1.17 pp | $26.16 | 1,489.09source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 5.417 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 26.155899 \| USD \| per_test \| Lower is better \| \| `latency` \| 1489.086 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.17 pp sourcePopulation 81 stderr 1.174 tokenTotals {"cache_read_tokens":1049116892,"cache_write_tokens":null,"input_tokens":1119192487,"output_tokens":10562148,"reasoning_tokens":3650237} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label openai/gpt-6-astra Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/overall/openai~1gpt-6-astra` | | | | | | | | 23 | [Claude Sonnet 4.6](https://benchize.com/models/claude-sonnet-4-6-max.md)Max | 5% | SE ±1.65 pp | $3.04 | 1,122.71source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 5 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 3.042346 \| USD \| per_test \| Lower is better \| \| `latency` \| 1122.707 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning true Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.65 pp sourcePopulation 81 stderr 1.646 tokenTotals {"cache_read_tokens":345650547,"cache_write_tokens":42275246,"input_tokens":387928425,"output_tokens":6856417,"reasoning_tokens":null} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label anthropic/claude-sonnet-4-6 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/overall/anthropic~1claude-sonnet-4-6` | | | | | | | | 23 | [Claude Sonnet 5](https://benchize.com/models/claude-sonnet-5-max.md)Max | 5% | SE ±1.65 pp | $8.95 | 2,331.58source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 5 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 8.953859 \| USD \| per_test \| Lower is better \| \| `latency` \| 2331.577 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.65 pp sourcePopulation 81 stderr 1.646 tokenTotals {"cache_read_tokens":3252147720,"cache_write_tokens":88748346,"input_tokens":3340913248,"output_tokens":20212831,"reasoning_tokens":null} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label anthropic/claude-sonnet-5 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/overall/anthropic~1claude-sonnet-5` | | | | | | | | 25 | [MiniMax M3](https://benchize.com/models/minimax-m3.md) | 4.17% | SE ±1.65 pp | $1.46 | 1,351.13source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 4.167 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 1.460248 \| USD \| per_test \| Lower is better \| \| `latency` \| 1351.134 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 512000 provider MiniMax reasoning Not reported Reasoning effort unknown Temperature 1 topP 0.95 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.65 pp sourcePopulation 81 stderr 1.646 tokenTotals {"cache_read_tokens":1070555232,"cache_write_tokens":0,"input_tokens":1114902164,"output_tokens":8397929,"reasoning_tokens":null} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label minimax/MiniMax-M3 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/overall/minimax~1MiniMax-M3` | | | | | | | | 26 | [GPT-5.5](https://benchize.com/models/gpt-5-5-xhigh.md)Extra high | 3.75% | SE ±1.17 pp | $4.6 | 733.85source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 3.75 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 4.597894 \| USD \| per_test \| Lower is better \| \| `latency` \| 733.847 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort xhigh Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.17 pp sourcePopulation 81 stderr 1.174 tokenTotals {"cache_read_tokens":341294080,"cache_write_tokens":null,"input_tokens":372470134,"output_tokens":4184434,"reasoning_tokens":1267952} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label openai/gpt-5.5 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/overall/openai~1gpt-5.5` | | | | | | | | 26 | [DeepSeek V4 Pro](https://benchize.com/models/deepseek-v4-pro-max.md)Max | 3.75% | SE ±1.43 pp | $0.74 | 401.52source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 3.75 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.737465 \| USD \| per_test \| Lower is better \| \| `latency` \| 401.522 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.43 pp sourcePopulation 81 stderr 1.431 tokenTotals {"cache_read_tokens":303855232,"cache_write_tokens":null,"input_tokens":321058309,"output_tokens":4167657,"reasoning_tokens":272693} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label deepseek/deepseek-v4-pro Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/overall/deepseek~1deepseek-v4-pro` | | | | | | | | 26 | [Claude Opus 5.5](https://benchize.com/models/claude-opus-5-5-max.md)Max | 3.75% | SE ±1.47 pp | $21.38 | 3,175.13source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 3.75 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 21.37949 \| USD \| per_test \| Lower is better \| \| `latency` \| 3175.128 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.47 pp sourcePopulation 81 stderr 1.469 tokenTotals {"cache_read_tokens":2286257168,"cache_write_tokens":176308820,"input_tokens":2462576678,"output_tokens":38472452,"reasoning_tokens":24925173} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label anthropic/claude-opus-5-5 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/overall/anthropic~1claude-opus-5-5` | | | | | | | | 29 | [Gemini 3.6 Flash](https://benchize.com/models/gemini-3-6-flash-high.md)High | 3.33% | SE ±1.17 pp | $1.9 | 984.49source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 3.333 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 1.895602 \| USD \| per_test \| Lower is better \| \| `latency` \| 984.49 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.17 pp sourcePopulation 81 stderr 1.174 tokenTotals {"cache_read_tokens":366081207,"cache_write_tokens":null,"input_tokens":434069662,"output_tokens":9410323,"reasoning_tokens":2894511} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label google/gemini-3.6-flash Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/overall/google~1gemini-3.6-flash` | | | | | | | | 30 | [GPT-6 Luna](https://benchize.com/models/gpt-6-luna-max.md)Max | 2.92% | SE ±1.36 pp | $0.3 | 1,051.83source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 2.917 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.303552 \| USD \| per_test \| Lower is better \| \| `latency` \| 1051.83 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.36 pp sourcePopulation 81 stderr 1.362 tokenTotals {"cache_read_tokens":1422964599,"cache_write_tokens":37829190,"input_tokens":1460813994,"output_tokens":12095727,"reasoning_tokens":6619071} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label openai/gpt-6-luna Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/overall/openai~1gpt-6-luna` | | | | | | | | 30 | [Claude Sonnet 5.5](https://benchize.com/models/claude-sonnet-5-5-max.md)Max | 2.92% | SE ±1.36 pp | $16.61 | 3,464.43source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 2.917 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 16.612583 \| USD \| per_test \| Lower is better \| \| `latency` \| 3464.426 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.36 pp sourcePopulation 81 stderr 1.362 tokenTotals {"cache_read_tokens":4582495471,"cache_write_tokens":168486127,"input_tokens":4750998856,"output_tokens":53560846,"reasoning_tokens":35373065} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label anthropic/claude-sonnet-5-5 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/overall/anthropic~1claude-sonnet-5-5` | | | | | | | | 32 | [Gemini 3.5 Flash](https://benchize.com/models/gemini-3-5-flash-high.md)High | 2.5% | SE ±0.83 pp | $2.12 | 478.41source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 2.5 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 2.116049 \| USD \| per_test \| Lower is better \| \| `latency` \| 478.414 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.83 pp sourcePopulation 81 stderr 0.833 tokenTotals {"cache_read_tokens":543998500,"cache_write_tokens":null,"input_tokens":606812300,"output_tokens":8678382,"reasoning_tokens":3806922} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label google/gemini-3.5-flash Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/overall/google~1gemini-3.5-flash` | | | | | | | | 32 | [GPT-5.6 Sol](https://benchize.com/models/gpt-5-6-sol-max.md)Max | 2.5% | SE ±0.83 pp | $10.37 | 1,370.3source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 2.5 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 10.374554 \| USD \| per_test \| Lower is better \| \| `latency` \| 1370.296 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.83 pp sourcePopulation 81 stderr 0.833 tokenTotals {"cache_read_tokens":1150617952,"cache_write_tokens":null,"input_tokens":1221154582,"output_tokens":10565145,"reasoning_tokens":2786144} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label openai/gpt-5.6-sol Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/overall/openai~1gpt-5.6-sol` | | | | | | | | 34 | [MiMo-V2.5-Pro](https://benchize.com/models/mimo-v2-5-pro.md) | 2.08% | SE ±0.83 pp | $0.11 | 544.23source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 2.083 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.107095 \| USD \| per_test \| Lower is better \| \| `latency` \| 544.226 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider Xiaomi reasoning Not reported Reasoning effort unknown Temperature 1 topP 0.95 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.83 pp sourcePopulation 81 stderr 0.833 tokenTotals {"cache_read_tokens":203250560,"cache_write_tokens":null,"input_tokens":224714337,"output_tokens":3198839,"reasoning_tokens":0} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label xiaomi/mimo-v2.5-pro Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/overall/xiaomi~1mimo-v2.5-pro` | | | | | | | | 35 | [MiMo-V2.5](https://benchize.com/models/mimo-v2-5.md) | 1.67% | SE ±0 pp | $0.04 | 419.28source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 1.667 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.036252 \| USD \| per_test \| Lower is better \| \| `latency` \| 419.277 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider Xiaomi reasoning Not reported Reasoning effort unknown Temperature 1 topP 0.95 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 81 stderr 0 tokenTotals {"cache_read_tokens":232673984,"cache_write_tokens":null,"input_tokens":252892290,"output_tokens":3100583,"reasoning_tokens":145588} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label xiaomi/mimo-v2.5 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/overall/xiaomi~1mimo-v2.5` | | | | | | | | 35 | [GPT-6 Sol](https://benchize.com/models/gpt-6-sol-max.md)Max | 1.67% | SE ±0.82 pp | $3.36 | 820.1source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 1.667 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 3.358015 \| USD \| per_test \| Lower is better \| \| `latency` \| 820.097 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.82 pp sourcePopulation 81 stderr 0.823 tokenTotals {"cache_read_tokens":791317282,"cache_write_tokens":31785134,"input_tokens":823114531,"output_tokens":7639311,"reasoning_tokens":3222223} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label openai/gpt-6-sol Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/overall/openai~1gpt-6-sol` | | | | | | | | 37 | [GPT-5.6 Luna](https://benchize.com/models/gpt-5-6-luna-max.md)Max | 1.25% | SE ±0.83 pp | $0.38 | 656.36source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 1.25 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.383981 \| USD \| per_test \| Lower is better \| \| `latency` \| 656.357 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.83 pp sourcePopulation 81 stderr 0.833 tokenTotals {"cache_read_tokens":909992290,"cache_write_tokens":null,"input_tokens":993049543,"output_tokens":9388666,"reasoning_tokens":2979345} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label openai/gpt-5.6-luna Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/overall/openai~1gpt-5.6-luna` | | | | | | | | 37 | [Claude Haiku 5.5](https://benchize.com/models/claude-haiku-5-5-max.md)Max | 1.25% | SE ±0.72 pp | $3.39 | 2,113.92source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 1.25 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 3.394654 \| USD \| per_test \| Lower is better \| \| `latency` \| 2113.923 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.72 pp sourcePopulation 81 stderr 0.716 tokenTotals {"cache_read_tokens":4002588737,"cache_write_tokens":124738510,"input_tokens":4127341953,"output_tokens":54990908,"reasoning_tokens":37472036} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label anthropic/claude-haiku-5-5 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/overall/anthropic~1claude-haiku-5-5` | | | | | | | | 39 | [GPT-5.6 Terra](https://benchize.com/models/gpt-5-6-terra-max.md)Max | 0.83% | SE ±0.83 pp | $3.2 | 1,024.89source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0.833 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 3.202399 \| USD \| per_test \| Lower is better \| \| `latency` \| 1024.886 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.83 pp sourcePopulation 81 stderr 0.833 tokenTotals {"cache_read_tokens":740704262,"cache_write_tokens":null,"input_tokens":802970365,"output_tokens":9301238,"reasoning_tokens":2973597} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label openai/gpt-5.6-terra Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/overall/openai~1gpt-5.6-terra` | | | | | | | | 40 | [Nemotron 3 Ultra](https://benchize.com/models/nemotron-3-ultra-550b-a55b.md) | 0.42% | SE ±0 pp | — | 2,048.54source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0.417 \| percent \| reported \| Higher is better \| \| `latency` \| 2048.539 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider Nvidia reasoning Not reported Reasoning effort unknown Temperature 1 topP 0.95 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 81 stderr 0 tokenTotals {"cache_read_tokens":48583680,"cache_write_tokens":null,"input_tokens":305706389,"output_tokens":3758581,"reasoning_tokens":null} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label nvidia/nemotron-3-ultra-550b-a55b Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/overall/nvidia~1nemotron-3-ultra-550b-a55b` | | | | | | | | 40 | [Mistral Medium 3.5](https://benchize.com/models/mistral-medium-3-5-high.md)High | 0.42% | SE ±0 pp | $6.07 | 3,694.95source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0.417 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 6.074265 \| USD \| per_test \| Lower is better \| \| `latency` \| 3694.945 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 80000 provider Mistral AI reasoning Not reported Reasoning effort high Temperature 0.7 topP 0.95 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 81 stderr 0 tokenTotals {"cache_read_tokens":null,"cache_write_tokens":null,"input_tokens":439770514,"output_tokens":5184631,"reasoning_tokens":null} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label mistralai/mistral-medium-3.5 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/overall/mistralai~1mistral-medium-3.5` | | | | | | | | 42 | [MiniMax M2.7](https://benchize.com/models/minimax-m2-7.md) | 0% | SE ±0 pp | $0.49 | 373.93source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.489594 \| USD \| per_test \| Lower is better \| \| `latency` \| 373.931 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 80000 provider MiniMax reasoning Not reported Reasoning effort unknown Temperature 1 topP 0.95 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 81 stderr 0 tokenTotals {"cache_read_tokens":11540069,"cache_write_tokens":18939,"input_tokens":194662499,"output_tokens":2889064,"reasoning_tokens":177130} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label minimax/MiniMax-M2.7 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/overall/minimax~1MiniMax-M2.7` | | | | | | | | 42 | [Laguna M 1](https://benchize.com/models/laguna-m-1.md) | 0% | SE ±0 pp | $0.53 | 465.12source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.527685 \| USD \| per_test \| Lower is better \| \| `latency` \| 465.122 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 32768 provider Poolside reasoning Not reported Reasoning effort unknown Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 81 stderr 0 tokenTotals {"cache_read_tokens":495461278,"cache_write_tokens":null,"input_tokens":547314889,"output_tokens":1917329,"reasoning_tokens":177714} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label poolside/laguna-m.1 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/overall/poolside~1laguna-m.1` | | | | | | | | 42 | [Mercury 2.5](https://benchize.com/models/mercury-2-5-high.md)High | 0% | SE ±0 pp | $0.19 | 131.54source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.192614 \| USD \| per_test \| Lower is better \| \| `latency` \| 131.54 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Inception reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 81 stderr 0 tokenTotals {"cache_read_tokens":55653173,"cache_write_tokens":null,"input_tokens":159575776,"output_tokens":1621389,"reasoning_tokens":1524161} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label inception/mercury-2.5 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/overall/inception~1mercury-2.5` | | | | | | | | 42 | [Gemini 3.1 Pro Preview](https://benchize.com/models/gemini-3-1-pro-preview-high.md)High | 0% | SE ±0 pp | $1.15 | 436.69source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 1.149757 \| USD \| per_test \| Lower is better \| \| `latency` \| 436.69 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 81 stderr 0 tokenTotals {"cache_read_tokens":204622317,"cache_write_tokens":null,"input_tokens":230732445,"output_tokens":1350000,"reasoning_tokens":503310} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label google/gemini-3.1-pro-preview Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/overall/google~1gemini-3.1-pro-preview` | | | | | | | | 42 | [Laguna Xs 2](https://benchize.com/models/laguna-xs-2.md) | 0% | SE ±0 pp | $0.28 | 1,194.2source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.284127 \| USD \| per_test \| Lower is better \| \| `latency` \| 1194.199 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 32768 provider Poolside reasoning Not reported Reasoning effort unknown Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 81 stderr 0 tokenTotals {"cache_read_tokens":189201152,"cache_write_tokens":null,"input_tokens":322083675,"output_tokens":1329814,"reasoning_tokens":131024} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label poolside/laguna-xs.2 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/overall/poolside~1laguna-xs.2` | | | | | | | | 42 | [Gemini 3.5 Flash-Lite](https://benchize.com/models/gemini-3-5-flash-lite-high.md)High | 0% | SE ±0 pp | $0.33 | 232.9source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.331805 \| USD \| per_test \| Lower is better \| \| `latency` \| 232.895 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 81 stderr 0 tokenTotals {"cache_read_tokens":146158528,"cache_write_tokens":null,"input_tokens":234750879,"output_tokens":3541635,"reasoning_tokens":1058834} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label google/gemini-3.5-flash-lite Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/overall/google~1gemini-3.5-flash-lite` | | | | | | | | 42 | [Gemini 3.1 Flash Lite Preview](https://benchize.com/models/gemini-3-1-flash-lite-preview-high.md)High | 0% | SE ±0 pp | $0.08 | 60.55source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.084048 \| USD \| per_test \| Lower is better \| \| `latency` \| 60.554 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 81 stderr 0 tokenTotals {"cache_read_tokens":138155856,"cache_write_tokens":null,"input_tokens":160066368,"output_tokens":769494,"reasoning_tokens":360545} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label google/gemini-3.1-flash-lite-preview Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/overall/google~1gemini-3.1-flash-lite-preview` | | | | | | | | 42 | [Gemini 3 Flash Preview](https://benchize.com/models/gemini-3-flash-preview-high.md)High | 0% | SE ±0 pp | $0.26 | 202.51source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.264375 \| USD \| per_test \| Lower is better \| \| `latency` \| 202.513 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 81 stderr 0 tokenTotals {"cache_read_tokens":190051973,"cache_write_tokens":null,"input_tokens":225950385,"output_tokens":1424392,"reasoning_tokens":580723} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label google/gemini-3-flash-preview Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/overall/google~1gemini-3-flash-preview` | | | | | | | --- Cite as: Harvey's Legal Agent Benchmark v1 / overall — Benchize, https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Aoverall (retrieved 2026-10-21)