# LegalBench / rule tasks > Source: Benchize (https://benchize.com) — https://benchize.com/benchmarks/vals-ai%3Alegal_bench%3Arule_tasks. When citing or reusing this content, credit "Benchize" and link to https://benchize.com/benchmarks/vals-ai%3Alegal_bench%3Arule_tasks. Evaluating language models on a wide range of open source legal reasoning tasks. Canonical URL: https://benchize.com/benchmarks/vals-ai%3Alegal_bench%3Arule_tasks Last updated: Wed, 21 Oct 2026 18:52:20 GMT Reproduction and quotation must name Benchize, link to the canonical HTML URL, and retain individual source citations. Evaluating language models on a wide range of open source legal reasoning tasks. Professional work · Models tested: 44 · Score publishers: Vals AI ## Model results | # | Model | Score | ± | Cost | Latency | | | --- | --- | --- | --- | --- | --- | --- | | 1 | [Gemini 4 Argon](https://benchize.com/models/gemini-4-argon-high.md)High | 90.77% | SE ±1.12 pp | $0.02 | 19.21source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 90.774 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.021453 \| USD \| per_test \| Lower is better \| \| `latency` \| 19.214 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 262144 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.12 pp sourcePopulation 151 stderr 1.123 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_bench.D6uTgA3b.json) Source Vals AI Original model label google/gemini-4-argon Source version 1 Observed 2026-10-21T16:45:53.390Z Evidence SHA-256 `c8486ba756baa5cd3955d035edff67440e8db23e69d62ad1a9e18d15b8c4bc52` Source locator `/tasks/rule_tasks/google~1gemini-4-argon` | | | | | | | | 2 | [Claude Fable 5](https://benchize.com/models/claude-fable-5-max.md)Max | 88.33% | SE ±1.15 pp | $0.05 | 15.69source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 88.33 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.053962 \| USD \| per_test \| Lower is better \| \| `latency` \| 15.686 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.15 pp sourcePopulation 151 stderr 1.145 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_bench.D6uTgA3b.json) Source Vals AI Original model label anthropic/claude-fable-5 Source version 1 Observed 2026-10-21T16:45:53.390Z Evidence SHA-256 `c8486ba756baa5cd3955d035edff67440e8db23e69d62ad1a9e18d15b8c4bc52` Source locator `/tasks/rule_tasks/anthropic~1claude-fable-5` | | | | | | | | 3 | [Gemini 3.1 Pro Preview](https://benchize.com/models/gemini-3-1-pro-preview-high.md)High | 88.29% | SE ±1.15 pp | $0.00883 | 13.15source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 88.294 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.00883 \| USD \| per_test \| Lower is better \| \| `latency` \| 13.146 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.15 pp sourcePopulation 151 stderr 1.152 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_bench.D6uTgA3b.json) Source Vals AI Original model label google/gemini-3.1-pro-preview Source version 1 Observed 2026-10-21T16:45:53.390Z Evidence SHA-256 `c8486ba756baa5cd3955d035edff67440e8db23e69d62ad1a9e18d15b8c4bc52` Source locator `/tasks/rule_tasks/google~1gemini-3.1-pro-preview` | | | | | | | | 4 | [Claude Fable 5.1](https://benchize.com/models/claude-fable-5-1-max.md)Max | 88.13% | SE ±1.41 pp | $0.13 | 33.43source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 88.125 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.130251 \| USD \| per_test \| Lower is better \| \| `latency` \| 33.428 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.41 pp sourcePopulation 151 stderr 1.413 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_bench.D6uTgA3b.json) Source Vals AI Original model label anthropic/claude-fable-5-1 Source version 1 Observed 2026-10-21T16:45:53.390Z Evidence SHA-256 `c8486ba756baa5cd3955d035edff67440e8db23e69d62ad1a9e18d15b8c4bc52` Source locator `/tasks/rule_tasks/anthropic~1claude-fable-5-1` | | | | | | | | 5 | [Gemini 3.5 Flash](https://benchize.com/models/gemini-3-5-flash-high.md)High | 88% | SE ±1.73 pp | $0.01 | 35.43source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 87.996 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.01073 \| USD \| per_test \| Lower is better \| \| `latency` \| 35.426 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.73 pp sourcePopulation 151 stderr 1.731 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_bench.D6uTgA3b.json) Source Vals AI Original model label google/gemini-3.5-flash Source version 1 Observed 2026-10-21T16:45:53.390Z Evidence SHA-256 `c8486ba756baa5cd3955d035edff67440e8db23e69d62ad1a9e18d15b8c4bc52` Source locator `/tasks/rule_tasks/google~1gemini-3.5-flash` | | | | | | | | 6 | [Gemini 3.8 Flash](https://benchize.com/models/gemini-3-8-flash-high.md)High | 87.08% | SE ±1.24 pp | $0.00785 | 3.74source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 87.076 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.007846 \| USD \| per_test \| Lower is better \| \| `latency` \| 3.736 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.24 pp sourcePopulation 151 stderr 1.242 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_bench.D6uTgA3b.json) Source Vals AI Original model label google/gemini-3.8-flash Source version 1 Observed 2026-10-21T16:45:53.390Z Evidence SHA-256 `c8486ba756baa5cd3955d035edff67440e8db23e69d62ad1a9e18d15b8c4bc52` Source locator `/tasks/rule_tasks/google~1gemini-3.8-flash` | | | | | | | | 7 | [GPT-5.6 Sol](https://benchize.com/models/gpt-5-6-sol-max.md)Max | 86.44% | SE ±1.34 pp | $0.03 | 14.84source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 86.443 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.025965 \| USD \| per_test \| Lower is better \| \| `latency` \| 14.835 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.34 pp sourcePopulation 151 stderr 1.335 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_bench.D6uTgA3b.json) Source Vals AI Original model label openai/gpt-5.6-sol Source version 1 Observed 2026-10-21T16:45:53.390Z Evidence SHA-256 `c8486ba756baa5cd3955d035edff67440e8db23e69d62ad1a9e18d15b8c4bc52` Source locator `/tasks/rule_tasks/openai~1gpt-5.6-sol` | | | | | | | | 8 | [Gemini 3.7 Flash](https://benchize.com/models/gemini-3-7-flash-high.md)High | 85.85% | SE ±1.36 pp | $0.00501 | 3.07source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 85.851 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.005009 \| USD \| per_test \| Lower is better \| \| `latency` \| 3.067 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.36 pp sourcePopulation 151 stderr 1.359 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_bench.D6uTgA3b.json) Source Vals AI Original model label google/gemini-3.7-flash Source version 1 Observed 2026-10-21T16:45:53.390Z Evidence SHA-256 `c8486ba756baa5cd3955d035edff67440e8db23e69d62ad1a9e18d15b8c4bc52` Source locator `/tasks/rule_tasks/google~1gemini-3.7-flash` | | | | | | | | 9 | [Gemini 3.6 Flash](https://benchize.com/models/gemini-3-6-flash-high.md)High | 85.73% | SE ±1.49 pp | $0.00559 | 4.17source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 85.726 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.005592 \| USD \| per_test \| Lower is better \| \| `latency` \| 4.165 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.49 pp sourcePopulation 151 stderr 1.488 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_bench.D6uTgA3b.json) Source Vals AI Original model label google/gemini-3.6-flash Source version 1 Observed 2026-10-21T16:45:53.390Z Evidence SHA-256 `c8486ba756baa5cd3955d035edff67440e8db23e69d62ad1a9e18d15b8c4bc52` Source locator `/tasks/rule_tasks/google~1gemini-3.6-flash` | | | | | | | | 10 | [GPT-5.5](https://benchize.com/models/gpt-5-5-xhigh.md)Extra high | 85.45% | SE ±1.49 pp | $0.04 | 50.03source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 85.453 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.040762 \| USD \| per_test \| Lower is better \| \| `latency` \| 50.029 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort xhigh Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.49 pp sourcePopulation 151 stderr 1.488 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_bench.D6uTgA3b.json) Source Vals AI Original model label openai/gpt-5.5 Source version 1 Observed 2026-10-21T16:45:53.390Z Evidence SHA-256 `c8486ba756baa5cd3955d035edff67440e8db23e69d62ad1a9e18d15b8c4bc52` Source locator `/tasks/rule_tasks/openai~1gpt-5.5` | | | | | | | | 11 | [Gemini 3 Flash Preview](https://benchize.com/models/gemini-3-flash-preview-high.md)High | 85.29% | SE ±1.28 pp | $0.00211 | 5.22source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 85.291 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.002112 \| USD \| per_test \| Lower is better \| \| `latency` \| 5.224 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.28 pp sourcePopulation 151 stderr 1.281 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_bench.D6uTgA3b.json) Source Vals AI Original model label google/gemini-3-flash-preview Source version 1 Observed 2026-10-21T16:45:53.390Z Evidence SHA-256 `c8486ba756baa5cd3955d035edff67440e8db23e69d62ad1a9e18d15b8c4bc52` Source locator `/tasks/rule_tasks/google~1gemini-3-flash-preview` | | | | | | | | 12 | [Claude Opus 4.7](https://benchize.com/models/claude-opus-4-7-max.md)Max | 85% | SE ±1.54 pp | $0.00675 | 46.69source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 84.996 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.006751 \| USD \| per_test \| Lower is better \| \| `latency` \| 46.694 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.54 pp sourcePopulation 151 stderr 1.544 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_bench.D6uTgA3b.json) Source Vals AI Original model label anthropic/claude-opus-4-7 Source version 1 Observed 2026-10-21T16:45:53.390Z Evidence SHA-256 `c8486ba756baa5cd3955d035edff67440e8db23e69d62ad1a9e18d15b8c4bc52` Source locator `/tasks/rule_tasks/anthropic~1claude-opus-4-7` | | | | | | | | 13 | [Claude Opus 5](https://benchize.com/models/claude-opus-5-max.md)Max | 84.81% | SE ±1.48 pp | $0.02 | 11.2source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 84.805 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.020142 \| USD \| per_test \| Lower is better \| \| `latency` \| 11.197 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.48 pp sourcePopulation 151 stderr 1.477 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_bench.D6uTgA3b.json) Source Vals AI Original model label anthropic/claude-opus-5 Source version 1 Observed 2026-10-21T16:45:53.390Z Evidence SHA-256 `c8486ba756baa5cd3955d035edff67440e8db23e69d62ad1a9e18d15b8c4bc52` Source locator `/tasks/rule_tasks/anthropic~1claude-opus-5` | | | | | | | | 14 | [Muse Spark 1.2](https://benchize.com/models/muse-spark-1-2-xhigh.md)Extra high | 84.73% | SE ±1.6 pp | $0.00901 | 42.42source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 84.725 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.009008 \| USD \| per_test \| Lower is better \| \| `latency` \| 42.418 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 131072 provider Meta reasoning Not reported Reasoning effort xhigh Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.6 pp sourcePopulation 151 stderr 1.597 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_bench.D6uTgA3b.json) Source Vals AI Original model label meta/muse_spark_1_2 Source version 1 Observed 2026-10-21T16:45:53.390Z Evidence SHA-256 `c8486ba756baa5cd3955d035edff67440e8db23e69d62ad1a9e18d15b8c4bc52` Source locator `/tasks/rule_tasks/meta~1muse_spark_1_2` | | | | | | | | 15 | [Muse Spark 1.1](https://benchize.com/models/muse-spark-1-1-xhigh.md)Extra high | 83.56% | SE ±1.51 pp | $0.00965 | 22.92source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 83.562 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.009646 \| USD \| per_test \| Lower is better \| \| `latency` \| 22.919 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider Meta reasoning Not reported Reasoning effort xhigh Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.51 pp sourcePopulation 151 stderr 1.509 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_bench.D6uTgA3b.json) Source Vals AI Original model label meta/muse_spark_1_1 Source version 1 Observed 2026-10-21T16:45:53.390Z Evidence SHA-256 `c8486ba756baa5cd3955d035edff67440e8db23e69d62ad1a9e18d15b8c4bc52` Source locator `/tasks/rule_tasks/meta~1muse_spark_1_1` | | | | | | | | 16 | [Gemini 3.1 Flash Lite Preview](https://benchize.com/models/gemini-3-1-flash-lite-preview-high.md)High | 82.17% | SE ±1.58 pp | $0.000287 | 3.14source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 82.168 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.000287 \| USD \| per_test \| Lower is better \| \| `latency` \| 3.143 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.58 pp sourcePopulation 151 stderr 1.576 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_bench.D6uTgA3b.json) Source Vals AI Original model label google/gemini-3.1-flash-lite-preview Source version 1 Observed 2026-10-21T16:45:53.390Z Evidence SHA-256 `c8486ba756baa5cd3955d035edff67440e8db23e69d62ad1a9e18d15b8c4bc52` Source locator `/tasks/rule_tasks/google~1gemini-3.1-flash-lite-preview` | | | | | | | | 17 | [Gemini 3.5 Flash-Lite](https://benchize.com/models/gemini-3-5-flash-lite-high.md)High | 81.71% | SE ±1.6 pp | $0.00162 | 2.34source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 81.708 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.001616 \| USD \| per_test \| Lower is better \| \| `latency` \| 2.344 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.6 pp sourcePopulation 151 stderr 1.602 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_bench.D6uTgA3b.json) Source Vals AI Original model label google/gemini-3.5-flash-lite Source version 1 Observed 2026-10-21T16:45:53.390Z Evidence SHA-256 `c8486ba756baa5cd3955d035edff67440e8db23e69d62ad1a9e18d15b8c4bc52` Source locator `/tasks/rule_tasks/google~1gemini-3.5-flash-lite` | | | | | | | | 18 | [MiniMax M2.7](https://benchize.com/models/minimax-m2-7.md) | 80.99% | SE ±1.56 pp | $0.000568 | 11.26source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 80.988 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.000568 \| USD \| per_test \| Lower is better \| \| `latency` \| 11.261 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 196608 provider MiniMax reasoning Not reported Reasoning effort unknown Temperature 1 topP 0.95 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.56 pp sourcePopulation 151 stderr 1.559 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_bench.D6uTgA3b.json) Source Vals AI Original model label minimax/MiniMax-M2.7 Source version 1 Observed 2026-10-21T16:45:53.390Z Evidence SHA-256 `c8486ba756baa5cd3955d035edff67440e8db23e69d62ad1a9e18d15b8c4bc52` Source locator `/tasks/rule_tasks/minimax~1MiniMax-M2.7` | | | | | | | | 19 | [MiniMax M3](https://benchize.com/models/minimax-m3.md) | 80.97% | SE ±1.76 pp | $0.00104 | 11.83source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 80.972 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.001039 \| USD \| per_test \| Lower is better \| \| `latency` \| 11.832 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 512000 provider MiniMax reasoning Not reported Reasoning effort unknown Temperature 1 topP 0.95 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.76 pp sourcePopulation 151 stderr 1.76 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_bench.D6uTgA3b.json) Source Vals AI Original model label minimax/MiniMax-M3 Source version 1 Observed 2026-10-21T16:45:53.390Z Evidence SHA-256 `c8486ba756baa5cd3955d035edff67440e8db23e69d62ad1a9e18d15b8c4bc52` Source locator `/tasks/rule_tasks/minimax~1MiniMax-M3` | | | | | | | | 20 | [GPT-5.6 Terra](https://benchize.com/models/gpt-5-6-terra-max.md)Max | 80.53% | SE ±1.64 pp | $0.00402 | 5.02source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 80.528 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.004017 \| USD \| per_test \| Lower is better \| \| `latency` \| 5.02 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.64 pp sourcePopulation 151 stderr 1.635 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_bench.D6uTgA3b.json) Source Vals AI Original model label openai/gpt-5.6-terra Source version 1 Observed 2026-10-21T16:45:53.390Z Evidence SHA-256 `c8486ba756baa5cd3955d035edff67440e8db23e69d62ad1a9e18d15b8c4bc52` Source locator `/tasks/rule_tasks/openai~1gpt-5.6-terra` | | | | | | | | 21 | [GPT-4 Turbo](https://benchize.com/models/gpt-4-turbo-high.md)High | 80.26% | SE ±1.9 pp | — | 3.28source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 80.26 \| percent \| reported \| Higher is better \| \| `latency` \| 3.282 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 4096 provider OpenAI reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.9 pp sourcePopulation 151 stderr 1.897 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_bench.D6uTgA3b.json) Source Vals AI Original model label openai/gpt-4-turbo Source version 1 Observed 2026-10-21T16:45:53.390Z Evidence SHA-256 `c8486ba756baa5cd3955d035edff67440e8db23e69d62ad1a9e18d15b8c4bc52` Source locator `/tasks/rule_tasks/openai~1gpt-4-turbo` | | | | | | | | 22 | [Claude Opus 4.8](https://benchize.com/models/claude-opus-4-8-max.md)Max | 79.98% | SE ±1.72 pp | $0.00658 | 4.08source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 79.976 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.006584 \| USD \| per_test \| Lower is better \| \| `latency` \| 4.075 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.72 pp sourcePopulation 151 stderr 1.72 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_bench.D6uTgA3b.json) Source Vals AI Original model label anthropic/claude-opus-4-8 Source version 1 Observed 2026-10-21T16:45:53.390Z Evidence SHA-256 `c8486ba756baa5cd3955d035edff67440e8db23e69d62ad1a9e18d15b8c4bc52` Source locator `/tasks/rule_tasks/anthropic~1claude-opus-4-8` | | | | | | | | 23 | [Nemotron 3 Ultra](https://benchize.com/models/nemotron-3-ultra-550b-a55b.md) | 79.39% | SE ±1.76 pp | — | 7.98source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 79.393 \| percent \| reported \| Higher is better \| \| `latency` \| 7.98 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider Nvidia reasoning Not reported Reasoning effort unknown Temperature Not reported topP 0.95 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.76 pp sourcePopulation 151 stderr 1.763 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_bench.D6uTgA3b.json) Source Vals AI Original model label nvidia/nemotron-3-ultra-550b-a55b Source version 1 Observed 2026-10-21T16:45:53.390Z Evidence SHA-256 `c8486ba756baa5cd3955d035edff67440e8db23e69d62ad1a9e18d15b8c4bc52` Source locator `/tasks/rule_tasks/nvidia~1nemotron-3-ultra-550b-a55b` | | | | | | | | 24 | [Claude Sonnet 5](https://benchize.com/models/claude-sonnet-5-max.md)Max | 79.33% | SE ±1.74 pp | $0.0068 | 7.87source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 79.331 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.006804 \| USD \| per_test \| Lower is better \| \| `latency` \| 7.866 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.74 pp sourcePopulation 151 stderr 1.742 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_bench.D6uTgA3b.json) Source Vals AI Original model label anthropic/claude-sonnet-5 Source version 1 Observed 2026-10-21T16:45:53.390Z Evidence SHA-256 `c8486ba756baa5cd3955d035edff67440e8db23e69d62ad1a9e18d15b8c4bc52` Source locator `/tasks/rule_tasks/anthropic~1claude-sonnet-5` | | | | | | | | 25 | [Mistral Large 3 2512](https://benchize.com/models/mistral-large-2512.md) | 79.13% | SE ±1.5 pp | $0.000344 | 5.15source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 79.13 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.000344 \| USD \| per_test \| Lower is better \| \| `latency` \| 5.148 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider Mistral AI reasoning Not reported Reasoning effort unknown Temperature 0.065 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.5 pp sourcePopulation 151 stderr 1.496 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_bench.D6uTgA3b.json) Source Vals AI Original model label mistralai/mistral-large-2512 Source version 1 Observed 2026-10-21T16:45:53.390Z Evidence SHA-256 `c8486ba756baa5cd3955d035edff67440e8db23e69d62ad1a9e18d15b8c4bc52` Source locator `/tasks/rule_tasks/mistralai~1mistral-large-2512` | | | | | | | | 26 | [DeepSeek V4 Pro 0813](https://benchize.com/models/deepseek-v4-pro-0813-max.md)Max | 78.98% | SE ±1.56 pp | $0.00951 | 51.49source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 78.978 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.009512 \| USD \| per_test \| Lower is better \| \| `latency` \| 51.492 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.56 pp sourcePopulation 151 stderr 1.556 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_bench.D6uTgA3b.json) Source Vals AI Original model label deepseek/deepseek-v4-pro-0813 Source version 1 Observed 2026-10-21T16:45:53.390Z Evidence SHA-256 `c8486ba756baa5cd3955d035edff67440e8db23e69d62ad1a9e18d15b8c4bc52` Source locator `/tasks/rule_tasks/deepseek~1deepseek-v4-pro-0813` | | | | | | | | 27 | [DeepSeek V4.1 Flash](https://benchize.com/models/deepseek-v4-1-flash-high.md)High | 77.98% | SE ±1.69 pp | $0.00321 | 14.28source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 77.981 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.003209 \| USD \| per_test \| Lower is better \| \| `latency` \| 14.284 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.69 pp sourcePopulation 151 stderr 1.691 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_bench.D6uTgA3b.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash Source version 1 Observed 2026-10-21T16:45:53.390Z Evidence SHA-256 `c8486ba756baa5cd3955d035edff67440e8db23e69d62ad1a9e18d15b8c4bc52` Source locator `/tasks/rule_tasks/deepseek~1deepseek-v4.1-flash` | | | | | | | | 28 | [GPT-5.6 Luna](https://benchize.com/models/gpt-5-6-luna-max.md)Max | 77.74% | SE ±1.6 pp | $0.00272 | 15.67source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 77.736 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.002719 \| USD \| per_test \| Lower is better \| \| `latency` \| 15.668 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.6 pp sourcePopulation 151 stderr 1.596 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_bench.D6uTgA3b.json) Source Vals AI Original model label openai/gpt-5.6-luna Source version 1 Observed 2026-10-21T16:45:53.390Z Evidence SHA-256 `c8486ba756baa5cd3955d035edff67440e8db23e69d62ad1a9e18d15b8c4bc52` Source locator `/tasks/rule_tasks/openai~1gpt-5.6-luna` | | | | | | | | 29 | [GPT-4o (2024-11-20)](https://benchize.com/models/gpt-4o-2024-11-20-high.md)High | 77.43% | SE ±1.9 pp | — | 0.88source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 77.429 \| percent \| reported \| Higher is better \| \| `latency` \| 0.882 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 16384 provider OpenAI reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.9 pp sourcePopulation 151 stderr 1.9 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_bench.D6uTgA3b.json) Source Vals AI Original model label openai/gpt-4o-2024-11-20 Source version 1 Observed 2026-10-21T16:45:53.390Z Evidence SHA-256 `c8486ba756baa5cd3955d035edff67440e8db23e69d62ad1a9e18d15b8c4bc52` Source locator `/tasks/rule_tasks/openai~1gpt-4o-2024-11-20` | | | | | | | | 30 | [DeepSeek V4 Pro](https://benchize.com/models/deepseek-v4-pro-max.md)Max | 77.43% | SE ±1.7 pp | $0.00592 | 70.61source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 77.425 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.005923 \| USD \| per_test \| Lower is better \| \| `latency` \| 70.613 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.7 pp sourcePopulation 151 stderr 1.699 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_bench.D6uTgA3b.json) Source Vals AI Original model label deepseek/deepseek-v4-pro Source version 1 Observed 2026-10-21T16:45:53.390Z Evidence SHA-256 `c8486ba756baa5cd3955d035edff67440e8db23e69d62ad1a9e18d15b8c4bc52` Source locator `/tasks/rule_tasks/deepseek~1deepseek-v4-pro` | | | | | | | | 31 | [DeepSeek V4 Flash 0731](https://benchize.com/models/deepseek-v4-flash-0731-high.md)High | 76.29% | SE ±1.73 pp | $0.000826 | 6.87source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 76.287 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.000826 \| USD \| per_test \| Lower is better \| \| `latency` \| 6.867 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.73 pp sourcePopulation 151 stderr 1.728 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_bench.D6uTgA3b.json) Source Vals AI Original model label deepseek/deepseek-v4-flash-0731 Source version 1 Observed 2026-10-21T16:45:53.390Z Evidence SHA-256 `c8486ba756baa5cd3955d035edff67440e8db23e69d62ad1a9e18d15b8c4bc52` Source locator `/tasks/rule_tasks/deepseek~1deepseek-v4-flash-0731` | | | | | | | | 32 | [Mercury 2.5](https://benchize.com/models/mercury-2-5-high.md)High | 75.48% | SE ±2 pp | $0.000731 | 4.52source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 75.484 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.000731 \| USD \| per_test \| Lower is better \| \| `latency` \| 4.523 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Inception reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2 pp sourcePopulation 151 stderr 2 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_bench.D6uTgA3b.json) Source Vals AI Original model label inception/mercury-2.5 Source version 1 Observed 2026-10-21T16:45:53.390Z Evidence SHA-256 `c8486ba756baa5cd3955d035edff67440e8db23e69d62ad1a9e18d15b8c4bc52` Source locator `/tasks/rule_tasks/inception~1mercury-2.5` | | | | | | | | 33 | [GPT-4o (2024-08-06)](https://benchize.com/models/gpt-4o-2024-08-06-high.md)High | 74.72% | SE ±2.36 pp | — | 1.18source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 74.718 \| percent \| reported \| Higher is better \| \| `latency` \| 1.178 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 16384 provider OpenAI reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.36 pp sourcePopulation 151 stderr 2.358 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_bench.D6uTgA3b.json) Source Vals AI Original model label openai/gpt-4o-2024-08-06 Source version 1 Observed 2026-10-21T16:45:53.390Z Evidence SHA-256 `c8486ba756baa5cd3955d035edff67440e8db23e69d62ad1a9e18d15b8c4bc52` Source locator `/tasks/rule_tasks/openai~1gpt-4o-2024-08-06` | | | | | | | | 34 | [MiniMax M2.1](https://benchize.com/models/minimax-m2-1.md) | 74.55% | SE ±1.83 pp | $0.00159 | 21.45source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 74.551 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.001587 \| USD \| per_test \| Lower is better \| \| `latency` \| 21.452 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 131000 provider MiniMax reasoning Not reported Reasoning effort unknown Temperature 1 topP 0.95 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.83 pp sourcePopulation 151 stderr 1.826 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_bench.D6uTgA3b.json) Source Vals AI Original model label minimax/MiniMax-M2.1 Source version 1 Observed 2026-10-21T16:45:53.390Z Evidence SHA-256 `c8486ba756baa5cd3955d035edff67440e8db23e69d62ad1a9e18d15b8c4bc52` Source locator `/tasks/rule_tasks/minimax~1MiniMax-M2.1` | | | | | | | | 35 | [Claude Sonnet 4.6](https://benchize.com/models/claude-sonnet-4-6-max.md)Max | 74.1% | SE ±1.56 pp | $0.00642 | 14.16source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 74.097 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.006421 \| USD \| per_test \| Lower is better \| \| `latency` \| 14.158 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning true Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.56 pp sourcePopulation 151 stderr 1.56 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_bench.D6uTgA3b.json) Source Vals AI Original model label anthropic/claude-sonnet-4-6 Source version 1 Observed 2026-10-21T16:45:53.390Z Evidence SHA-256 `c8486ba756baa5cd3955d035edff67440e8db23e69d62ad1a9e18d15b8c4bc52` Source locator `/tasks/rule_tasks/anthropic~1claude-sonnet-4-6` | | | | | | | | 36 | [Gemini 2.5 Flash Lite Preview 09-2025](https://benchize.com/models/gemini-2-5-flash-lite-preview-09-2025.md) | 73.98% | SE ±1.6 pp | — | 0.78source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 73.981 \| percent \| reported \| Higher is better \| \| `latency` \| 0.783 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65535 provider Google reasoning Not reported Reasoning effort unknown Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.6 pp sourcePopulation 151 stderr 1.598 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_bench.D6uTgA3b.json) Source Vals AI Original model label google/gemini-2.5-flash-lite-preview-09-2025 Source version 1 Observed 2026-10-21T16:45:53.390Z Evidence SHA-256 `c8486ba756baa5cd3955d035edff67440e8db23e69d62ad1a9e18d15b8c4bc52` Source locator `/tasks/rule_tasks/google~1gemini-2.5-flash-lite-preview-09-2025` | | | | | | | | 37 | [Command A](https://benchize.com/models/command-a.md) | 73.6% | SE ±1.56 pp | — | 1.39source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 73.598 \| percent \| reported \| Higher is better \| \| `latency` \| 1.385 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 8000 provider Cohere reasoning Not reported Reasoning effort unknown Temperature 0.3 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.56 pp sourcePopulation 151 stderr 1.556 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_bench.D6uTgA3b.json) Source Vals AI Original model label cohere/command-a-03-2025 Source version 1 Observed 2026-10-21T16:45:53.390Z Evidence SHA-256 `c8486ba756baa5cd3955d035edff67440e8db23e69d62ad1a9e18d15b8c4bc52` Source locator `/tasks/rule_tasks/cohere~1command-a-03-2025` | | | | | | | | 38 | [MiMo-V2.5-Pro](https://benchize.com/models/mimo-v2-5-pro.md) | 72.98% | SE ±1.84 pp | $0.000606 | 23.39source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 72.98 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.000606 \| USD \| per_test \| Lower is better \| \| `latency` \| 23.385 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider Xiaomi reasoning Not reported Reasoning effort unknown Temperature 1 topP 0.95 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.84 pp sourcePopulation 151 stderr 1.841 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_bench.D6uTgA3b.json) Source Vals AI Original model label xiaomi/mimo-v2.5-pro Source version 1 Observed 2026-10-21T16:45:53.390Z Evidence SHA-256 `c8486ba756baa5cd3955d035edff67440e8db23e69d62ad1a9e18d15b8c4bc52` Source locator `/tasks/rule_tasks/xiaomi~1mimo-v2.5-pro` | | | | | | | | 39 | [MiniMax M2.5](https://benchize.com/models/minimax-m2-5.md) | 72.55% | SE ±1.8 pp | $0.00181 | 27.84source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 72.55 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.001806 \| USD \| per_test \| Lower is better \| \| `latency` \| 27.838 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 196608 provider MiniMax reasoning Not reported Reasoning effort unknown Temperature 1 topP 0.95 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.8 pp sourcePopulation 151 stderr 1.804 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_bench.D6uTgA3b.json) Source Vals AI Original model label minimax/MiniMax-M2.5 Source version 1 Observed 2026-10-21T16:45:53.390Z Evidence SHA-256 `c8486ba756baa5cd3955d035edff67440e8db23e69d62ad1a9e18d15b8c4bc52` Source locator `/tasks/rule_tasks/minimax~1MiniMax-M2.5` | | | | | | | | 40 | [MiMo-V2.5](https://benchize.com/models/mimo-v2-5.md) | 69.58% | SE ±2.04 pp | $0.000227 | 11.25source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 69.582 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.000227 \| USD \| per_test \| Lower is better \| \| `latency` \| 11.246 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider Xiaomi reasoning Not reported Reasoning effort unknown Temperature 1 topP 0.95 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.04 pp sourcePopulation 151 stderr 2.038 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_bench.D6uTgA3b.json) Source Vals AI Original model label xiaomi/mimo-v2.5 Source version 1 Observed 2026-10-21T16:45:53.390Z Evidence SHA-256 `c8486ba756baa5cd3955d035edff67440e8db23e69d62ad1a9e18d15b8c4bc52` Source locator `/tasks/rule_tasks/xiaomi~1mimo-v2.5` | | | | | | | | 41 | [GPT-3.5 Turbo](https://benchize.com/models/gpt-3-5-turbo-high.md)High | 66.17% | SE ±2.66 pp | — | 0.78source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 66.174 \| percent \| reported \| Higher is better \| \| `latency` \| 0.777 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 4096 provider OpenAI reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.66 pp sourcePopulation 151 stderr 2.659 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_bench.D6uTgA3b.json) Source Vals AI Original model label openai/gpt-3.5-turbo Source version 1 Observed 2026-10-21T16:45:53.390Z Evidence SHA-256 `c8486ba756baa5cd3955d035edff67440e8db23e69d62ad1a9e18d15b8c4bc52` Source locator `/tasks/rule_tasks/openai~1gpt-3.5-turbo` | | | | | | | | 42 | [Laguna Xs 2](https://benchize.com/models/laguna-xs-2.md) | 63.29% | SE ±2.15 pp | — | 16.79source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 63.288 \| percent \| reported \| Higher is better \| \| `latency` \| 16.793 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 32768 provider Poolside reasoning Not reported Reasoning effort unknown Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.15 pp sourcePopulation 151 stderr 2.145 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_bench.D6uTgA3b.json) Source Vals AI Original model label poolside/laguna-xs.2 Source version 1 Observed 2026-10-21T16:45:53.390Z Evidence SHA-256 `c8486ba756baa5cd3955d035edff67440e8db23e69d62ad1a9e18d15b8c4bc52` Source locator `/tasks/rule_tasks/poolside~1laguna-xs.2` | | | | | | | | 43 | [Laguna M 1](https://benchize.com/models/laguna-m-1.md) | 62.45% | SE ±2.27 pp | — | 69.82source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 62.451 \| percent \| reported \| Higher is better \| \| `latency` \| 69.822 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 32768 provider Poolside reasoning Not reported Reasoning effort unknown Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.27 pp sourcePopulation 151 stderr 2.269 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_bench.D6uTgA3b.json) Source Vals AI Original model label poolside/laguna-m.1 Source version 1 Observed 2026-10-21T16:45:53.390Z Evidence SHA-256 `c8486ba756baa5cd3955d035edff67440e8db23e69d62ad1a9e18d15b8c4bc52` Source locator `/tasks/rule_tasks/poolside~1laguna-m.1` | | | | | | | | 44 | [Command A Plus](https://benchize.com/models/command-a-plus.md) | 46.72% | SE ±2.41 pp | $0.12 | 85.57source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 46.72 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.123978 \| USD \| per_test \| Lower is better \| \| `latency` \| 85.57 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 64000 provider Cohere reasoning Not reported Reasoning effort unknown Temperature 0.9 topP 0.95 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.41 pp sourcePopulation 151 stderr 2.408 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_bench.D6uTgA3b.json) Source Vals AI Original model label cohere/command-a-plus-05-2026 Source version 1 Observed 2026-10-21T16:45:53.390Z Evidence SHA-256 `c8486ba756baa5cd3955d035edff67440e8db23e69d62ad1a9e18d15b8c4bc52` Source locator `/tasks/rule_tasks/cohere~1command-a-plus-05-2026` | | | | | | | --- Cite as: LegalBench / rule tasks — Benchize, https://benchize.com/benchmarks/vals-ai%3Alegal_bench%3Arule_tasks (retrieved 2026-10-21)