# Mercury 2.5High > Source: Benchize (https://benchize.com) — https://benchize.com/models/mercury-2-5-high. When citing or reusing this content, credit "Benchize" and link to https://benchize.com/models/mercury-2-5-high. Inception Canonical URL: https://benchize.com/models/mercury-2-5-high Last updated: Thu, 22 Oct 2026 06:20:09 GMT Reproduction and quotation must name Benchize, link to the canonical HTML URL, and retain individual source citations. Inception ## Benchmarks | Benchmark | Score | ± | Rank | Cost | Latency | Source | | | --- | --- | --- | --- | --- | --- | --- | --- | | [Tax Agent Bench / weighted pass current temporal analysis](https://benchize.com/benchmarks/vals-ai%3Atax_agent_bench%3Aweighted_pass__current_temporal_analysis.md) | 13.2% | SE ±4.97 pp | 45 / 46 | $0.02 | 36.09source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 13.201 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.022182 \| USD \| per_test \| Lower is better \| \| `latency` \| 36.086 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Inception reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±4.97 pp sourcePopulation 73 stderr 4.973 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) Source Vals AI Original model label inception/mercury-2.5 Source version 1 Observed 2026-10-22T02:10:11.407Z Evidence SHA-256 `446db182b67efb7923b3187a4040ec3a3ea492b8ad699a2b6d708d714cc063ac` Source locator `/tasks/weighted_pass__current_temporal_analysis/inception~1mercury-2.5` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / structured finance securitization](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Astructured_finance_securitization.md) | 0% | SE ±0 pp | 12 / 49 | $0.23 | 79.68source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.225971 \| USD \| per_test \| Lower is better \| \| `latency` \| 79.682 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Inception reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 81 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label inception/mercury-2.5 Source version 1 Observed 2026-10-22T02:10:09.422Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/structured_finance_securitization/inception~1mercury-2.5` | | | | | | | | | [EMB / misc](https://benchize.com/benchmarks/vals-ai%3Aemb%3Amisc.md) | 12.54% | SE ±3.58 pp | 46 / 47 | $0.33 | 304.06source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 12.539 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.327239 \| USD \| per_test \| Lower is better \| \| `latency` \| 304.064 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Inception reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±3.58 pp sourcePopulation 76 stderr 3.581 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) Source Vals AI Original model label inception/mercury-2.5 Source version 1 Observed 2026-10-22T02:10:12.907Z Evidence SHA-256 `d807aee620ea47152a1fca0257a8dd12a1be4512c76cc1744c1ae9e285180918` Source locator `/tasks/misc/inception~1mercury-2.5` | | | | | | | | | [Tax Agent Bench / forms filings](https://benchize.com/benchmarks/vals-ai%3Atax_agent_bench%3Aforms_filings.md) | 0% | SE ±0 pp | 41 / 46 | $0.03 | 41.9source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.031988 \| USD \| per_test \| Lower is better \| \| `latency` \| 41.899 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Inception reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 73 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) Source Vals AI Original model label inception/mercury-2.5 Source version 1 Observed 2026-10-22T02:10:11.407Z Evidence SHA-256 `446db182b67efb7923b3187a4040ec3a3ea492b8ad699a2b6d708d714cc063ac` Source locator `/tasks/forms_filings/inception~1mercury-2.5` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / criteria pass rate](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate.md) | 40.9% | SE ±0.59 pp | 48 / 49 | $0.19 | 131.54source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 40.903 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.192614 \| USD \| per_test \| Lower is better \| \| `latency` \| 131.54 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Inception reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.59 pp sourcePopulation 81 stderr 0.592 tokenTotals {"cache_read_tokens":55653173,"cache_write_tokens":null,"input_tokens":159575776,"output_tokens":1621389,"reasoning_tokens":1524161} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label inception/mercury-2.5 Source version 1 Observed 2026-10-22T02:10:09.422Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate/inception~1mercury-2.5` | | | | | | | | | [Finance Agent (v2) / general quantitative analysis](https://benchize.com/benchmarks/vals-ai%3Afabv2%3Ageneral_quantitative_analysis.md) | 31.35% | SE ±1.74 pp | 48 / 50 | $0.11 | 125.38source duration | [Vals AI · 2](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 31.353 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.109433 \| USD \| per_test \| Lower is better \| \| `latency` \| 125.376 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Inception reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.74 pp sourcePopulation 80 stderr 1.741 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) Source Vals AI Original model label inception/mercury-2.5 Source version 2 Observed 2026-10-22T02:10:12.404Z Evidence SHA-256 `f731b20a26c126725d125519f2b5841aa45254656dba859a9570d7ce7e5d8191` Source locator `/tasks/general_quantitative_analysis/inception~1mercury-2.5` | | | | | | | | | [CyberBench v1.1 / poc](https://benchize.com/benchmarks/benchmark%3Acyber%3Apoc.md) | 26.67% | SE ±5.71 pp | 22 / 28 | $0.12 | 1,616.78source duration | [Vals AI · 1.1](https://www.vals.ai/_astro/benchmark_view_cyberbench.DS6UOlu0.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 26.667 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.122978 \| USD \| per_test \| Lower is better \| \| `latency` \| 1616.777 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Inception reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±5.71 pp sourcePopulation 47 stderr 5.709 tokenTotals {"cache_read_tokens":131523068,"cache_write_tokens":null,"input_tokens":151807555,"output_tokens":921785,"reasoning_tokens":861715} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_cyberbench.DS6UOlu0.json) Source Vals AI Original model label inception/mercury-2.5 Source version 1.1 Observed 2026-10-22T02:10:29.837Z Evidence SHA-256 `943f593f929f8971a6b746269a2851b5febc2aad4e2f1066895f715033cdc7f1` Source locator `/tasks/poc/inception~1mercury-2.5` | | | | | | | | | [SkillsBench / with skills](https://benchize.com/benchmarks/vals-ai%3Askillsbench%3Awith_skills.md) | 18.06% | SE ±3.08 pp | 20 / 20 | $0.1 | 49.36source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_skillsbench.BE2TXB-M.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 18.061 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.101286 \| USD \| per_test \| Lower is better \| \| `latency` \| 49.364 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness OpenHands maxOutputTokens 65536 provider Inception reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±3.08 pp sourcePopulation 35 stderr 3.079 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_skillsbench.BE2TXB-M.json) Source Vals AI Original model label inception/mercury-2.5 Source version 1 Observed 2026-10-22T02:10:25.807Z Evidence SHA-256 `aaad242949d81b7c7c61c36d4895d4460e9146760c1e6b4b38d235a16043bc61` Source locator `/tasks/with_skills/inception~1mercury-2.5` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / criteria pass rate emerging companies venture capital](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__emerging_companies_venture_capital.md) | 49.15% | SE ±2.93 pp | 46 / 49 | $0.15 | 70.56source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 49.147 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.150699 \| USD \| per_test \| Lower is better \| \| `latency` \| 70.562 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Inception reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.93 pp sourcePopulation 81 stderr 2.926 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label inception/mercury-2.5 Source version 1 Observed 2026-10-22T02:10:09.422Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__emerging_companies_venture_capital/inception~1mercury-2.5` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / healthcare life sciences](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Ahealthcare_life_sciences.md) | 0% | SE ±0 pp | 24 / 49 | $0.21 | 112.73source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.210725 \| USD \| per_test \| Lower is better \| \| `latency` \| 112.726 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Inception reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 81 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label inception/mercury-2.5 Source version 1 Observed 2026-10-22T02:10:09.422Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/healthcare_life_sciences/inception~1mercury-2.5` | | | | | | | | | [Terminal-Bench 2.1 / hard](https://benchize.com/benchmarks/vals-ai%3Aterminal-bench-2-1%3Ahard.md) | 12.22% | SE ±2.22 pp | 41 / 43 | $0.64 | 2,043.81source duration | [Vals AI · 2.1](https://www.vals.ai/_astro/benchmark_view_terminal_bench_2_1.CdyoObsA.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 12.222 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.637794 \| USD \| per_test \| Lower is better \| \| `latency` \| 2043.805 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Inception reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.22 pp sourcePopulation 76 stderr 2.222 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_terminal_bench_2_1.CdyoObsA.json) Source Vals AI Original model label inception/mercury-2.5 Source version 2.1 Observed 2026-10-22T02:10:27.811Z Evidence SHA-256 `a5306805cdf86495bdcceff6b2c8534a3768b9853f5077895dab975b809b054f` Source locator `/tasks/hard/inception~1mercury-2.5` | | | | | | | | | [Code Migration / cli](https://benchize.com/benchmarks/vals-ai%3Acode-migration%3Acli.md) | 6.02% | SE ±1.4 pp | 43 / 47 | $0.52 | 760.46source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_code_migration.Dttzoqk3.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 6.023 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.522043 \| USD \| per_test \| Lower is better \| \| `latency` \| 760.46 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Inception reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.4 pp sourcePopulation 79 stderr 1.398 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_code_migration.Dttzoqk3.json) Source Vals AI Original model label inception/mercury-2.5 Source version 1 Observed 2026-10-22T02:10:27.307Z Evidence SHA-256 `99194597f2c76c402195be47839409046e18bd30454616da6dd40b736a088fa4` Source locator `/tasks/cli/inception~1mercury-2.5` | | | | | | | | | [Finance Agent (v2) / financial modeling](https://benchize.com/benchmarks/vals-ai%3Afabv2%3Afinancial_modeling.md) | 2.78% | SE ±0.09 pp | 47 / 50 | $0.1 | 113.17source duration | [Vals AI · 2](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 2.775 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.098027 \| USD \| per_test \| Lower is better \| \| `latency` \| 113.172 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Inception reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.09 pp sourcePopulation 80 stderr 0.085 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) Source Vals AI Original model label inception/mercury-2.5 Source version 2 Observed 2026-10-22T02:10:12.404Z Evidence SHA-256 `f731b20a26c126725d125519f2b5841aa45254656dba859a9570d7ce7e5d8191` Source locator `/tasks/financial_modeling/inception~1mercury-2.5` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / criteria pass rate funds asset management](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__funds_asset_management.md) | 35.28% | SE ±2.58 pp | 46 / 49 | $0.25 | 322.28source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 35.277 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.253859 \| USD \| per_test \| Lower is better \| \| `latency` \| 322.28 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Inception reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.58 pp sourcePopulation 81 stderr 2.584 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label inception/mercury-2.5 Source version 1 Observed 2026-10-22T02:10:09.422Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__funds_asset_management/inception~1mercury-2.5` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / criteria pass rate healthcare life sciences](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__healthcare_life_sciences.md) | 42.57% | SE ±2.85 pp | 47 / 49 | $0.21 | 112.73source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 42.574 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.210725 \| USD \| per_test \| Lower is better \| \| `latency` \| 112.726 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Inception reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.85 pp sourcePopulation 81 stderr 2.845 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label inception/mercury-2.5 Source version 1 Observed 2026-10-22T02:10:09.422Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__healthcare_life_sciences/inception~1mercury-2.5` | | | | | | | | | [LegalBench / interpretation tasks](https://benchize.com/benchmarks/vals-ai%3Alegal_bench%3Ainterpretation_tasks.md) | 81.91% | SE ±0.2 pp | 21 / 44 | $0.000802 | 3.74source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_bench.D6uTgA3b.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 81.911 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.000802 \| USD \| per_test \| Lower is better \| \| `latency` \| 3.739 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Inception reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.2 pp sourcePopulation 151 stderr 0.201 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_bench.D6uTgA3b.json) Source Vals AI Original model label inception/mercury-2.5 Source version 1 Observed 2026-10-22T02:10:08.909Z Evidence SHA-256 `c8486ba756baa5cd3955d035edff67440e8db23e69d62ad1a9e18d15b8c4bc52` Source locator `/tasks/interpretation_tasks/inception~1mercury-2.5` | | | | | | | | | [Legal Research Bench / administrative regulatory](https://benchize.com/benchmarks/vals-ai%3Alegal_research%3Aadministrative_regulatory.md) | 4.55% | SE ±2.58 pp | 47 / 50 | $0.05 | 41.91source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 4.545 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.046871 \| USD \| per_test \| Lower is better \| \| `latency` \| 41.907 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Inception reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.58 pp sourcePopulation 82 stderr 2.584 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) Source Vals AI Original model label inception/mercury-2.5 Source version 1 Observed 2026-10-22T02:10:07.934Z Evidence SHA-256 `26c58dfd47d0d2dc86f33c6d056f30309e0ee63570e7cd752c593b385f9d862e` Source locator `/tasks/administrative_regulatory/inception~1mercury-2.5` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / trusts estates private client](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Atrusts_estates_private_client.md) | 0% | SE ±0 pp | 15 / 49 | $0.21 | 81.35source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.209388 \| USD \| per_test \| Lower is better \| \| `latency` \| 81.35 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Inception reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 81 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label inception/mercury-2.5 Source version 1 Observed 2026-10-22T02:10:09.422Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/trusts_estates_private_client/inception~1mercury-2.5` | | | | | | | | | [Tax Agent Bench / rule source lookup](https://benchize.com/benchmarks/vals-ai%3Atax_agent_bench%3Arule_source_lookup.md) | 2.33% | SE ±2.33 pp | 46 / 46 | $0.02 | 29.33source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 2.333 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.021089 \| USD \| per_test \| Lower is better \| \| `latency` \| 29.325 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Inception reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.33 pp sourcePopulation 73 stderr 2.333 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) Source Vals AI Original model label inception/mercury-2.5 Source version 1 Observed 2026-10-22T02:10:11.407Z Evidence SHA-256 `446db182b67efb7923b3187a4040ec3a3ea492b8ad699a2b6d708d714cc063ac` Source locator `/tasks/rule_source_lookup/inception~1mercury-2.5` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / banking finance](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Abanking_finance.md) | 0% | SE ±0 pp | 5 / 49 | $0.12 | 189.08source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.12378 \| USD \| per_test \| Lower is better \| \| `latency` \| 189.08 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Inception reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 81 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label inception/mercury-2.5 Source version 1 Observed 2026-10-22T02:10:09.422Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/banking_finance/inception~1mercury-2.5` | | | | | | | | | [SkillsBench / overall](https://benchize.com/benchmarks/vals-ai%3Askillsbench%3Aoverall.md) | 18.06% | SE ±3.08 pp | 20 / 20 | $0.1 | 49.36source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_skillsbench.BE2TXB-M.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 18.061 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.101286 \| USD \| per_test \| Lower is better \| \| `latency` \| 49.364 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness OpenHands maxOutputTokens 65536 provider Inception reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±3.08 pp sourcePopulation 35 stderr 3.079 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_skillsbench.BE2TXB-M.json) Source Vals AI Original model label inception/mercury-2.5 Source version 1 Observed 2026-10-22T02:10:25.807Z Evidence SHA-256 `aaad242949d81b7c7c61c36d4895d4460e9146760c1e6b4b38d235a16043bc61` Source locator `/tasks/overall/inception~1mercury-2.5` | | | | | | | | | [Legal Research Bench / constitutional civil rights](https://benchize.com/benchmarks/vals-ai%3Alegal_research%3Aconstitutional_civil_rights.md) | 2.63% | SE ±2.63 pp | 47 / 50 | $0.06 | 53.04source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 2.632 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.058844 \| USD \| per_test \| Lower is better \| \| `latency` \| 53.037 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Inception reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.63 pp sourcePopulation 82 stderr 2.632 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) Source Vals AI Original model label inception/mercury-2.5 Source version 1 Observed 2026-10-22T02:10:07.934Z Evidence SHA-256 `26c58dfd47d0d2dc86f33c6d056f30309e0ee63570e7cd752c593b385f9d862e` Source locator `/tasks/constitutional_civil_rights/inception~1mercury-2.5` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / criteria pass rate energy natural resources](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__energy_natural_resources.md) | 28.05% | SE ±2.59 pp | 47 / 49 | $0.16 | 382.35source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 28.053 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.164729 \| USD \| per_test \| Lower is better \| \| `latency` \| 382.348 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Inception reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.59 pp sourcePopulation 81 stderr 2.585 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label inception/mercury-2.5 Source version 1 Observed 2026-10-22T02:10:09.422Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__energy_natural_resources/inception~1mercury-2.5` | | | | | | | | | [Tax Agent Bench / weighted pass](https://benchize.com/benchmarks/vals-ai%3Atax_agent_bench%3Aweighted_pass.md) | 12.41% | SE ±1.81 pp | 46 / 46 | $0.02 | 29.67source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 12.41 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.023633 \| USD \| per_test \| Lower is better \| \| `latency` \| 29.67 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Inception reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.81 pp sourcePopulation 73 stderr 1.805 tokenTotals {"cache_read_tokens":29554557,"cache_write_tokens":null,"input_tokens":43610163,"output_tokens":1545275,"reasoning_tokens":1248847} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) Source Vals AI Original model label inception/mercury-2.5 Source version 1 Observed 2026-10-22T02:10:11.407Z Evidence SHA-256 `446db182b67efb7923b3187a4040ec3a3ea492b8ad699a2b6d708d714cc063ac` Source locator `/tasks/weighted_pass/inception~1mercury-2.5` | | | | | | | | | [Legal Research Bench / criminal](https://benchize.com/benchmarks/vals-ai%3Alegal_research%3Acriminal.md) | 1.75% | SE ±1.75 pp | 47 / 50 | $0.06 | 48.52source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 1.754 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.059734 \| USD \| per_test \| Lower is better \| \| `latency` \| 48.522 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Inception reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.75 pp sourcePopulation 82 stderr 1.754 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) Source Vals AI Original model label inception/mercury-2.5 Source version 1 Observed 2026-10-22T02:10:07.934Z Evidence SHA-256 `26c58dfd47d0d2dc86f33c6d056f30309e0ee63570e7cd752c593b385f9d862e` Source locator `/tasks/criminal/inception~1mercury-2.5` | | | | | | | | | [Legal Research Bench / weighted pass administrative regulatory](https://benchize.com/benchmarks/vals-ai%3Alegal_research%3Aweighted_pass__administrative_regulatory.md) | 55.23% | SE ±3.32 pp | 49 / 50 | $0.05 | 41.91source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 55.228 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.046871 \| USD \| per_test \| Lower is better \| \| `latency` \| 41.907 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Inception reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±3.32 pp sourcePopulation 82 stderr 3.324 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) Source Vals AI Original model label inception/mercury-2.5 Source version 1 Observed 2026-10-22T02:10:07.934Z Evidence SHA-256 `26c58dfd47d0d2dc86f33c6d056f30309e0ee63570e7cd752c593b385f9d862e` Source locator `/tasks/weighted_pass__administrative_regulatory/inception~1mercury-2.5` | | | | | | | | | [Code Migration / code quality](https://benchize.com/benchmarks/vals-ai%3Acode-migration%3Acode_quality.md) | 2.74% | SE ±0.07 pp | 42 / 47 | $0.63 | 974.08source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_code_migration.Dttzoqk3.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 2.737 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.627223 \| USD \| per_test \| Lower is better \| \| `latency` \| 974.083 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Inception reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.07 pp sourcePopulation 79 stderr 0.067 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_code_migration.Dttzoqk3.json) Source Vals AI Original model label inception/mercury-2.5 Source version 1 Observed 2026-10-22T02:10:27.307Z Evidence SHA-256 `99194597f2c76c402195be47839409046e18bd30454616da6dd40b736a088fa4` Source locator `/tasks/code_quality/inception~1mercury-2.5` | | | | | | | | | [LegalBench / conclusion tasks](https://benchize.com/benchmarks/vals-ai%3Alegal_bench%3Aconclusion_tasks.md) | 88.54% | SE ±0.84 pp | 23 / 44 | $0.000651 | 3.43source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_bench.D6uTgA3b.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 88.542 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.000651 \| USD \| per_test \| Lower is better \| \| `latency` \| 3.431 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Inception reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.84 pp sourcePopulation 151 stderr 0.844 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_bench.D6uTgA3b.json) Source Vals AI Original model label inception/mercury-2.5 Source version 1 Observed 2026-10-22T02:10:08.909Z Evidence SHA-256 `c8486ba756baa5cd3955d035edff67440e8db23e69d62ad1a9e18d15b8c4bc52` Source locator `/tasks/conclusion_tasks/inception~1mercury-2.5` | | | | | | | | | [Legal Research Bench / weighted pass](https://benchize.com/benchmarks/vals-ai%3Alegal_research%3Aweighted_pass.md) | 58.67% | SE ±1.89 pp | 48 / 50 | $0.06 | 47.61source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 58.67 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.058112 \| USD \| per_test \| Lower is better \| \| `latency` \| 47.609 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Inception reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.89 pp sourcePopulation 82 stderr 1.894 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) Source Vals AI Original model label inception/mercury-2.5 Source version 1 Observed 2026-10-22T02:10:07.934Z Evidence SHA-256 `26c58dfd47d0d2dc86f33c6d056f30309e0ee63570e7cd752c593b385f9d862e` Source locator `/tasks/weighted_pass/inception~1mercury-2.5` | | | | | | | | | [SkillsBench / no skills](https://benchize.com/benchmarks/vals-ai%3Askillsbench%3Ano_skills.md) | 9.13% | SE ±2.18 pp | 20 / 20 | $0.07 | 36.37source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_skillsbench.BE2TXB-M.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 9.133 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.074157 \| USD \| per_test \| Lower is better \| \| `latency` \| 36.37 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness OpenHands maxOutputTokens 65536 provider Inception reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.18 pp sourcePopulation 35 stderr 2.183 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_skillsbench.BE2TXB-M.json) Source Vals AI Original model label inception/mercury-2.5 Source version 1 Observed 2026-10-22T02:10:25.807Z Evidence SHA-256 `aaad242949d81b7c7c61c36d4895d4460e9146760c1e6b4b38d235a16043bc61` Source locator `/tasks/no_skills/inception~1mercury-2.5` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / funds asset management](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Afunds_asset_management.md) | 0% | SE ±0 pp | 11 / 49 | $0.25 | 322.28source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.253859 \| USD \| per_test \| Lower is better \| \| `latency` \| 322.28 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Inception reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 81 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label inception/mercury-2.5 Source version 1 Observed 2026-10-22T02:10:09.422Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/funds_asset_management/inception~1mercury-2.5` | | | | | | | | | [Tax Agent Bench / current temporal analysis](https://benchize.com/benchmarks/vals-ai%3Atax_agent_bench%3Acurrent_temporal_analysis.md) | 2.33% | SE ±2.33 pp | 45 / 46 | $0.02 | 36.09source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 2.333 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.022182 \| USD \| per_test \| Lower is better \| \| `latency` \| 36.086 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Inception reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.33 pp sourcePopulation 73 stderr 2.333 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) Source Vals AI Original model label inception/mercury-2.5 Source version 1 Observed 2026-10-22T02:10:11.407Z Evidence SHA-256 `446db182b67efb7923b3187a4040ec3a3ea492b8ad699a2b6d708d714cc063ac` Source locator `/tasks/current_temporal_analysis/inception~1mercury-2.5` | | | | | | | | | [Finance Agent (v2) / general qualitative analysis](https://benchize.com/benchmarks/vals-ai%3Afabv2%3Ageneral_qualitative_analysis.md) | 29.48% | SE ±1.08 pp | 48 / 50 | $0.06 | 94.17source duration | [Vals AI · 2](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 29.476 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.059496 \| USD \| per_test \| Lower is better \| \| `latency` \| 94.169 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Inception reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.08 pp sourcePopulation 80 stderr 1.076 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) Source Vals AI Original model label inception/mercury-2.5 Source version 2 Observed 2026-10-22T02:10:12.404Z Evidence SHA-256 `f731b20a26c126725d125519f2b5841aa45254656dba859a9570d7ce7e5d8191` Source locator `/tasks/general_qualitative_analysis/inception~1mercury-2.5` | | | | | | | | | [Finance Agent (v2) / earnings analysis](https://benchize.com/benchmarks/vals-ai%3Afabv2%3Aearnings_analysis.md) | 15.39% | SE ±1.75 pp | 48 / 50 | $0.07 | 104.1source duration | [Vals AI · 2](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 15.394 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.074738 \| USD \| per_test \| Lower is better \| \| `latency` \| 104.1 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Inception reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.75 pp sourcePopulation 80 stderr 1.754 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) Source Vals AI Original model label inception/mercury-2.5 Source version 2 Observed 2026-10-22T02:10:12.404Z Evidence SHA-256 `f731b20a26c126725d125519f2b5841aa45254656dba859a9570d7ce7e5d8191` Source locator `/tasks/earnings_analysis/inception~1mercury-2.5` | | | | | | | | | [IOI / ioi2024](https://benchize.com/benchmarks/vals-ai%3Aioi%3Aioi2024.md) | 2.67% | SE ±1.61 pp | 29 / 29 | $0.13 | 203.2source duration | [Vals AI · 2](https://www.vals.ai/_astro/benchmark_view_ioi.DV1xD5vC.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 2.667 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.132547 \| USD \| per_test \| Lower is better \| \| `latency` \| 203.195 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Inception reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.61 pp sourcePopulation 45 stderr 1.61 tokenTotals {"cache_read_tokens":7344903,"cache_write_tokens":0,"input_tokens":9837909,"output_tokens":182920,"reasoning_tokens":175725} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_ioi.DV1xD5vC.json) Source Vals AI Original model label inception/mercury-2.5 Source version 2 Observed 2026-10-22T02:10:26.312Z Evidence SHA-256 `63a24003cd68e5620c2ddb114b077adec8c5e5bf5916bb04ced543c48d731a45` Source locator `/tasks/ioi2024/inception~1mercury-2.5` | | | | | | | | | [EMB / comps](https://benchize.com/benchmarks/vals-ai%3Aemb%3Acomps.md) | 11.52% | SE ±4.02 pp | 46 / 47 | $0.48 | 317.21source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 11.518 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.481209 \| USD \| per_test \| Lower is better \| \| `latency` \| 317.21 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Inception reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±4.02 pp sourcePopulation 76 stderr 4.015 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) Source Vals AI Original model label inception/mercury-2.5 Source version 1 Observed 2026-10-22T02:10:12.907Z Evidence SHA-256 `d807aee620ea47152a1fca0257a8dd12a1be4512c76cc1744c1ae9e285180918` Source locator `/tasks/comps/inception~1mercury-2.5` | | | | | | | | | [Legal Research Bench / weighted pass constitutional civil rights](https://benchize.com/benchmarks/vals-ai%3Alegal_research%3Aweighted_pass__constitutional_civil_rights.md) | 62.89% | SE ±4.37 pp | 49 / 50 | $0.06 | 53.04source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 62.892 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.058844 \| USD \| per_test \| Lower is better \| \| `latency` \| 53.037 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Inception reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±4.37 pp sourcePopulation 82 stderr 4.365 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) Source Vals AI Original model label inception/mercury-2.5 Source version 1 Observed 2026-10-22T02:10:07.934Z Evidence SHA-256 `26c58dfd47d0d2dc86f33c6d056f30309e0ee63570e7cd752c593b385f9d862e` Source locator `/tasks/weighted_pass__constitutional_civil_rights/inception~1mercury-2.5` | | | | | | | | | [Finance Agent (v2) / adjustments](https://benchize.com/benchmarks/vals-ai%3Afabv2%3Aadjustments.md) | 12.42% | SE ±0.4 pp | 48 / 50 | $0.21 | 177.24source duration | [Vals AI · 2](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 12.415 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.208655 \| USD \| per_test \| Lower is better \| \| `latency` \| 177.236 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Inception reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.4 pp sourcePopulation 80 stderr 0.396 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) Source Vals AI Original model label inception/mercury-2.5 Source version 2 Observed 2026-10-22T02:10:12.404Z Evidence SHA-256 `f731b20a26c126725d125519f2b5841aa45254656dba859a9570d7ce7e5d8191` Source locator `/tasks/adjustments/inception~1mercury-2.5` | | | | | | | | | [Tax Agent Bench / numbers calculations](https://benchize.com/benchmarks/vals-ai%3Atax_agent_bench%3Anumbers_calculations.md) | 8% | SE ±4.53 pp | 46 / 46 | $0.02 | 22source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 8 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.015262 \| USD \| per_test \| Lower is better \| \| `latency` \| 22.004 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Inception reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±4.53 pp sourcePopulation 73 stderr 4.533 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) Source Vals AI Original model label inception/mercury-2.5 Source version 1 Observed 2026-10-22T02:10:11.407Z Evidence SHA-256 `446db182b67efb7923b3187a4040ec3a3ea492b8ad699a2b6d708d714cc063ac` Source locator `/tasks/numbers_calculations/inception~1mercury-2.5` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / criteria pass rate arbitration international dispute resolution](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__arbitration_international_dispute_resolution.md) | 48.03% | SE ±3 pp | 44 / 49 | $0.29 | 151.01source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 48.029 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.294423 \| USD \| per_test \| Lower is better \| \| `latency` \| 151.01 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Inception reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±3 pp sourcePopulation 81 stderr 2.996 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label inception/mercury-2.5 Source version 1 Observed 2026-10-22T02:10:09.422Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__arbitration_international_dispute_resolution/inception~1mercury-2.5` | | | | | | | | | [Finance Agent (v2) / precedents](https://benchize.com/benchmarks/vals-ai%3Afabv2%3Aprecedents.md) | 7.45% | SE ±0.68 pp | 48 / 50 | $0.1 | 131.26source duration | [Vals AI · 2](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 7.448 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.097081 \| USD \| per_test \| Lower is better \| \| `latency` \| 131.26 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Inception reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.68 pp sourcePopulation 80 stderr 0.684 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) Source Vals AI Original model label inception/mercury-2.5 Source version 2 Observed 2026-10-22T02:10:12.404Z Evidence SHA-256 `f731b20a26c126725d125519f2b5841aa45254656dba859a9570d7ce7e5d8191` Source locator `/tasks/precedents/inception~1mercury-2.5` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / criteria pass rate corporate ma](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__corporate_ma.md) | 51.28% | SE ±2.83 pp | 46 / 49 | $0.2 | 97.04source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 51.282 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.197043 \| USD \| per_test \| Lower is better \| \| `latency` \| 97.04 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Inception reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.83 pp sourcePopulation 81 stderr 2.834 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label inception/mercury-2.5 Source version 1 Observed 2026-10-22T02:10:09.422Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__corporate_ma/inception~1mercury-2.5` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / tax](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Atax.md) | 0% | SE ±0 pp | 3 / 49 | $0.18 | 75.26source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.179616 \| USD \| per_test \| Lower is better \| \| `latency` \| 75.256 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Inception reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 81 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label inception/mercury-2.5 Source version 1 Observed 2026-10-22T02:10:09.422Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/tax/inception~1mercury-2.5` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / bankruptcy restructuring](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Abankruptcy_restructuring.md) | 0% | SE ±0 pp | 3 / 49 | $0.23 | 170.69source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.234753 \| USD \| per_test \| Lower is better \| \| `latency` \| 170.688 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Inception reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 81 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label inception/mercury-2.5 Source version 1 Observed 2026-10-22T02:10:09.422Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/bankruptcy_restructuring/inception~1mercury-2.5` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / emerging companies venture capital](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Aemerging_companies_venture_capital.md) | 0% | SE ±0 pp | 18 / 49 | $0.15 | 70.56source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.150699 \| USD \| per_test \| Lower is better \| \| `latency` \| 70.562 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Inception reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 81 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label inception/mercury-2.5 Source version 1 Observed 2026-10-22T02:10:09.422Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/emerging_companies_venture_capital/inception~1mercury-2.5` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / corporate ma](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acorporate_ma.md) | 0% | SE ±0 pp | 21 / 49 | $0.2 | 97.04source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.197043 \| USD \| per_test \| Lower is better \| \| `latency` \| 97.04 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Inception reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 81 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label inception/mercury-2.5 Source version 1 Observed 2026-10-22T02:10:09.422Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/corporate_ma/inception~1mercury-2.5` | | | | | | | | | [LegalBench / rule tasks](https://benchize.com/benchmarks/vals-ai%3Alegal_bench%3Arule_tasks.md) | 75.48% | SE ±2 pp | 32 / 44 | $0.000731 | 4.52source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_bench.D6uTgA3b.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 75.484 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.000731 \| USD \| per_test \| Lower is better \| \| `latency` \| 4.523 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Inception reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2 pp sourcePopulation 151 stderr 2 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_bench.D6uTgA3b.json) Source Vals AI Original model label inception/mercury-2.5 Source version 1 Observed 2026-10-22T02:10:08.909Z Evidence SHA-256 `c8486ba756baa5cd3955d035edff67440e8db23e69d62ad1a9e18d15b8c4bc52` Source locator `/tasks/rule_tasks/inception~1mercury-2.5` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / criteria pass rate international trade sanctions](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__international_trade_sanctions.md) | 59.52% | SE ±3.1 pp | 44 / 49 | $0.18 | 86.68source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 59.524 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.180981 \| USD \| per_test \| Lower is better \| \| `latency` \| 86.68 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Inception reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±3.1 pp sourcePopulation 81 stderr 3.098 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label inception/mercury-2.5 Source version 1 Observed 2026-10-22T02:10:09.422Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__international_trade_sanctions/inception~1mercury-2.5` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / real estate](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Areal_estate.md) | 0% | SE ±0 pp | 6 / 49 | $0.2 | 286.22source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.203965 \| USD \| per_test \| Lower is better \| \| `latency` \| 286.224 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Inception reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 81 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label inception/mercury-2.5 Source version 1 Observed 2026-10-22T02:10:09.422Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/real_estate/inception~1mercury-2.5` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / criteria pass rate tax](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__tax.md) | 27.43% | SE ±2.58 pp | 48 / 49 | $0.18 | 75.26source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 27.425 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.179616 \| USD \| per_test \| Lower is better \| \| `latency` \| 75.256 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Inception reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.58 pp sourcePopulation 81 stderr 2.584 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label inception/mercury-2.5 Source version 1 Observed 2026-10-22T02:10:09.422Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__tax/inception~1mercury-2.5` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / criteria pass rate litigation dispute resolution](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__litigation_dispute_resolution.md) | 31.46% | SE ±2.85 pp | 48 / 49 | $0.2 | 130.01source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 31.461 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.199512 \| USD \| per_test \| Lower is better \| \| `latency` \| 130.01 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Inception reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.85 pp sourcePopulation 81 stderr 2.847 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label inception/mercury-2.5 Source version 1 Observed 2026-10-22T02:10:09.422Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__litigation_dispute_resolution/inception~1mercury-2.5` | | | | | | | | | [Code Migration / cobol](https://benchize.com/benchmarks/vals-ai%3Acode-migration%3Acobol.md) | 0% | SE ±0 pp | 46 / 47 | $0.94 | 1,614.95source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_code_migration.Dttzoqk3.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.94276 \| USD \| per_test \| Lower is better \| \| `latency` \| 1614.952 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Inception reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 79 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_code_migration.Dttzoqk3.json) Source Vals AI Original model label inception/mercury-2.5 Source version 1 Observed 2026-10-22T02:10:27.307Z Evidence SHA-256 `99194597f2c76c402195be47839409046e18bd30454616da6dd40b736a088fa4` Source locator `/tasks/cobol/inception~1mercury-2.5` | | | | | | | | | [Legal Research Bench / weighted pass civil litigation](https://benchize.com/benchmarks/vals-ai%3Alegal_research%3Aweighted_pass__civil_litigation.md) | 65.55% | SE ±5.36 pp | 48 / 50 | $0.06 | 47.03source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 65.55 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.064712 \| USD \| per_test \| Lower is better \| \| `latency` \| 47.033 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Inception reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±5.36 pp sourcePopulation 82 stderr 5.36 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) Source Vals AI Original model label inception/mercury-2.5 Source version 1 Observed 2026-10-22T02:10:07.934Z Evidence SHA-256 `26c58dfd47d0d2dc86f33c6d056f30309e0ee63570e7cd752c593b385f9d862e` Source locator `/tasks/weighted_pass__civil_litigation/inception~1mercury-2.5` | | | | | | | | | [ProofBench v1.1 / overall](https://benchize.com/benchmarks/vals-ai%3Aproof_bench%3Aoverall.md) | 3% | SE ±1.71 pp | 34 / 34 | $0.04 | 201.73source duration | [Vals AI · 1.1](https://www.vals.ai/_astro/benchmark_view_proof_bench.DKi8pOIu.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 3 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.044105 \| USD \| per_test \| Lower is better \| \| `latency` \| 201.725 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Inception reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.71 pp sourcePopulation 53 stderr 1.714 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_proof_bench.DKi8pOIu.json) Source Vals AI Original model label inception/mercury-2.5 Source version 1.1 Observed 2026-10-22T02:10:16.653Z Evidence SHA-256 `79cc4dccb4f7f5ec4d61d3df0449a4a8712041cd70525ed36d545776eb537c24` Source locator `/tasks/overall/inception~1mercury-2.5` | | | | | | | | | [EMB / template](https://benchize.com/benchmarks/vals-ai%3Aemb%3Atemplate.md) | 4.83% | SE ±1.65 pp | 47 / 47 | $0.43 | 326.51source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 4.833 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.429915 \| USD \| per_test \| Lower is better \| \| `latency` \| 326.512 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Inception reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.65 pp sourcePopulation 76 stderr 1.652 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) Source Vals AI Original model label inception/mercury-2.5 Source version 1 Observed 2026-10-22T02:10:12.907Z Evidence SHA-256 `d807aee620ea47152a1fca0257a8dd12a1be4512c76cc1744c1ae9e285180918` Source locator `/tasks/template/inception~1mercury-2.5` | | | | | | | | | [Terminal-Bench-Science / physical sciences](https://benchize.com/benchmarks/vals-ai%3Aterminal-bench-science%3Aphysical_sciences.md) | 0% | SE ±0 pp | 24 / 30 | — | — | [Vals AI · 0.1](https://www.vals.ai/_astro/benchmark_view_terminal_bench_science.w7DPgclX.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| ### Configuration computeEffort Not reported harness Mini-SWE-agent maxOutputTokens 65536 provider Inception reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 42 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_terminal_bench_science.w7DPgclX.json) Source Vals AI Original model label inception/mercury-2.5 Source version 0.1 Observed 2026-10-22T02:10:18.658Z Evidence SHA-256 `40d2823a3a4edd8a0b135a7e4de390767f72d6fafe5c78e2055218a3f49e99c9` Source locator `/tasks/physical_sciences/inception~1mercury-2.5` | | | | | | | | | [IOI / ioi2025](https://benchize.com/benchmarks/vals-ai%3Aioi%3Aioi2025.md) | 3.5% | SE ±2.69 pp | 29 / 29 | $0.19 | 239.92source duration | [Vals AI · 2](https://www.vals.ai/_astro/benchmark_view_ioi.DV1xD5vC.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 3.5 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.187303 \| USD \| per_test \| Lower is better \| \| `latency` \| 239.919 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Inception reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.69 pp sourcePopulation 45 stderr 2.685 tokenTotals {"cache_read_tokens":7109497,"cache_write_tokens":0,"input_tokens":11283658,"output_tokens":173264,"reasoning_tokens":165909} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_ioi.DV1xD5vC.json) Source Vals AI Original model label inception/mercury-2.5 Source version 2 Observed 2026-10-22T02:10:26.312Z Evidence SHA-256 `63a24003cd68e5620c2ddb114b077adec8c5e5bf5916bb04ced543c48d731a45` Source locator `/tasks/ioi2025/inception~1mercury-2.5` | | | | | | | | | [LegalBench / rhetoric tasks](https://benchize.com/benchmarks/vals-ai%3Alegal_bench%3Arhetoric_tasks.md) | 80.31% | SE ±0.86 pp | 24 / 44 | $0.00084 | 4.26source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_bench.D6uTgA3b.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 80.314 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.00084 \| USD \| per_test \| Lower is better \| \| `latency` \| 4.26 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Inception reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.86 pp sourcePopulation 151 stderr 0.861 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_bench.D6uTgA3b.json) Source Vals AI Original model label inception/mercury-2.5 Source version 1 Observed 2026-10-22T02:10:08.909Z Evidence SHA-256 `c8486ba756baa5cd3955d035edff67440e8db23e69d62ad1a9e18d15b8c4bc52` Source locator `/tasks/rhetoric_tasks/inception~1mercury-2.5` | | | | | | | | | [Legal Research Bench / weighted pass family](https://benchize.com/benchmarks/vals-ai%3Alegal_research%3Aweighted_pass__family.md) | 37.56% | SE ±5.48 pp | 48 / 50 | $0.06 | 47.19source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 37.561 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.062993 \| USD \| per_test \| Lower is better \| \| `latency` \| 47.19 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Inception reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±5.48 pp sourcePopulation 82 stderr 5.479 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) Source Vals AI Original model label inception/mercury-2.5 Source version 1 Observed 2026-10-22T02:10:07.934Z Evidence SHA-256 `26c58dfd47d0d2dc86f33c6d056f30309e0ee63570e7cd752c593b385f9d862e` Source locator `/tasks/weighted_pass__family/inception~1mercury-2.5` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / data privacy cybersecurity](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Adata_privacy_cybersecurity.md) | 0% | SE ±0 pp | 13 / 49 | $0.15 | 77.59source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.14917 \| USD \| per_test \| Lower is better \| \| `latency` \| 77.594 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Inception reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 81 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label inception/mercury-2.5 Source version 1 Observed 2026-10-22T02:10:09.422Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/data_privacy_cybersecurity/inception~1mercury-2.5` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / capital markets](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acapital_markets.md) | 0% | SE ±0 pp | 16 / 49 | $0.12 | 64.01source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.121463 \| USD \| per_test \| Lower is better \| \| `latency` \| 64.014 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Inception reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 81 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label inception/mercury-2.5 Source version 1 Observed 2026-10-22T02:10:09.422Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/capital_markets/inception~1mercury-2.5` | | | | | | | | | [Finance Agent (v2) / comparables](https://benchize.com/benchmarks/vals-ai%3Afabv2%3Acomparables.md) | 25.5% | SE ±2.55 pp | 48 / 50 | $0.12 | 170.84source duration | [Vals AI · 2](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 25.496 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.115052 \| USD \| per_test \| Lower is better \| \| `latency` \| 170.84 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Inception reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.55 pp sourcePopulation 80 stderr 2.549 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) Source Vals AI Original model label inception/mercury-2.5 Source version 2 Observed 2026-10-22T02:10:12.404Z Evidence SHA-256 `f731b20a26c126725d125519f2b5841aa45254656dba859a9570d7ce7e5d8191` Source locator `/tasks/comparables/inception~1mercury-2.5` | | | | | | | | | [Terminal-Bench 4.0 / media](https://benchize.com/benchmarks/vals-ai%3Aterminal-bench-4%3Amedia.md) | 0% | SE ±0 pp | 24 / 36 | — | — | [Vals AI · 4.0](https://www.vals.ai/_astro/benchmark_view_terminal_bench_4.CAnGyzC_.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Inception reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 52 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_terminal_bench_4.CAnGyzC_.json) Source Vals AI Original model label inception/mercury-2.5 Source version 4.0 Observed 2026-10-22T02:10:25.308Z Evidence SHA-256 `98f6964580e901f1aa143db664bd92aa558415f643569492545d67c47d7f519b` Source locator `/tasks/media/inception~1mercury-2.5` | | | | | | | | | [Tax Agent Bench / fact pattern analysis](https://benchize.com/benchmarks/vals-ai%3Atax_agent_bench%3Afact_pattern_analysis.md) | 2.33% | SE ±2.33 pp | 45 / 46 | $0.02 | 34.14source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 2.326 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.024313 \| USD \| per_test \| Lower is better \| \| `latency` \| 34.135 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Inception reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.33 pp sourcePopulation 73 stderr 2.326 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) Source Vals AI Original model label inception/mercury-2.5 Source version 1 Observed 2026-10-22T02:10:11.407Z Evidence SHA-256 `446db182b67efb7923b3187a4040ec3a3ea492b8ad699a2b6d708d714cc063ac` Source locator `/tasks/fact_pattern_analysis/inception~1mercury-2.5` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / energy natural resources](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Aenergy_natural_resources.md) | 0% | SE ±0 pp | 28 / 49 | $0.16 | 382.35source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.164729 \| USD \| per_test \| Lower is better \| \| `latency` \| 382.348 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Inception reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 81 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label inception/mercury-2.5 Source version 1 Observed 2026-10-22T02:10:09.422Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/energy_natural_resources/inception~1mercury-2.5` | | | | | | | | | [Terminal-Bench 2.1 / medium](https://benchize.com/benchmarks/vals-ai%3Aterminal-bench-2-1%3Amedium.md) | 44.24% | SE ±0.61 pp | 39 / 43 | $0.35 | 1,114.8source duration | [Vals AI · 2.1](https://www.vals.ai/_astro/benchmark_view_terminal_bench_2_1.CdyoObsA.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 44.242 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.347888 \| USD \| per_test \| Lower is better \| \| `latency` \| 1114.803 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Inception reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.61 pp sourcePopulation 76 stderr 0.606 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_terminal_bench_2_1.CdyoObsA.json) Source Vals AI Original model label inception/mercury-2.5 Source version 2.1 Observed 2026-10-22T02:10:27.811Z Evidence SHA-256 `a5306805cdf86495bdcceff6b2c8534a3768b9853f5077895dab975b809b054f` Source locator `/tasks/medium/inception~1mercury-2.5` | | | | | | | | | [Tax Agent Bench / weighted pass rule source lookup](https://benchize.com/benchmarks/vals-ai%3Atax_agent_bench%3Aweighted_pass__rule_source_lookup.md) | 8.28% | SE ±3.98 pp | 46 / 46 | $0.02 | 29.33source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 8.279 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.021089 \| USD \| per_test \| Lower is better \| \| `latency` \| 29.325 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Inception reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±3.98 pp sourcePopulation 73 stderr 3.981 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) Source Vals AI Original model label inception/mercury-2.5 Source version 1 Observed 2026-10-22T02:10:11.407Z Evidence SHA-256 `446db182b67efb7923b3187a4040ec3a3ea492b8ad699a2b6d708d714cc063ac` Source locator `/tasks/weighted_pass__rule_source_lookup/inception~1mercury-2.5` | | | | | | | | | [Terminal-Bench 2.1 / easy](https://benchize.com/benchmarks/vals-ai%3Aterminal-bench-2-1%3Aeasy.md) | 66.67% | SE ±8.33 pp | 37 / 43 | $4.78 | 15,328.54source duration | [Vals AI · 2.1](https://www.vals.ai/_astro/benchmark_view_terminal_bench_2_1.CdyoObsA.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 66.667 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 4.783458 \| USD \| per_test \| Lower is better \| \| `latency` \| 15328.538 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Inception reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±8.33 pp sourcePopulation 76 stderr 8.333 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_terminal_bench_2_1.CdyoObsA.json) Source Vals AI Original model label inception/mercury-2.5 Source version 2.1 Observed 2026-10-22T02:10:27.811Z Evidence SHA-256 `a5306805cdf86495bdcceff6b2c8534a3768b9853f5077895dab975b809b054f` Source locator `/tasks/easy/inception~1mercury-2.5` | | | | | | | | | [Terminal-Bench-Science / engineering sciences](https://benchize.com/benchmarks/vals-ai%3Aterminal-bench-science%3Aengineering_sciences.md) | 0% | SE ±0 pp | 12 / 30 | — | — | [Vals AI · 0.1](https://www.vals.ai/_astro/benchmark_view_terminal_bench_science.w7DPgclX.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| ### Configuration computeEffort Not reported harness Mini-SWE-agent maxOutputTokens 65536 provider Inception reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 42 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_terminal_bench_science.w7DPgclX.json) Source Vals AI Original model label inception/mercury-2.5 Source version 0.1 Observed 2026-10-22T02:10:18.658Z Evidence SHA-256 `40d2823a3a4edd8a0b135a7e4de390767f72d6fafe5c78e2055218a3f49e99c9` Source locator `/tasks/engineering_sciences/inception~1mercury-2.5` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / criteria pass rate immigration](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__immigration.md) | 36.73% | SE ±2.91 pp | 45 / 49 | $0.15 | 77.05source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 36.727 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.149835 \| USD \| per_test \| Lower is better \| \| `latency` \| 77.046 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Inception reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.91 pp sourcePopulation 81 stderr 2.912 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label inception/mercury-2.5 Source version 1 Observed 2026-10-22T02:10:09.422Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__immigration/inception~1mercury-2.5` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / antitrust competition](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Aantitrust_competition.md) | 0% | SE ±0 pp | 7 / 49 | $0.13 | 116.35source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.128388 \| USD \| per_test \| Lower is better \| \| `latency` \| 116.346 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Inception reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 81 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label inception/mercury-2.5 Source version 1 Observed 2026-10-22T02:10:09.422Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/antitrust_competition/inception~1mercury-2.5` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / white collar defense investigations](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Awhite_collar_defense_investigations.md) | 0% | SE ±0 pp | 1 / 49 | $0.18 | 119.38source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.176704 \| USD \| per_test \| Lower is better \| \| `latency` \| 119.378 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Inception reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 81 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label inception/mercury-2.5 Source version 1 Observed 2026-10-22T02:10:09.422Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/white_collar_defense_investigations/inception~1mercury-2.5` | | | | | | | | | [Finance Agent (v2) / overall](https://benchize.com/benchmarks/vals-ai%3Afabv2%3Aoverall.md) | 18.89% | SE ±0.22 pp | 48 / 50 | $0.11 | 92.48source duration | [Vals AI · 2](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 18.894 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.106441 \| USD \| per_test \| Lower is better \| \| `latency` \| 92.482 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Inception reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.22 pp sourcePopulation 80 stderr 0.216 tokenTotals {"cache_read_tokens":191920653,"cache_write_tokens":null,"input_tokens":376083870,"output_tokens":5831330,"reasoning_tokens":4955938} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) Source Vals AI Original model label inception/mercury-2.5 Source version 2 Observed 2026-10-22T02:10:12.404Z Evidence SHA-256 `f731b20a26c126725d125519f2b5841aa45254656dba859a9570d7ce7e5d8191` Source locator `/tasks/overall/inception~1mercury-2.5` | | | | | | | | | [Terminal-Bench-Science / life sciences](https://benchize.com/benchmarks/vals-ai%3Aterminal-bench-science%3Alife_sciences.md) | 0% | SE ±0 pp | 24 / 30 | — | — | [Vals AI · 0.1](https://www.vals.ai/_astro/benchmark_view_terminal_bench_science.w7DPgclX.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| ### Configuration computeEffort Not reported harness Mini-SWE-agent maxOutputTokens 65536 provider Inception reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 42 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_terminal_bench_science.w7DPgclX.json) Source Vals AI Original model label inception/mercury-2.5 Source version 0.1 Observed 2026-10-22T02:10:18.658Z Evidence SHA-256 `40d2823a3a4edd8a0b135a7e4de390767f72d6fafe5c78e2055218a3f49e99c9` Source locator `/tasks/life_sciences/inception~1mercury-2.5` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / criteria pass rate corporate governance](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__corporate_governance.md) | 43.84% | SE ±2.91 pp | 47 / 49 | $0.27 | 105.77source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 43.836 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.271182 \| USD \| per_test \| Lower is better \| \| `latency` \| 105.77 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Inception reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.91 pp sourcePopulation 81 stderr 2.909 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label inception/mercury-2.5 Source version 1 Observed 2026-10-22T02:10:09.422Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__corporate_governance/inception~1mercury-2.5` | | | | | | | | | [EMB / dcf](https://benchize.com/benchmarks/vals-ai%3Aemb%3Adcf.md) | 4.72% | SE ±1.92 pp | 46 / 47 | $0.41 | 280.22source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 4.721 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.411431 \| USD \| per_test \| Lower is better \| \| `latency` \| 280.224 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Inception reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.92 pp sourcePopulation 76 stderr 1.918 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) Source Vals AI Original model label inception/mercury-2.5 Source version 1 Observed 2026-10-22T02:10:12.907Z Evidence SHA-256 `d807aee620ea47152a1fca0257a8dd12a1be4512c76cc1744c1ae9e285180918` Source locator `/tasks/dcf/inception~1mercury-2.5` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / intellectual property](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Aintellectual_property.md) | 0% | SE ±0 pp | 6 / 49 | $0.21 | 91.85source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.212309 \| USD \| per_test \| Lower is better \| \| `latency` \| 91.85 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Inception reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 81 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label inception/mercury-2.5 Source version 1 Observed 2026-10-22T02:10:09.422Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/intellectual_property/inception~1mercury-2.5` | | | | | | | | | [MedCode / overall](https://benchize.com/benchmarks/vals-ai%3Amedcode%3Aoverall.md) | 31.33% | SE ±1.95 pp | 44 / 49 | $0.00454 | 7.93source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_medcode.DehpEDgG.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 31.326 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.004535 \| USD \| per_test \| Lower is better \| \| `latency` \| 7.93 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Inception reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.95 pp sourcePopulation 107 stderr 1.953 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_medcode.DehpEDgG.json) Source Vals AI Original model label inception/mercury-2.5 Source version 1 Observed 2026-10-22T02:10:14.899Z Evidence SHA-256 `988a6ae1e07e33ce031572ed4a804be11c36ee97a1dd75cbe4740fff3b534f8c` Source locator `/tasks/overall/inception~1mercury-2.5` | | | | | | | | | [Terminal-Bench 4.0 / ml](https://benchize.com/benchmarks/vals-ai%3Aterminal-bench-4%3Aml.md) | 0% | SE ±0 pp | 33 / 36 | — | — | [Vals AI · 4.0](https://www.vals.ai/_astro/benchmark_view_terminal_bench_4.CAnGyzC_.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Inception reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 52 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_terminal_bench_4.CAnGyzC_.json) Source Vals AI Original model label inception/mercury-2.5 Source version 4.0 Observed 2026-10-22T02:10:25.308Z Evidence SHA-256 `98f6964580e901f1aa143db664bd92aa558415f643569492545d67c47d7f519b` Source locator `/tasks/ml/inception~1mercury-2.5` | | | | | | | | | [Terminal-Bench 4.0 / operations](https://benchize.com/benchmarks/vals-ai%3Aterminal-bench-4%3Aoperations.md) | 0% | SE ±0 pp | 34 / 36 | — | — | [Vals AI · 4.0](https://www.vals.ai/_astro/benchmark_view_terminal_bench_4.CAnGyzC_.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Inception reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 52 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_terminal_bench_4.CAnGyzC_.json) Source Vals AI Original model label inception/mercury-2.5 Source version 4.0 Observed 2026-10-22T02:10:25.308Z Evidence SHA-256 `98f6964580e901f1aa143db664bd92aa558415f643569492545d67c47d7f519b` Source locator `/tasks/operations/inception~1mercury-2.5` | | | | | | | | | [Finance Agent (v2) / disclosure analysis](https://benchize.com/benchmarks/vals-ai%3Afabv2%3Adisclosure_analysis.md) | 19.06% | SE ±1.36 pp | 48 / 50 | $0.11 | 119.47source duration | [Vals AI · 2](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 19.063 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.111069 \| USD \| per_test \| Lower is better \| \| `latency` \| 119.47 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Inception reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.36 pp sourcePopulation 80 stderr 1.364 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) Source Vals AI Original model label inception/mercury-2.5 Source version 2 Observed 2026-10-22T02:10:12.404Z Evidence SHA-256 `f731b20a26c126725d125519f2b5841aa45254656dba859a9570d7ce7e5d8191` Source locator `/tasks/disclosure_analysis/inception~1mercury-2.5` | | | | | | | | | [CyberBench v1.1 / overall](https://benchize.com/benchmarks/benchmark%3Acyber%3Aoverall.md) | 45.48% | SE ±6.06 pp | 23 / 28 | $0.12 | 1,139.17source duration | [Vals AI · 1.1](https://www.vals.ai/_astro/benchmark_view_cyberbench.DS6UOlu0.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 45.476 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.12462 \| USD \| per_test \| Lower is better \| \| `latency` \| 1139.168 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Inception reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±6.06 pp sourcePopulation 47 stderr 6.056 tokenTotals {"cache_read_tokens":138406142,"cache_write_tokens":null,"input_tokens":157635879,"output_tokens":814145,"reasoning_tokens":748273} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_cyberbench.DS6UOlu0.json) Source Vals AI Original model label inception/mercury-2.5 Source version 1.1 Observed 2026-10-22T02:10:29.837Z Evidence SHA-256 `943f593f929f8971a6b746269a2851b5febc2aad4e2f1066895f715033cdc7f1` Source locator `/tasks/overall/inception~1mercury-2.5` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / overall](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Aoverall.md) | 0% | SE ±0 pp | 42 / 49 | $0.19 | 131.54source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.192614 \| USD \| per_test \| Lower is better \| \| `latency` \| 131.54 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Inception reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 81 stderr 0 tokenTotals {"cache_read_tokens":55653173,"cache_write_tokens":null,"input_tokens":159575776,"output_tokens":1621389,"reasoning_tokens":1524161} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label inception/mercury-2.5 Source version 1 Observed 2026-10-22T02:10:09.422Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/overall/inception~1mercury-2.5` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / criteria pass rate real estate](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__real_estate.md) | 45.19% | SE ±2.69 pp | 45 / 49 | $0.2 | 286.22source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 45.19 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.203965 \| USD \| per_test \| Lower is better \| \| `latency` \| 286.224 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Inception reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.69 pp sourcePopulation 81 stderr 2.691 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label inception/mercury-2.5 Source version 1 Observed 2026-10-22T02:10:09.422Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__real_estate/inception~1mercury-2.5` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / criteria pass rate environmental esg](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__environmental_esg.md) | 43.01% | SE ±2.97 pp | 45 / 49 | $0.22 | 86.02source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 43.011 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.222426 \| USD \| per_test \| Lower is better \| \| `latency` \| 86.018 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Inception reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.97 pp sourcePopulation 81 stderr 2.969 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label inception/mercury-2.5 Source version 1 Observed 2026-10-22T02:10:09.422Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__environmental_esg/inception~1mercury-2.5` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / criteria pass rate banking finance](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__banking_finance.md) | 36.8% | SE ±2.95 pp | 48 / 49 | $0.12 | 189.08source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 36.803 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.12378 \| USD \| per_test \| Lower is better \| \| `latency` \| 189.08 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Inception reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.95 pp sourcePopulation 81 stderr 2.946 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label inception/mercury-2.5 Source version 1 Observed 2026-10-22T02:10:09.422Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__banking_finance/inception~1mercury-2.5` | | | | | | | | | [Terminal-Bench 4.0 / hardware](https://benchize.com/benchmarks/vals-ai%3Aterminal-bench-4%3Ahardware.md) | 0% | SE ±0 pp | 21 / 36 | — | — | [Vals AI · 4.0](https://www.vals.ai/_astro/benchmark_view_terminal_bench_4.CAnGyzC_.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Inception reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 52 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_terminal_bench_4.CAnGyzC_.json) Source Vals AI Original model label inception/mercury-2.5 Source version 4.0 Observed 2026-10-22T02:10:25.308Z Evidence SHA-256 `98f6964580e901f1aa143db664bd92aa558415f643569492545d67c47d7f519b` Source locator `/tasks/hardware/inception~1mercury-2.5` | | | | | | | | | [EMB / scratch](https://benchize.com/benchmarks/vals-ai%3Aemb%3Ascratch.md) | 12.44% | SE ±1.85 pp | 46 / 47 | $0.41 | 275source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 12.442 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.4053 \| USD \| per_test \| Lower is better \| \| `latency` \| 275.003 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Inception reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.85 pp sourcePopulation 76 stderr 1.846 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) Source Vals AI Original model label inception/mercury-2.5 Source version 1 Observed 2026-10-22T02:10:12.907Z Evidence SHA-256 `d807aee620ea47152a1fca0257a8dd12a1be4512c76cc1744c1ae9e285180918` Source locator `/tasks/scratch/inception~1mercury-2.5` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / criteria pass rate white collar defense investigations](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__white_collar_defense_investigations.md) | 41.08% | SE ±2.86 pp | 46 / 49 | $0.18 | 119.38source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 41.077 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.176704 \| USD \| per_test \| Lower is better \| \| `latency` \| 119.378 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Inception reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.86 pp sourcePopulation 81 stderr 2.86 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label inception/mercury-2.5 Source version 1 Observed 2026-10-22T02:10:09.422Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__white_collar_defense_investigations/inception~1mercury-2.5` | | | | | | | | | [Code Migration / overall](https://benchize.com/benchmarks/vals-ai%3Acode-migration%3Aoverall.md) | 4.52% | SE ±1.05 pp | 47 / 47 | $0.63 | 974.08source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_code_migration.Dttzoqk3.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 4.517 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.627223 \| USD \| per_test \| Lower is better \| \| `latency` \| 974.083 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Inception reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.05 pp sourcePopulation 79 stderr 1.048 tokenTotals {"cache_read_tokens":735400505,"cache_write_tokens":null,"input_tokens":959228311,"output_tokens":6764397,"reasoning_tokens":6460465} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_code_migration.Dttzoqk3.json) Source Vals AI Original model label inception/mercury-2.5 Source version 1 Observed 2026-10-22T02:10:27.307Z Evidence SHA-256 `99194597f2c76c402195be47839409046e18bd30454616da6dd40b736a088fa4` Source locator `/tasks/overall/inception~1mercury-2.5` | | | | | | | | | [Legal Research Bench / weighted pass immigration](https://benchize.com/benchmarks/vals-ai%3Alegal_research%3Aweighted_pass__immigration.md) | 64.38% | SE ±7.91 pp | 46 / 50 | $0.05 | 38.57source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 64.376 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.046126 \| USD \| per_test \| Lower is better \| \| `latency` \| 38.568 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Inception reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±7.91 pp sourcePopulation 82 stderr 7.908 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) Source Vals AI Original model label inception/mercury-2.5 Source version 1 Observed 2026-10-22T02:10:07.934Z Evidence SHA-256 `26c58dfd47d0d2dc86f33c6d056f30309e0ee63570e7cd752c593b385f9d862e` Source locator `/tasks/weighted_pass__immigration/inception~1mercury-2.5` | | | | | | | | | [EMB / overall](https://benchize.com/benchmarks/vals-ai%3Aemb%3Aoverall.md) | 8.79% | SE ±1.28 pp | 46 / 47 | $0.42 | 300.76source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 8.789 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.417608 \| USD \| per_test \| Lower is better \| \| `latency` \| 300.757 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Inception reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.28 pp sourcePopulation 76 stderr 1.275 tokenTotals {"cache_read_tokens":238888010,"cache_write_tokens":null,"input_tokens":407236592,"output_tokens":5531340,"reasoning_tokens":5399594} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) Source Vals AI Original model label inception/mercury-2.5 Source version 1 Observed 2026-10-22T02:10:12.907Z Evidence SHA-256 `d807aee620ea47152a1fca0257a8dd12a1be4512c76cc1744c1ae9e285180918` Source locator `/tasks/overall/inception~1mercury-2.5` | | | | | | | | | [Terminal-Bench 4.0 / software](https://benchize.com/benchmarks/vals-ai%3Aterminal-bench-4%3Asoftware.md) | 0% | SE ±0 pp | 36 / 36 | — | — | [Vals AI · 4.0](https://www.vals.ai/_astro/benchmark_view_terminal_bench_4.CAnGyzC_.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Inception reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 52 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_terminal_bench_4.CAnGyzC_.json) Source Vals AI Original model label inception/mercury-2.5 Source version 4.0 Observed 2026-10-22T02:10:25.308Z Evidence SHA-256 `98f6964580e901f1aa143db664bd92aa558415f643569492545d67c47d7f519b` Source locator `/tasks/software/inception~1mercury-2.5` | | | | | | | | | [Finance Agent (v2) / market analysis](https://benchize.com/benchmarks/vals-ai%3Afabv2%3Amarket_analysis.md) | 26.62% | SE ±0.4 pp | 48 / 50 | $0.08 | 71.92source duration | [Vals AI · 2](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 26.624 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.082934 \| USD \| per_test \| Lower is better \| \| `latency` \| 71.922 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Inception reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.4 pp sourcePopulation 80 stderr 0.4 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) Source Vals AI Original model label inception/mercury-2.5 Source version 2 Observed 2026-10-22T02:10:12.404Z Evidence SHA-256 `f731b20a26c126725d125519f2b5841aa45254656dba859a9570d7ce7e5d8191` Source locator `/tasks/market_analysis/inception~1mercury-2.5` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / immigration](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Aimmigration.md) | 0% | SE ±0 pp | 6 / 49 | $0.15 | 77.05source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.149835 \| USD \| per_test \| Lower is better \| \| `latency` \| 77.046 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Inception reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 81 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label inception/mercury-2.5 Source version 1 Observed 2026-10-22T02:10:09.422Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/immigration/inception~1mercury-2.5` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / criteria pass rate data privacy cybersecurity](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__data_privacy_cybersecurity.md) | 47.35% | SE ±2.97 pp | 46 / 49 | $0.15 | 77.59source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 47.35 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.14917 \| USD \| per_test \| Lower is better \| \| `latency` \| 77.594 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Inception reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.97 pp sourcePopulation 81 stderr 2.973 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label inception/mercury-2.5 Source version 1 Observed 2026-10-22T02:10:09.422Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__data_privacy_cybersecurity/inception~1mercury-2.5` | | | | | | | | | [Terminal-Bench-Science / overall](https://benchize.com/benchmarks/vals-ai%3Aterminal-bench-science%3Aoverall.md) | 0% | SE ±0 pp | 28 / 30 | $0.18 | 1,693.95source duration | [Vals AI · 0.1](https://www.vals.ai/_astro/benchmark_view_terminal_bench_science.w7DPgclX.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.17804 \| USD \| per_test \| Lower is better \| \| `latency` \| 1693.946 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Mini-SWE-agent maxOutputTokens 65536 provider Inception reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 42 stderr 0 tokenTotals {"cache_read_tokens":228897807,"cache_write_tokens":0,"input_tokens":485663780,"output_tokens":6077416,"reasoning_tokens":2995809} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_terminal_bench_science.w7DPgclX.json) Source Vals AI Original model label inception/mercury-2.5 Source version 0.1 Observed 2026-10-22T02:10:18.658Z Evidence SHA-256 `40d2823a3a4edd8a0b135a7e4de390767f72d6fafe5c78e2055218a3f49e99c9` Source locator `/tasks/overall/inception~1mercury-2.5` | | | | | | | | | [Legal Research Bench / civil litigation](https://benchize.com/benchmarks/vals-ai%3Alegal_research%3Acivil_litigation.md) | 3.7% | SE ±3.7 pp | 47 / 50 | $0.06 | 47.03source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 3.704 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.064712 \| USD \| per_test \| Lower is better \| \| `latency` \| 47.033 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Inception reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±3.7 pp sourcePopulation 82 stderr 3.704 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) Source Vals AI Original model label inception/mercury-2.5 Source version 1 Observed 2026-10-22T02:10:07.934Z Evidence SHA-256 `26c58dfd47d0d2dc86f33c6d056f30309e0ee63570e7cd752c593b385f9d862e` Source locator `/tasks/civil_litigation/inception~1mercury-2.5` | | | | | | | | | [Tax Agent Bench / weighted pass fact pattern analysis](https://benchize.com/benchmarks/vals-ai%3Atax_agent_bench%3Aweighted_pass__fact_pattern_analysis.md) | 11.67% | SE ±3.78 pp | 46 / 46 | $0.02 | 34.14source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 11.667 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.024313 \| USD \| per_test \| Lower is better \| \| `latency` \| 34.135 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Inception reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±3.78 pp sourcePopulation 73 stderr 3.775 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) Source Vals AI Original model label inception/mercury-2.5 Source version 1 Observed 2026-10-22T02:10:11.407Z Evidence SHA-256 `446db182b67efb7923b3187a4040ec3a3ea492b8ad699a2b6d708d714cc063ac` Source locator `/tasks/weighted_pass__fact_pattern_analysis/inception~1mercury-2.5` | | | | | | | | | [CyberBench v1.1 / patch](https://benchize.com/benchmarks/benchmark%3Acyber%3Apatch.md) | 64.29% | SE ±6.4 pp | 28 / 28 | $0.13 | 661.56source duration | [Vals AI · 1.1](https://www.vals.ai/_astro/benchmark_view_cyberbench.DS6UOlu0.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 64.286 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.126262 \| USD \| per_test \| Lower is better \| \| `latency` \| 661.559 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Inception reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±6.4 pp sourcePopulation 47 stderr 6.403 tokenTotals {"cache_read_tokens":145289216,"cache_write_tokens":null,"input_tokens":163464203,"output_tokens":706505,"reasoning_tokens":634831} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_cyberbench.DS6UOlu0.json) Source Vals AI Original model label inception/mercury-2.5 Source version 1.1 Observed 2026-10-22T02:10:29.837Z Evidence SHA-256 `943f593f929f8971a6b746269a2851b5febc2aad4e2f1066895f715033cdc7f1` Source locator `/tasks/patch/inception~1mercury-2.5` | | | | | | | | [Next page](https://benchize.com/models/mercury-2-5-high.md?cursor=a9f1e56cbf0f069fee42e2e091311d5c41d7eaec4d90a1b173b99f709876a081) --- Cite as: Mercury 2.5High — Benchize, https://benchize.com/models/mercury-2-5-high (retrieved 2026-10-22)