# DeepSeek V4 ProMax > Source: Benchize (https://benchize.com) — https://benchize.com/models/deepseek-v4-pro-max. When citing or reusing this content, credit "Benchize" and link to https://benchize.com/models/deepseek-v4-pro-max. DeepSeek Canonical URL: https://benchize.com/models/deepseek-v4-pro-max Last updated: Thu, 22 Oct 2026 07:10:12 GMT Reproduction and quotation must name Benchize, link to the canonical HTML URL, and retain individual source citations. DeepSeek ## Benchmarks | Benchmark | Score | ± | Rank | Cost | Latency | Source | | | --- | --- | --- | --- | --- | --- | --- | --- | | [Harvey's Legal Agent Benchmark v1 / criteria pass rate white collar defense investigations](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__white_collar_defense_investigations.md) | 83.5% | SE ±2.16 pp | 23 / 49 | $0.6 | 389.94source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 83.502 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.597321 \| USD \| per_test \| Lower is better \| \| `latency` \| 389.944 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.16 pp sourcePopulation 81 stderr 2.157 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label deepseek/deepseek-v4-pro Source version 1 Observed 2026-10-22T02:10:09.422Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__white_collar_defense_investigations/deepseek~1deepseek-v4-pro` | | | | | | | | | [MMLU Pro / physics](https://benchize.com/benchmarks/vals-ai%3Ammlu_pro%3Aphysics.md) | 93.46% | SE ±0.7 pp | 10 / 40 | $0.04 | 561.23source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_mmlu_pro.y416pXDi.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 93.457 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.038088 \| USD \| per_test \| Lower is better \| \| `latency` \| 561.23 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.7 pp sourcePopulation 138 stderr 0.701 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_mmlu_pro.y416pXDi.json) Source Vals AI Original model label deepseek/deepseek-v4-pro Source version 1 Observed 2026-10-22T02:10:21.194Z Evidence SHA-256 `dab025507f46c716dd8d145e90ec0a737d9c56219853f41e4b1cc3c5e8c222b4` Source locator `/tasks/physics/deepseek~1deepseek-v4-pro` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / emerging companies venture capital](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Aemerging_companies_venture_capital.md) | 0% | SE ±0 pp | 18 / 49 | $0.76 | 415.08source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.76098 \| USD \| per_test \| Lower is better \| \| `latency` \| 415.078 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 81 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label deepseek/deepseek-v4-pro Source version 1 Observed 2026-10-22T02:10:09.422Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/emerging_companies_venture_capital/deepseek~1deepseek-v4-pro` | | | | | | | | | [Legal Research Bench / criminal](https://benchize.com/benchmarks/vals-ai%3Alegal_research%3Acriminal.md) | 15.79% | SE ±4.87 pp | 39 / 50 | $0.62 | 1,004.09source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 15.789 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.615927 \| USD \| per_test \| Lower is better \| \| `latency` \| 1004.085 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±4.87 pp sourcePopulation 82 stderr 4.873 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) Source Vals AI Original model label deepseek/deepseek-v4-pro Source version 1 Observed 2026-10-22T02:10:07.934Z Evidence SHA-256 `26c58dfd47d0d2dc86f33c6d056f30309e0ee63570e7cd752c593b385f9d862e` Source locator `/tasks/criminal/deepseek~1deepseek-v4-pro` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / criteria pass rate capital markets](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__capital_markets.md) | 86.28% | SE ±2.16 pp | 33 / 49 | $0.76 | 350.4source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 86.275 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.764042 \| USD \| per_test \| Lower is better \| \| `latency` \| 350.396 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.16 pp sourcePopulation 81 stderr 2.159 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label deepseek/deepseek-v4-pro Source version 1 Observed 2026-10-22T02:10:09.422Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__capital_markets/deepseek~1deepseek-v4-pro` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / overall](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Aoverall.md) | 3.75% | SE ±1.43 pp | 26 / 49 | $0.74 | 401.52source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 3.75 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.737465 \| USD \| per_test \| Lower is better \| \| `latency` \| 401.522 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.43 pp sourcePopulation 81 stderr 1.431 tokenTotals {"cache_read_tokens":303855232,"cache_write_tokens":null,"input_tokens":321058309,"output_tokens":4167657,"reasoning_tokens":272693} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label deepseek/deepseek-v4-pro Source version 1 Observed 2026-10-22T02:10:09.422Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/overall/deepseek~1deepseek-v4-pro` | | | | | | | | | [AIME 2026](https://benchize.com/benchmarks/aime-2026.md) | 96.67% | 95% CI 93.46–99.88% | 6 / 21 | 0.08usd per problem | — | [matharena · aime--aime_2026](https://matharena.ai/competition_tables/aime--aime_2026) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 96.67 \| percent \| competition \| Higher is better \| \| `cost_per_problem` \| 0.082 \| usd_per_problem \| competition \| Lower is better \| ### Configuration competition aime--aime_2026 predicted false Reasoning effort max sourceModelName DeepSeek-v4-Pro (Max) sourceModelVersion deepseek/deepseek_v4_pro thinkingEnabled true ### Reported statistics Uncertainty 95% confidence interval: 93.46–99.88% confidenceInterval95 3.21 problems 30 ### Source [Original source](https://matharena.ai/competition_tables/aime--aime_2026) Source matharena Original model label DeepSeek-v4-Pro (Max) Source version aime--aime_2026 Observed 2026-10-22T05:30:13.366Z Evidence SHA-256 `58b3ac1e0805975eba566c493b8e0165d41eb7ca517218106e112529fb6342de` Source locator `/table/rows/8` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / employment labor](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Aemployment_labor.md) | 0% | SE ±0 pp | 1 / 49 | $0.69 | 465.32source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.691985 \| USD \| per_test \| Lower is better \| \| `latency` \| 465.318 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 81 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label deepseek/deepseek-v4-pro Source version 1 Observed 2026-10-22T02:10:09.422Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/employment_labor/deepseek~1deepseek-v4-pro` | | | | | | | | | [MMLU Pro / biology](https://benchize.com/benchmarks/vals-ai%3Ammlu_pro%3Abiology.md) | 92.89% | SE ±0.96 pp | 22 / 40 | $0.01 | 636.88source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_mmlu_pro.y416pXDi.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 92.887 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.012403 \| USD \| per_test \| Lower is better \| \| `latency` \| 636.88 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.96 pp sourcePopulation 138 stderr 0.96 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_mmlu_pro.y416pXDi.json) Source Vals AI Original model label deepseek/deepseek-v4-pro Source version 1 Observed 2026-10-22T02:10:21.194Z Evidence SHA-256 `dab025507f46c716dd8d145e90ec0a737d9c56219853f41e4b1cc3c5e8c222b4` Source locator `/tasks/biology/deepseek~1deepseek-v4-pro` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / arbitration international dispute resolution](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Aarbitration_international_dispute_resolution.md) | 0% | SE ±0 pp | 4 / 49 | $0.89 | 395.84source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.887292 \| USD \| per_test \| Lower is better \| \| `latency` \| 395.842 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 81 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label deepseek/deepseek-v4-pro Source version 1 Observed 2026-10-22T02:10:09.422Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/arbitration_international_dispute_resolution/deepseek~1deepseek-v4-pro` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / criteria pass rate intellectual property](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__intellectual_property.md) | 88.34% | SE ±1.91 pp | 31 / 49 | $0.7 | 380.94source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 88.339 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.704723 \| USD \| per_test \| Lower is better \| \| `latency` \| 380.938 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.91 pp sourcePopulation 81 stderr 1.911 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label deepseek/deepseek-v4-pro Source version 1 Observed 2026-10-22T02:10:09.422Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__intellectual_property/deepseek~1deepseek-v4-pro` | | | | | | | | | [Legal Research Bench / family](https://benchize.com/benchmarks/vals-ai%3Alegal_research%3Afamily.md) | 9.09% | SE ±6.27 pp | 34 / 50 | $0.82 | 1,317.91source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 9.091 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.816394 \| USD \| per_test \| Lower is better \| \| `latency` \| 1317.912 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±6.27 pp sourcePopulation 82 stderr 6.273 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) Source Vals AI Original model label deepseek/deepseek-v4-pro Source version 1 Observed 2026-10-22T02:10:07.934Z Evidence SHA-256 `26c58dfd47d0d2dc86f33c6d056f30309e0ee63570e7cd752c593b385f9d862e` Source locator `/tasks/family/deepseek~1deepseek-v4-pro` | | | | | | | | | [EMB / misc](https://benchize.com/benchmarks/vals-ai%3Aemb%3Amisc.md) | 57.65% | SE ±6.55 pp | 24 / 47 | $0.71 | 811.88source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 57.647 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.714205 \| USD \| per_test \| Lower is better \| \| `latency` \| 811.882 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±6.55 pp sourcePopulation 76 stderr 6.554 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) Source Vals AI Original model label deepseek/deepseek-v4-pro Source version 1 Observed 2026-10-22T02:10:12.907Z Evidence SHA-256 `d807aee620ea47152a1fca0257a8dd12a1be4512c76cc1744c1ae9e285180918` Source locator `/tasks/misc/deepseek~1deepseek-v4-pro` | | | | | | | | | [Terminal-Bench Science 0.1 · life](https://benchize.com/benchmarks/terminal-science%3Av0-1-eval%3Alife.md) | 0% | SE ±0 pp | 9 / 9 | $107.43 | — | [terminal science](https://www.terminal-bench-science.ai/api/leaderboard?package=terminal-bench-science%2Fterminal-bench-science&name=v0-1-eval) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `total_cost` \| 107.43456482000002 \| USD \| all_trials \| Lower is better \| \| `total_tokens` \| 2083306722 \| tokens \| all_trials \| Lower is better \| ### Configuration agent Codex benchmarkVersion v0-1-eval domain life Reasoning effort max ### Reported statistics Uncertainty Standard error: ±0 pp accuracyStandardError 0 passes 0 rank 16 tasks 57 ### Source [Original source](https://www.terminal-bench-science.ai/api/leaderboard?package=terminal-bench-science%2Fterminal-bench-science&name=v0-1-eval) Source terminal science Original model label DeepSeek V4 Pro Evaluated 2026-09-16T13:45:27.369938+00:00 Observed 2026-10-21T16:32:46.684Z Evidence SHA-256 `86e6278ca64da88b69c1efd6059a33248bd300f27673098f60184286b470c4a7` Source locator `/rows/15/metrics/domain_metrics/life` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / criteria pass rate funds asset management](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__funds_asset_management.md) | 80.76% | SE ±2.13 pp | 26 / 49 | $0.96 | 523.88source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 80.758 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.955566 \| USD \| per_test \| Lower is better \| \| `latency` \| 523.878 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.13 pp sourcePopulation 81 stderr 2.132 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label deepseek/deepseek-v4-pro Source version 1 Observed 2026-10-22T02:10:09.422Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__funds_asset_management/deepseek~1deepseek-v4-pro` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / funds asset management](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Afunds_asset_management.md) | 0% | SE ±0 pp | 11 / 49 | $0.96 | 523.88source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.955566 \| USD \| per_test \| Lower is better \| \| `latency` \| 523.878 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 81 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label deepseek/deepseek-v4-pro Source version 1 Observed 2026-10-22T02:10:09.422Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/funds_asset_management/deepseek~1deepseek-v4-pro` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / healthcare life sciences](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Ahealthcare_life_sciences.md) | 0% | SE ±0 pp | 24 / 49 | $0.85 | 435.38source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.848233 \| USD \| per_test \| Lower is better \| \| `latency` \| 435.376 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 81 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label deepseek/deepseek-v4-pro Source version 1 Observed 2026-10-22T02:10:09.422Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/healthcare_life_sciences/deepseek~1deepseek-v4-pro` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / criteria pass rate banking finance](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__banking_finance.md) | 86.25% | SE ±2.1 pp | 29 / 49 | $0.61 | 311.9source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 86.245 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.609606 \| USD \| per_test \| Lower is better \| \| `latency` \| 311.896 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.1 pp sourcePopulation 81 stderr 2.104 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label deepseek/deepseek-v4-pro Source version 1 Observed 2026-10-22T02:10:09.422Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__banking_finance/deepseek~1deepseek-v4-pro` | | | | | | | | | [Tax Agent Bench / current temporal analysis](https://benchize.com/benchmarks/vals-ai%3Atax_agent_bench%3Acurrent_temporal_analysis.md) | 36.6% | SE ±8.93 pp | 19 / 46 | $1.14 | 1,727.13source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 36.595 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 1.136219 \| USD \| per_test \| Lower is better \| \| `latency` \| 1727.127 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±8.93 pp sourcePopulation 73 stderr 8.931 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) Source Vals AI Original model label deepseek/deepseek-v4-pro Source version 1 Observed 2026-10-22T02:10:11.407Z Evidence SHA-256 `446db182b67efb7923b3187a4040ec3a3ea492b8ad699a2b6d708d714cc063ac` Source locator `/tasks/current_temporal_analysis/deepseek~1deepseek-v4-pro` | | | | | | | | | [TapTap Maker public](https://benchize.com/benchmarks/taptap-maker%3Apublic.md) | 70.52% | Range 70.52–70.52% | 25 / 30 | $0.27 | — | [taptap maker](https://maker.taptap.cn/leaderboard/data/editions/2026-10-18-v6-voxel.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `l2_pass_rate` \| 0.7052 \| score_fraction \| reported \| Higher is better \| \| `cost_per_case` \| 0.2667 \| USD \| per_case \| Lower is better \| ### Configuration datasetVersion 0.2.3 edition 2026-10-18-v6-voxel harness claude Reasoning effort max reasoningMode model-managed ### Reported statistics Uncertainty Reported interval: 70.52–70.52% interval [0.7052,0.7052] intervalMethod bootstrap-case rank 29 rankInterval [27,29] ### Source [Original source](https://maker.taptap.cn/leaderboard/data/editions/2026-10-18-v6-voxel.json) Source taptap maker Original model label deepseek-v4-pro Observed 2026-10-22T06:30:04.044Z Evidence SHA-256 `ad2d7f1adeae0548a4edc0b5e75d11449b443449ff76533d04bc1fb1b6d7035e` Source locator `/public/28` | | | | | | | | | [Tax Agent Bench / forms filings](https://benchize.com/benchmarks/vals-ai%3Atax_agent_bench%3Aforms_filings.md) | 13.33% | SE ±6.31 pp | 23 / 46 | $0.87 | 1,588.79source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 13.333 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.867778 \| USD \| per_test \| Lower is better \| \| `latency` \| 1588.793 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±6.31 pp sourcePopulation 73 stderr 6.312 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) Source Vals AI Original model label deepseek/deepseek-v4-pro Source version 1 Observed 2026-10-22T02:10:11.407Z Evidence SHA-256 `446db182b67efb7923b3187a4040ec3a3ea492b8ad699a2b6d708d714cc063ac` Source locator `/tasks/forms_filings/deepseek~1deepseek-v4-pro` | | | | | | | | | [Terminal-Bench 4.0 / security](https://benchize.com/benchmarks/vals-ai%3Aterminal-bench-4%3Asecurity.md) | 26.67% | SE ±6.67 pp | 20 / 36 | — | — | [Vals AI · 4.0](https://www.vals.ai/_astro/benchmark_view_terminal_bench_4.CAnGyzC_.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 26.667 \| percent \| reported \| Higher is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±6.67 pp sourcePopulation 52 stderr 6.667 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_terminal_bench_4.CAnGyzC_.json) Source Vals AI Original model label deepseek/deepseek-v4-pro Source version 4.0 Observed 2026-10-22T02:10:25.308Z Evidence SHA-256 `98f6964580e901f1aa143db664bd92aa558415f643569492545d67c47d7f519b` Source locator `/tasks/security/deepseek~1deepseek-v4-pro` | | | | | | | | | [Terminal-Bench 4.0 / software](https://benchize.com/benchmarks/vals-ai%3Aterminal-bench-4%3Asoftware.md) | 22.22% | SE ±3.21 pp | 20 / 36 | — | — | [Vals AI · 4.0](https://www.vals.ai/_astro/benchmark_view_terminal_bench_4.CAnGyzC_.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 22.222 \| percent \| reported \| Higher is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±3.21 pp sourcePopulation 52 stderr 3.208 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_terminal_bench_4.CAnGyzC_.json) Source Vals AI Original model label deepseek/deepseek-v4-pro Source version 4.0 Observed 2026-10-22T02:10:25.308Z Evidence SHA-256 `98f6964580e901f1aa143db664bd92aa558415f643569492545d67c47d7f519b` Source locator `/tasks/software/deepseek~1deepseek-v4-pro` | | | | | | | | | [Tax Agent Bench / overall](https://benchize.com/benchmarks/vals-ai%3Atax_agent_bench%3Aoverall.md) | 25.17% | SE ±3.06 pp | 26 / 46 | $0.72 | 1,244.64source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 25.17 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.719152 \| USD \| per_test \| Lower is better \| \| `latency` \| 1244.635 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±3.06 pp sourcePopulation 73 stderr 3.058 tokenTotals {"cache_read_tokens":134364160,"cache_write_tokens":null,"input_tokens":171099768,"output_tokens":21311454,"reasoning_tokens":17772506} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) Source Vals AI Original model label deepseek/deepseek-v4-pro Source version 1 Observed 2026-10-22T02:10:11.407Z Evidence SHA-256 `446db182b67efb7923b3187a4040ec3a3ea492b8ad699a2b6d708d714cc063ac` Source locator `/tasks/overall/deepseek~1deepseek-v4-pro` | | | | | | | | | [USAMO 2026](https://benchize.com/benchmarks/matharena%3Ausamo--usamo_2026.md) | 60.71% | 95% CI 41.17–80.25% | 4 / 8 | 0.5usd per problem | — | [matharena · usamo--usamo_2026](https://matharena.ai/competition_tables/usamo--usamo_2026) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 60.71 \| percent \| competition \| Higher is better \| \| `cost_per_problem` \| 0.50 \| usd_per_problem \| competition \| Lower is better \| ### Configuration competition usamo--usamo_2026 predicted false Reasoning effort max sourceModelName DeepSeek-v4-Pro (Max) sourceModelVersion deepseek/deepseek_v4_pro thinkingEnabled true ### Reported statistics Uncertainty 95% confidence interval: 41.17–80.25% confidenceInterval95 19.54 problems 6 ### Source [Original source](https://matharena.ai/competition_tables/usamo--usamo_2026) Source matharena Original model label DeepSeek-v4-Pro (Max) Source version usamo--usamo_2026 Observed 2026-10-22T05:30:15.570Z Evidence SHA-256 `2c6f85fa189ac482d54d597e1cbbe64803cda65cd85e78add44087d94799756b` Source locator `/table/rows/3` | | | | | | | | | [EMB / template](https://benchize.com/benchmarks/vals-ai%3Aemb%3Atemplate.md) | 46.07% | SE ±4.38 pp | 37 / 47 | $0.8 | 777.8source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 46.07 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.799253 \| USD \| per_test \| Lower is better \| \| `latency` \| 777.796 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±4.38 pp sourcePopulation 76 stderr 4.377 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) Source Vals AI Original model label deepseek/deepseek-v4-pro Source version 1 Observed 2026-10-22T02:10:12.907Z Evidence SHA-256 `d807aee620ea47152a1fca0257a8dd12a1be4512c76cc1744c1ae9e285180918` Source locator `/tasks/template/deepseek~1deepseek-v4-pro` | | | | | | | | | [ArXivMath Overall](https://benchize.com/benchmarks/matharena%3Aoverall--final_answer_comps.md) | 76.62% | 95% CI 73.77–79.47% | 6 / 20 | 0.23usd per problem | — | [matharena · overall--final_answer_comps](https://matharena.ai/competition_tables/overall--final_answer_comps) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 76.62 \| percent \| competition \| Higher is better \| \| `cost_per_problem` \| 0.23 \| usd_per_problem \| competition \| Lower is better \| ### Configuration competition overall--final_answer_comps predicted false Reasoning effort max sourceModelName DeepSeek-v4-Pro (Max) sourceModelVersion deepseek/deepseek_v4_pro thinkingEnabled true ### Reported statistics Uncertainty 95% confidence interval: 73.77–79.47% confidenceInterval95 2.85 problems Not reported ### Source [Original source](https://matharena.ai/competition_tables/overall--final_answer_comps) Source matharena Original model label DeepSeek-v4-Pro (Max) Source version overall--final_answer_comps Observed 2026-10-22T05:30:11.571Z Evidence SHA-256 `8b08b20c1ac5aec9a829d234469bc39d40b0eb189790d84b7264c2756ffa161e` Source locator `/table/rows/6` | | | | | | | | | [Legal Research Bench / health](https://benchize.com/benchmarks/vals-ai%3Alegal_research%3Ahealth.md) | 46.67% | SE ±9.26 pp | 32 / 50 | $0.81 | 999.26source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 46.667 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.81017 \| USD \| per_test \| Lower is better \| \| `latency` \| 999.255 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±9.26 pp sourcePopulation 82 stderr 9.264 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) Source Vals AI Original model label deepseek/deepseek-v4-pro Source version 1 Observed 2026-10-22T02:10:07.934Z Evidence SHA-256 `26c58dfd47d0d2dc86f33c6d056f30309e0ee63570e7cd752c593b385f9d862e` Source locator `/tasks/health/deepseek~1deepseek-v4-pro` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / immigration](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Aimmigration.md) | 0% | SE ±0 pp | 6 / 49 | $0.52 | 333.84source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.519471 \| USD \| per_test \| Lower is better \| \| `latency` \| 333.842 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 81 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label deepseek/deepseek-v4-pro Source version 1 Observed 2026-10-22T02:10:09.422Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/immigration/deepseek~1deepseek-v4-pro` | | | | | | | | | [Terminal-Bench 4.0 / overall](https://benchize.com/benchmarks/vals-ai%3Aterminal-bench-4%3Aoverall.md) | 11.11% | SE ±1.34 pp | 27 / 36 | $3.45 | 5,146.97source duration | [Vals AI · 4.0](https://www.vals.ai/_astro/benchmark_view_terminal_bench_4.CAnGyzC_.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 11.111 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 3.452522 \| USD \| per_test \| Lower is better \| \| `latency` \| 5146.973 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.34 pp sourcePopulation 52 stderr 1.336 tokenTotals {"cache_read_tokens":3737754965,"cache_write_tokens":0,"input_tokens":3748960409,"output_tokens":12276277,"reasoning_tokens":7244257} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_terminal_bench_4.CAnGyzC_.json) Source Vals AI Original model label deepseek/deepseek-v4-pro Source version 4.0 Observed 2026-10-22T02:10:25.308Z Evidence SHA-256 `98f6964580e901f1aa143db664bd92aa558415f643569492545d67c47d7f519b` Source locator `/tasks/overall/deepseek~1deepseek-v4-pro` | | | | | | | | | [Finance Agent (v2) / overall](https://benchize.com/benchmarks/vals-ai%3Afabv2%3Aoverall.md) | 44.08% | SE ±0.65 pp | 38 / 50 | $0.88 | 1,526.18source duration | [Vals AI · 2](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 44.083 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.879086 \| USD \| per_test \| Lower is better \| \| `latency` \| 1526.183 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.65 pp sourcePopulation 80 stderr 0.653 tokenTotals {"cache_read_tokens":284032811,"cache_write_tokens":null,"input_tokens":508078472,"output_tokens":22058372,"reasoning_tokens":17233163} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) Source Vals AI Original model label deepseek/deepseek-v4-pro Source version 2 Observed 2026-10-22T02:10:12.404Z Evidence SHA-256 `f731b20a26c126725d125519f2b5841aa45254656dba859a9570d7ce7e5d8191` Source locator `/tasks/overall/deepseek~1deepseek-v4-pro` | | | | | | | | | [Legal Research Bench / overall](https://benchize.com/benchmarks/vals-ai%3Alegal_research%3Aoverall.md) | 23.08% | SE ±2.93 pp | 38 / 50 | $0.68 | 1,024.62source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 23.077 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.682218 \| USD \| per_test \| Lower is better \| \| `latency` \| 1024.619 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.93 pp sourcePopulation 82 stderr 2.928 tokenTotals {"cache_read_tokens":300570368,"cache_write_tokens":0,"input_tokens":341401669,"output_tokens":11003300,"reasoning_tokens":8033786} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) Source Vals AI Original model label deepseek/deepseek-v4-pro Source version 1 Observed 2026-10-22T02:10:07.934Z Evidence SHA-256 `26c58dfd47d0d2dc86f33c6d056f30309e0ee63570e7cd752c593b385f9d862e` Source locator `/tasks/overall/deepseek~1deepseek-v4-pro` | | | | | | | | | [Public Benefits Bench v1 / multi turn](https://benchize.com/benchmarks/vals-ai%3Apublic-benefits-bench-v1%3Amulti_turn.md) | 48.71% | SE ±1.3 pp | 7 / 8 | $0.09 | 350.63source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_snap_v1.BPa0Hrre.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 48.714 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.086444 \| USD \| per_test \| Lower is better \| \| `latency` \| 350.63 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.3 pp sourcePopulation 13 stderr 1.3 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_snap_v1.BPa0Hrre.json) Source Vals AI Original model label deepseek/deepseek-v4-pro Source version 1 Observed 2026-10-22T02:10:32.821Z Evidence SHA-256 `b85e2b3cbdf03e13e2ffdb27ce4f108003a548d705a24ae26a0a115e2027475d` Source locator `/tasks/multi_turn/deepseek~1deepseek-v4-pro` | | | | | | | | | [MMLU-Pro](https://benchize.com/benchmarks/mmlu-pro.md) | 87.25% | SE ±0.34 pp | 18 / 40 | $0.03 | 577.62source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_mmlu_pro.y416pXDi.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 87.249 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.027032 \| USD \| per_test \| Lower is better \| \| `latency` \| 577.617 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.34 pp sourcePopulation 138 stderr 0.335 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_mmlu_pro.y416pXDi.json) Source Vals AI Original model label deepseek/deepseek-v4-pro Source version 1 Observed 2026-10-22T02:10:21.194Z Evidence SHA-256 `dab025507f46c716dd8d145e90ec0a737d9c56219853f41e4b1cc3c5e8c222b4` Source locator `/tasks/overall/deepseek~1deepseek-v4-pro` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / criteria pass rate energy natural resources](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__energy_natural_resources.md) | 93.4% | SE ±1.43 pp | 22 / 49 | $0.98 | 388.47source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 93.399 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.983444 \| USD \| per_test \| Lower is better \| \| `latency` \| 388.468 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.43 pp sourcePopulation 81 stderr 1.429 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label deepseek/deepseek-v4-pro Source version 1 Observed 2026-10-22T02:10:09.422Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__energy_natural_resources/deepseek~1deepseek-v4-pro` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / tax](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Atax.md) | 0% | SE ±0 pp | 3 / 49 | $0.77 | 381.64source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.767561 \| USD \| per_test \| Lower is better \| \| `latency` \| 381.644 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 81 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label deepseek/deepseek-v4-pro Source version 1 Observed 2026-10-22T02:10:09.422Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/tax/deepseek~1deepseek-v4-pro` | | | | | | | | | [TaxEval v2 / overall](https://benchize.com/benchmarks/vals-ai%3Atax_eval_v2%3Aoverall.md) | 72.08% | SE ±0.88 pp | 27 / 40 | $0.03 | 189.7source duration | [Vals AI · 2](https://www.vals.ai/_astro/benchmark_view_tax_eval_v2.B3yoffr8.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 72.077 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.033187 \| USD \| per_test \| Lower is better \| \| `latency` \| 189.695 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.88 pp sourcePopulation 145 stderr 0.877 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_tax_eval_v2.B3yoffr8.json) Source Vals AI Original model label deepseek/deepseek-v4-pro Source version 2 Observed 2026-10-22T02:10:14.414Z Evidence SHA-256 `8b354a6ae848f5bb25a2e8b061b5f576d5b9e0c11b941dc1e6bed078e6529cd6` Source locator `/tasks/overall/deepseek~1deepseek-v4-pro` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / criteria pass rate healthcare life sciences](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__healthcare_life_sciences.md) | 86.14% | SE ±1.99 pp | 27 / 49 | $0.85 | 435.38source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 86.139 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.848233 \| USD \| per_test \| Lower is better \| \| `latency` \| 435.376 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.99 pp sourcePopulation 81 stderr 1.988 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label deepseek/deepseek-v4-pro Source version 1 Observed 2026-10-22T02:10:09.422Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__healthcare_life_sciences/deepseek~1deepseek-v4-pro` | | | | | | | | | [Finance Agent (v2) / adjustments](https://benchize.com/benchmarks/vals-ai%3Afabv2%3Aadjustments.md) | 41.59% | SE ±1.19 pp | 38 / 50 | $0.92 | 1,486.77source duration | [Vals AI · 2](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 41.586 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.923173 \| USD \| per_test \| Lower is better \| \| `latency` \| 1486.77 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.19 pp sourcePopulation 80 stderr 1.185 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) Source Vals AI Original model label deepseek/deepseek-v4-pro Source version 2 Observed 2026-10-22T02:10:12.404Z Evidence SHA-256 `f731b20a26c126725d125519f2b5841aa45254656dba859a9570d7ce7e5d8191` Source locator `/tasks/adjustments/deepseek~1deepseek-v4-pro` | | | | | | | | | [SWE-bench / 15 min - 1 hour](https://benchize.com/benchmarks/vals-ai%3Aswebench%3A15%20min%20-%201%20hour.md) | 75.86% | SE ±2.65 pp | 19 / 39 | — | 716.77source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_swebench.Ae8ni7h0.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 75.862 \| percent \| reported \| Higher is better \| \| `latency` \| 716.772 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.65 pp sourcePopulation 88 stderr 2.654 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_swebench.Ae8ni7h0.json) Source Vals AI Original model label deepseek/deepseek-v4-pro Source version 1 Observed 2026-10-22T02:10:28.317Z Evidence SHA-256 `7a4e2e8485283caac500127419f68ec4bd5414c837ade114fc6afa5b2d34ccaa` Source locator `/tasks/15 min - 1 hour/deepseek~1deepseek-v4-pro` | | | | | | | | | [Terminal-Bench Science 0.1 · physical](https://benchize.com/benchmarks/terminal-science%3Av0-1-eval%3Aphysical.md) | 3.92% | SE ±2.72 pp | 8 / 9 | $140.45 | — | [terminal science](https://www.terminal-bench-science.ai/api/leaderboard?package=terminal-bench-science%2Fterminal-bench-science&name=v0-1-eval) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 3.9215686274509802 \| percent \| reported \| Higher is better \| \| `total_cost` \| 140.452167108 \| USD \| all_trials \| Lower is better \| \| `total_tokens` \| 2763809854 \| tokens \| all_trials \| Lower is better \| ### Configuration agent Codex benchmarkVersion v0-1-eval domain physical Reasoning effort max ### Reported statistics Uncertainty Standard error: ±2.72 pp accuracyStandardError 2.718052078759498 passes 2 rank 16 tasks 51 ### Source [Original source](https://www.terminal-bench-science.ai/api/leaderboard?package=terminal-bench-science%2Fterminal-bench-science&name=v0-1-eval) Source terminal science Original model label DeepSeek V4 Pro Evaluated 2026-09-16T13:45:27.369938+00:00 Observed 2026-10-21T16:32:46.684Z Evidence SHA-256 `86e6278ca64da88b69c1efd6059a33248bd300f27673098f60184286b470c4a7` Source locator `/rows/15/metrics/domain_metrics/physical` | | | | | | | | | [Public Benefits Bench v1 / web search](https://benchize.com/benchmarks/vals-ai%3Apublic-benefits-bench-v1%3Aweb_search.md) | 53.99% | SE ±1.3 pp | 4 / 8 | $0.21 | 658.98source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_snap_v1.BPa0Hrre.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 53.992 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.212826 \| USD \| per_test \| Lower is better \| \| `latency` \| 658.98 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.3 pp sourcePopulation 13 stderr 1.296 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_snap_v1.BPa0Hrre.json) Source Vals AI Original model label deepseek/deepseek-v4-pro Source version 1 Observed 2026-10-22T02:10:32.821Z Evidence SHA-256 `b85e2b3cbdf03e13e2ffdb27ce4f108003a548d705a24ae26a0a115e2027475d` Source locator `/tasks/web_search/deepseek~1deepseek-v4-pro` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / criteria pass rate corporate ma](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__corporate_ma.md) | 88.46% | SE ±1.81 pp | 29 / 49 | $0.86 | 451.45source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 88.462 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.855036 \| USD \| per_test \| Lower is better \| \| `latency` \| 451.45 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.81 pp sourcePopulation 81 stderr 1.812 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label deepseek/deepseek-v4-pro Source version 1 Observed 2026-10-22T02:10:09.422Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__corporate_ma/deepseek~1deepseek-v4-pro` | | | | | | | | | [Legal Research Bench / weighted pass civil litigation](https://benchize.com/benchmarks/vals-ai%3Alegal_research%3Aweighted_pass__civil_litigation.md) | 85.96% | SE ±3.53 pp | 27 / 50 | $0.74 | 1,095.54source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 85.96 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.736017 \| USD \| per_test \| Lower is better \| \| `latency` \| 1095.538 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±3.53 pp sourcePopulation 82 stderr 3.534 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) Source Vals AI Original model label deepseek/deepseek-v4-pro Source version 1 Observed 2026-10-22T02:10:07.934Z Evidence SHA-256 `26c58dfd47d0d2dc86f33c6d056f30309e0ee63570e7cd752c593b385f9d862e` Source locator `/tasks/weighted_pass__civil_litigation/deepseek~1deepseek-v4-pro` | | | | | | | | | [TaxEval v2 / stepwise reasoning](https://benchize.com/benchmarks/vals-ai%3Atax_eval_v2%3Astepwise_reasoning.md) | 83.57% | SE ±1.06 pp | 25 / 40 | $0.03 | 189.7source duration | [Vals AI · 2](https://www.vals.ai/_astro/benchmark_view_tax_eval_v2.B3yoffr8.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 83.565 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.033187 \| USD \| per_test \| Lower is better \| \| `latency` \| 189.695 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.06 pp sourcePopulation 145 stderr 1.06 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_tax_eval_v2.B3yoffr8.json) Source Vals AI Original model label deepseek/deepseek-v4-pro Source version 2 Observed 2026-10-22T02:10:14.414Z Evidence SHA-256 `8b354a6ae848f5bb25a2e8b061b5f576d5b9e0c11b941dc1e6bed078e6529cd6` Source locator `/tasks/stepwise_reasoning/deepseek~1deepseek-v4-pro` | | | | | | | | | [Tax Agent Bench / weighted pass](https://benchize.com/benchmarks/vals-ai%3Atax_agent_bench%3Aweighted_pass.md) | 58.5% | SE ±3.13 pp | 30 / 46 | $0.72 | 1,244.64source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 58.499 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.719152 \| USD \| per_test \| Lower is better \| \| `latency` \| 1244.635 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±3.13 pp sourcePopulation 73 stderr 3.132 tokenTotals {"cache_read_tokens":134364160,"cache_write_tokens":null,"input_tokens":171099768,"output_tokens":21311454,"reasoning_tokens":17772506} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) Source Vals AI Original model label deepseek/deepseek-v4-pro Source version 1 Observed 2026-10-22T02:10:11.407Z Evidence SHA-256 `446db182b67efb7923b3187a4040ec3a3ea492b8ad699a2b6d708d714cc063ac` Source locator `/tasks/weighted_pass/deepseek~1deepseek-v4-pro` | | | | | | | | | [MMLU Pro / other](https://benchize.com/benchmarks/vals-ai%3Ammlu_pro%3Aother.md) | 84.2% | SE ±1.2 pp | 19 / 40 | $0.01 | 420.75source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_mmlu_pro.y416pXDi.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 84.199 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.013978 \| USD \| per_test \| Lower is better \| \| `latency` \| 420.745 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.2 pp sourcePopulation 138 stderr 1.2 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_mmlu_pro.y416pXDi.json) Source Vals AI Original model label deepseek/deepseek-v4-pro Source version 1 Observed 2026-10-22T02:10:21.194Z Evidence SHA-256 `dab025507f46c716dd8d145e90ec0a737d9c56219853f41e4b1cc3c5e8c222b4` Source locator `/tasks/other/deepseek~1deepseek-v4-pro` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / criteria pass rate real estate](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__real_estate.md) | 89.21% | SE ±1.68 pp | 27 / 49 | $0.8 | 415.1source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 89.213 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.804107 \| USD \| per_test \| Lower is better \| \| `latency` \| 415.102 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.68 pp sourcePopulation 81 stderr 1.677 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label deepseek/deepseek-v4-pro Source version 1 Observed 2026-10-22T02:10:09.422Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__real_estate/deepseek~1deepseek-v4-pro` | | | | | | | | | [Legal Research Bench / civil litigation](https://benchize.com/benchmarks/vals-ai%3Alegal_research%3Acivil_litigation.md) | 33.33% | SE ±9.25 pp | 25 / 50 | $0.74 | 1,095.54source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 33.333 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.736017 \| USD \| per_test \| Lower is better \| \| `latency` \| 1095.538 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±9.25 pp sourcePopulation 82 stderr 9.245 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) Source Vals AI Original model label deepseek/deepseek-v4-pro Source version 1 Observed 2026-10-22T02:10:07.934Z Evidence SHA-256 `26c58dfd47d0d2dc86f33c6d056f30309e0ee63570e7cd752c593b385f9d862e` Source locator `/tasks/civil_litigation/deepseek~1deepseek-v4-pro` | | | | | | | | | [TapTap Maker heldout](https://benchize.com/benchmarks/taptap-maker%3Aheldout.md) | 56.51% | Range 56.51–56.51% | 18 / 30 | — | — | [taptap maker](https://maker.taptap.cn/leaderboard/data/editions/2026-10-18-v6-voxel.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `l2_pass_rate` \| 0.5651 \| score_fraction \| reported \| Higher is better \| ### Configuration datasetVersion 0.2.3 edition 2026-10-18-v6-voxel harness claude Reasoning effort max reasoningMode Not reported ### Reported statistics Uncertainty Reported interval: 56.51–56.51% interval [0.5651,0.5651] intervalMethod bootstrap-case rank 20 rankInterval [19,23] ### Source [Original source](https://maker.taptap.cn/leaderboard/data/editions/2026-10-18-v6-voxel.json) Source taptap maker Original model label deepseek-v4-pro Observed 2026-10-22T06:30:04.044Z Evidence SHA-256 `ad2d7f1adeae0548a4edc0b5e75d11449b443449ff76533d04bc1fb1b6d7035e` Source locator `/heldout/19` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / criteria pass rate emerging companies venture capital](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__emerging_companies_venture_capital.md) | 87.37% | SE ±1.94 pp | 24 / 49 | $0.76 | 415.08source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 87.372 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.76098 \| USD \| per_test \| Lower is better \| \| `latency` \| 415.078 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.94 pp sourcePopulation 81 stderr 1.944 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label deepseek/deepseek-v4-pro Source version 1 Observed 2026-10-22T02:10:09.422Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__emerging_companies_venture_capital/deepseek~1deepseek-v4-pro` | | | | | | | | | [Tax Agent Bench / fact pattern analysis](https://benchize.com/benchmarks/vals-ai%3Atax_agent_bench%3Afact_pattern_analysis.md) | 21.86% | SE ±6.19 pp | 31 / 46 | $0.65 | 1,278.56source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 21.86 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.651612 \| USD \| per_test \| Lower is better \| \| `latency` \| 1278.557 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±6.19 pp sourcePopulation 73 stderr 6.192 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) Source Vals AI Original model label deepseek/deepseek-v4-pro Source version 1 Observed 2026-10-22T02:10:11.407Z Evidence SHA-256 `446db182b67efb7923b3187a4040ec3a3ea492b8ad699a2b6d708d714cc063ac` Source locator `/tasks/fact_pattern_analysis/deepseek~1deepseek-v4-pro` | | | | | | | | | [Tax Agent Bench / weighted pass rule source lookup](https://benchize.com/benchmarks/vals-ai%3Atax_agent_bench%3Aweighted_pass__rule_source_lookup.md) | 55.08% | SE ±8.51 pp | 25 / 46 | $0.72 | 1,210.27source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 55.079 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.724403 \| USD \| per_test \| Lower is better \| \| `latency` \| 1210.271 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±8.51 pp sourcePopulation 73 stderr 8.513 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) Source Vals AI Original model label deepseek/deepseek-v4-pro Source version 1 Observed 2026-10-22T02:10:11.407Z Evidence SHA-256 `446db182b67efb7923b3187a4040ec3a3ea492b8ad699a2b6d708d714cc063ac` Source locator `/tasks/weighted_pass__rule_source_lookup/deepseek~1deepseek-v4-pro` | | | | | | | | | [HMMT Feb 2026](https://benchize.com/benchmarks/matharena%3Ahmmt--hmmt_feb_2026.md) | 93.94% | 95% CI 89.87–98.01% | 8 / 21 | 0.14usd per problem | — | [matharena · hmmt--hmmt_feb_2026](https://matharena.ai/competition_tables/hmmt--hmmt_feb_2026) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 93.94 \| percent \| competition \| Higher is better \| \| `cost_per_problem` \| 0.14 \| usd_per_problem \| competition \| Lower is better \| ### Configuration competition hmmt--hmmt_feb_2026 predicted false Reasoning effort max sourceModelName DeepSeek-v4-Pro (Max) sourceModelVersion deepseek/deepseek_v4_pro thinkingEnabled true ### Reported statistics Uncertainty 95% confidence interval: 89.87–98.01% confidenceInterval95 4.07 problems 33 ### Source [Original source](https://matharena.ai/competition_tables/hmmt--hmmt_feb_2026) Source matharena Original model label DeepSeek-v4-Pro (Max) Source version hmmt--hmmt_feb_2026 Observed 2026-10-22T05:30:13.595Z Evidence SHA-256 `0dce922c33a54d29d4a1872a42d74077b491915c80d8433f4b379ea0b78bb979` Source locator `/table/rows/10` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / corporate ma](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acorporate_ma.md) | 20% | SE ±20 pp | 1 / 49 | $0.86 | 451.45source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 20 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.855036 \| USD \| per_test \| Lower is better \| \| `latency` \| 451.45 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±20 pp sourcePopulation 81 stderr 20 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label deepseek/deepseek-v4-pro Source version 1 Observed 2026-10-22T02:10:09.422Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/corporate_ma/deepseek~1deepseek-v4-pro` | | | | | | | | | [Legal Research Bench / administrative regulatory](https://benchize.com/benchmarks/vals-ai%3Alegal_research%3Aadministrative_regulatory.md) | 28.79% | SE ±5.62 pp | 38 / 50 | $0.67 | 903.62source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 28.788 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.671217 \| USD \| per_test \| Lower is better \| \| `latency` \| 903.624 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±5.62 pp sourcePopulation 82 stderr 5.616 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) Source Vals AI Original model label deepseek/deepseek-v4-pro Source version 1 Observed 2026-10-22T02:10:07.934Z Evidence SHA-256 `26c58dfd47d0d2dc86f33c6d056f30309e0ee63570e7cd752c593b385f9d862e` Source locator `/tasks/administrative_regulatory/deepseek~1deepseek-v4-pro` | | | | | | | | | [Tax Agent Bench / weighted pass controversy precedence analysis](https://benchize.com/benchmarks/vals-ai%3Atax_agent_bench%3Aweighted_pass__controversy_precedence_analysis.md) | 54.84% | SE ±7.99 pp | 23 / 46 | $0.7 | 1,303.85source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 54.843 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.703688 \| USD \| per_test \| Lower is better \| \| `latency` \| 1303.848 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±7.99 pp sourcePopulation 73 stderr 7.987 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) Source Vals AI Original model label deepseek/deepseek-v4-pro Source version 1 Observed 2026-10-22T02:10:11.407Z Evidence SHA-256 `446db182b67efb7923b3187a4040ec3a3ea492b8ad699a2b6d708d714cc063ac` Source locator `/tasks/weighted_pass__controversy_precedence_analysis/deepseek~1deepseek-v4-pro` | | | | | | | | | [Terminal-Bench Science 0.1 · mathematical](https://benchize.com/benchmarks/terminal-science%3Av0-1-eval%3Amathematical.md) | 5.88% | SE ±3.29 pp | 6 / 9 | $146.13 | — | [terminal science](https://www.terminal-bench-science.ai/api/leaderboard?package=terminal-bench-science%2Fterminal-bench-science&name=v0-1-eval) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 5.88235294117647 \| percent \| reported \| Higher is better \| \| `total_cost` \| 146.126849924 \| USD \| all_trials \| Lower is better \| \| `total_tokens` \| 2672070982 \| tokens \| all_trials \| Lower is better \| ### Configuration agent Codex benchmarkVersion v0-1-eval domain mathematical Reasoning effort max ### Reported statistics Uncertainty Standard error: ±3.29 pp accuracyStandardError 3.294776668301199 passes 3 rank 16 tasks 51 ### Source [Original source](https://www.terminal-bench-science.ai/api/leaderboard?package=terminal-bench-science%2Fterminal-bench-science&name=v0-1-eval) Source terminal science Original model label DeepSeek V4 Pro Evaluated 2026-09-16T13:45:27.369938+00:00 Observed 2026-10-21T16:32:46.684Z Evidence SHA-256 `86e6278ca64da88b69c1efd6059a33248bd300f27673098f60184286b470c4a7` Source locator `/rows/15/metrics/domain_metrics/mathematical` | | | | | | | | | [MMLU Pro / computer science](https://benchize.com/benchmarks/vals-ai%3Ammlu_pro%3Acomputer_science.md) | 89.02% | SE ±1.56 pp | 22 / 40 | $0.03 | 753.86source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_mmlu_pro.y416pXDi.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 89.024 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.029423 \| USD \| per_test \| Lower is better \| \| `latency` \| 753.864 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.56 pp sourcePopulation 138 stderr 1.559 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_mmlu_pro.y416pXDi.json) Source Vals AI Original model label deepseek/deepseek-v4-pro Source version 1 Observed 2026-10-22T02:10:21.194Z Evidence SHA-256 `dab025507f46c716dd8d145e90ec0a737d9c56219853f41e4b1cc3c5e8c222b4` Source locator `/tasks/computer_science/deepseek~1deepseek-v4-pro` | | | | | | | | | [Terminal-Bench Science 0.1 · engineering](https://benchize.com/benchmarks/terminal-science%3Av0-1-eval%3Aengineering.md) | 11.11% | SE ±6.05 pp | 4 / 9 | $108.4 | — | [terminal science](https://www.terminal-bench-science.ai/api/leaderboard?package=terminal-bench-science%2Fterminal-bench-science&name=v0-1-eval) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 11.11111111111111 \| percent \| reported \| Higher is better \| \| `total_cost` \| 108.39618107999999 \| USD \| all_trials \| Lower is better \| \| `total_tokens` \| 2200281229 \| tokens \| all_trials \| Lower is better \| ### Configuration agent Codex benchmarkVersion v0-1-eval domain engineering Reasoning effort max ### Reported statistics Uncertainty Standard error: ±6.05 pp accuracyStandardError 6.04812282168686 passes 3 rank 16 tasks 27 ### Source [Original source](https://www.terminal-bench-science.ai/api/leaderboard?package=terminal-bench-science%2Fterminal-bench-science&name=v0-1-eval) Source terminal science Original model label DeepSeek V4 Pro Evaluated 2026-09-16T13:45:27.369938+00:00 Observed 2026-10-21T16:32:46.684Z Evidence SHA-256 `86e6278ca64da88b69c1efd6059a33248bd300f27673098f60184286b470c4a7` Source locator `/rows/15/metrics/domain_metrics/engineering` | | | | | | | | | [CorpFin v2 / exact pages](https://benchize.com/benchmarks/vals-ai%3Acorp_fin_v2%3Aexact_pages.md) | 59.21% | SE ±1.68 pp | 31 / 39 | $0.00778 | 84.03source duration | [Vals AI · 2](https://www.vals.ai/_astro/benchmark_view_corp_fin_v2.CectKwl1.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 59.207 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.007781 \| USD \| per_test \| Lower is better \| \| `latency` \| 84.032 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.68 pp sourcePopulation 134 stderr 1.678 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_corp_fin_v2.CectKwl1.json) Source Vals AI Original model label deepseek/deepseek-v4-pro Source version 2 Observed 2026-10-22T02:10:13.429Z Evidence SHA-256 `7cf2217ab3728d5a84bffce82aab9fc52d7f810b0cbe8c65b387df47f3d31c9b` Source locator `/tasks/exact_pages/deepseek~1deepseek-v4-pro` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / international trade sanctions](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Ainternational_trade_sanctions.md) | 0% | SE ±0 pp | 4 / 49 | $0.64 | 461.64source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.642446 \| USD \| per_test \| Lower is better \| \| `latency` \| 461.638 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 81 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label deepseek/deepseek-v4-pro Source version 1 Observed 2026-10-22T02:10:09.422Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/international_trade_sanctions/deepseek~1deepseek-v4-pro` | | | | | | | | | [EMB / overall](https://benchize.com/benchmarks/vals-ai%3Aemb%3Aoverall.md) | 51.62% | SE ±3.05 pp | 39 / 47 | $0.8 | 822.14source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 51.623 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.796757 \| USD \| per_test \| Lower is better \| \| `latency` \| 822.144 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±3.05 pp sourcePopulation 76 stderr 3.053 tokenTotals {"cache_read_tokens":527403648,"cache_write_tokens":null,"input_tokens":541981728,"output_tokens":9803133,"reasoning_tokens":3184172} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) Source Vals AI Original model label deepseek/deepseek-v4-pro Source version 1 Observed 2026-10-22T02:10:12.907Z Evidence SHA-256 `d807aee620ea47152a1fca0257a8dd12a1be4512c76cc1744c1ae9e285180918` Source locator `/tasks/overall/deepseek~1deepseek-v4-pro` | | | | | | | | | [MMLU Pro / business](https://benchize.com/benchmarks/vals-ai%3Ammlu_pro%3Abusiness.md) | 91.89% | SE ±0.97 pp | 16 / 40 | $0.03 | 399.14source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_mmlu_pro.y416pXDi.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 91.888 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.0266 \| USD \| per_test \| Lower is better \| \| `latency` \| 399.135 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.97 pp sourcePopulation 138 stderr 0.972 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_mmlu_pro.y416pXDi.json) Source Vals AI Original model label deepseek/deepseek-v4-pro Source version 1 Observed 2026-10-22T02:10:21.194Z Evidence SHA-256 `dab025507f46c716dd8d145e90ec0a737d9c56219853f41e4b1cc3c5e8c222b4` Source locator `/tasks/business/deepseek~1deepseek-v4-pro` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / white collar defense investigations](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Awhite_collar_defense_investigations.md) | 0% | SE ±0 pp | 1 / 49 | $0.6 | 389.94source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.597321 \| USD \| per_test \| Lower is better \| \| `latency` \| 389.944 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 81 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label deepseek/deepseek-v4-pro Source version 1 Observed 2026-10-22T02:10:09.422Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/white_collar_defense_investigations/deepseek~1deepseek-v4-pro` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / criteria pass rate bankruptcy restructuring](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__bankruptcy_restructuring.md) | 76.13% | SE ±2.51 pp | 31 / 49 | $0.71 | 377.69source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 76.125 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.705278 \| USD \| per_test \| Lower is better \| \| `latency` \| 377.686 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.51 pp sourcePopulation 81 stderr 2.512 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label deepseek/deepseek-v4-pro Source version 1 Observed 2026-10-22T02:10:09.422Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__bankruptcy_restructuring/deepseek~1deepseek-v4-pro` | | | | | | | | | [Finance Agent (v2) / general quantitative analysis](https://benchize.com/benchmarks/vals-ai%3Afabv2%3Ageneral_quantitative_analysis.md) | 65.75% | SE ±1.4 pp | 38 / 50 | $0.96 | 1,415.1source duration | [Vals AI · 2](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 65.75 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.961249 \| USD \| per_test \| Lower is better \| \| `latency` \| 1415.095 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.4 pp sourcePopulation 80 stderr 1.396 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) Source Vals AI Original model label deepseek/deepseek-v4-pro Source version 2 Observed 2026-10-22T02:10:12.404Z Evidence SHA-256 `f731b20a26c126725d125519f2b5841aa45254656dba859a9570d7ce7e5d8191` Source locator `/tasks/general_quantitative_analysis/deepseek~1deepseek-v4-pro` | | | | | | | | | [ArXivMath 01/2026](https://benchize.com/benchmarks/matharena%3Aarxiv--january.md) | 73.91% | 95% CI 61.22–86.6% | 2 / 18 | 0.45usd per problem | — | [matharena · arxiv--january](https://matharena.ai/competition_tables/arxiv--january) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 73.91 \| percent \| competition \| Higher is better \| \| `cost_per_problem` \| 0.45 \| usd_per_problem \| competition \| Lower is better \| ### Configuration competition arxiv--january predicted false Reasoning effort max sourceModelName DeepSeek-v4-Pro (Max) sourceModelVersion deepseek/deepseek_v4_pro thinkingEnabled true ### Reported statistics Uncertainty 95% confidence interval: 61.22–86.6% confidenceInterval95 12.69 problems 23 ### Source [Original source](https://matharena.ai/competition_tables/arxiv--january) Source matharena Original model label DeepSeek-v4-Pro (Max) Source version arxiv--january Observed 2026-10-22T05:30:07.835Z Evidence SHA-256 `e895f6ccd0711bb1a512b580bab39de8f977c57f83bf492527e4fc710315d706` Source locator `/table/rows/1` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / environmental esg](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Aenvironmental_esg.md) | 0% | SE ±0 pp | 1 / 49 | $0.65 | 412.29source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.64771 \| USD \| per_test \| Lower is better \| \| `latency` \| 412.29 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 81 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label deepseek/deepseek-v4-pro Source version 1 Observed 2026-10-22T02:10:09.422Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/environmental_esg/deepseek~1deepseek-v4-pro` | | | | | | | | | [Legal Research Bench / weighted pass criminal](https://benchize.com/benchmarks/vals-ai%3Alegal_research%3Aweighted_pass__criminal.md) | 76.39% | SE ±2.89 pp | 38 / 50 | $0.62 | 1,004.09source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 76.39 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.615927 \| USD \| per_test \| Lower is better \| \| `latency` \| 1004.085 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.89 pp sourcePopulation 82 stderr 2.887 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) Source Vals AI Original model label deepseek/deepseek-v4-pro Source version 1 Observed 2026-10-22T02:10:07.934Z Evidence SHA-256 `26c58dfd47d0d2dc86f33c6d056f30309e0ee63570e7cd752c593b385f9d862e` Source locator `/tasks/weighted_pass__criminal/deepseek~1deepseek-v4-pro` | | | | | | | | | [Tax Agent Bench / weighted pass numbers calculations](https://benchize.com/benchmarks/vals-ai%3Atax_agent_bench%3Aweighted_pass__numbers_calculations.md) | 62.87% | SE ±7.4 pp | 38 / 46 | $0.26 | 455.91source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 62.87 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.260482 \| USD \| per_test \| Lower is better \| \| `latency` \| 455.905 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±7.4 pp sourcePopulation 73 stderr 7.399 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) Source Vals AI Original model label deepseek/deepseek-v4-pro Source version 1 Observed 2026-10-22T02:10:11.407Z Evidence SHA-256 `446db182b67efb7923b3187a4040ec3a3ea492b8ad699a2b6d708d714cc063ac` Source locator `/tasks/weighted_pass__numbers_calculations/deepseek~1deepseek-v4-pro` | | | | | | | | | [EMB / comps](https://benchize.com/benchmarks/vals-ai%3Aemb%3Acomps.md) | 57.43% | SE ±6.12 pp | 28 / 47 | $1.29 | 867.45source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 57.434 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 1.285259 \| USD \| per_test \| Lower is better \| \| `latency` \| 867.448 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±6.12 pp sourcePopulation 76 stderr 6.116 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) Source Vals AI Original model label deepseek/deepseek-v4-pro Source version 1 Observed 2026-10-22T02:10:12.907Z Evidence SHA-256 `d807aee620ea47152a1fca0257a8dd12a1be4512c76cc1744c1ae9e285180918` Source locator `/tasks/comps/deepseek~1deepseek-v4-pro` | | | | | | | | | [Public Benefits Bench v1 / both](https://benchize.com/benchmarks/vals-ai%3Apublic-benefits-bench-v1%3Aboth.md) | 57.58% | SE ±1.29 pp | 6 / 8 | $0.36 | 977.87source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_snap_v1.BPa0Hrre.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 57.578 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.359673 \| USD \| per_test \| Lower is better \| \| `latency` \| 977.871 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.29 pp sourcePopulation 13 stderr 1.286 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_snap_v1.BPa0Hrre.json) Source Vals AI Original model label deepseek/deepseek-v4-pro Source version 1 Observed 2026-10-22T02:10:32.821Z Evidence SHA-256 `b85e2b3cbdf03e13e2ffdb27ce4f108003a548d705a24ae26a0a115e2027475d` Source locator `/tasks/both/deepseek~1deepseek-v4-pro` | | | | | | | | | [Finance Agent (v2) / general qualitative analysis](https://benchize.com/benchmarks/vals-ai%3Afabv2%3Ageneral_qualitative_analysis.md) | 60.34% | SE ±1.51 pp | 34 / 50 | $0.96 | 1,682.11source duration | [Vals AI · 2](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 60.341 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.959616 \| USD \| per_test \| Lower is better \| \| `latency` \| 1682.108 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.51 pp sourcePopulation 80 stderr 1.505 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) Source Vals AI Original model label deepseek/deepseek-v4-pro Source version 2 Observed 2026-10-22T02:10:12.404Z Evidence SHA-256 `f731b20a26c126725d125519f2b5841aa45254656dba859a9570d7ce7e5d8191` Source locator `/tasks/general_qualitative_analysis/deepseek~1deepseek-v4-pro` | | | | | | | | | [ProgramBench / almost](https://benchize.com/benchmarks/vals-ai%3Aprogrambench%3Aalmost.md) | 0% | SE ±0 pp | 34 / 40 | — | 3,778.1source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_programbench.C_J7ZU4X.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `latency` \| 3778.098 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 64 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_programbench.C_J7ZU4X.json) Source Vals AI Original model label deepseek/deepseek-v4-pro Source version 1 Observed 2026-10-22T02:10:24.817Z Evidence SHA-256 `e9bbb4378c7a058441a21d456baf9316b96c2cf5b58d371a3c7c803f5000bd8c` Source locator `/tasks/almost/deepseek~1deepseek-v4-pro` | | | | | | | | | [SkillsBench / overall](https://benchize.com/benchmarks/vals-ai%3Askillsbench%3Aoverall.md) | 51.27% | SE ±4.6 pp | 16 / 20 | $0.4 | 495.56source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_skillsbench.BE2TXB-M.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 51.274 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.397123 \| USD \| per_test \| Lower is better \| \| `latency` \| 495.558 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness OpenHands maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±4.6 pp sourcePopulation 35 stderr 4.6 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_skillsbench.BE2TXB-M.json) Source Vals AI Original model label deepseek/deepseek-v4-pro Source version 1 Observed 2026-10-22T02:10:25.807Z Evidence SHA-256 `aaad242949d81b7c7c61c36d4895d4460e9146760c1e6b4b38d235a16043bc61` Source locator `/tasks/overall/deepseek~1deepseek-v4-pro` | | | | | | | | | [MMLU Pro / psychology](https://benchize.com/benchmarks/vals-ai%3Ammlu_pro%3Apsychology.md) | 87.97% | SE ±1.17 pp | 15 / 40 | $0.009 | 492.98source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_mmlu_pro.y416pXDi.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 87.97 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.009001 \| USD \| per_test \| Lower is better \| \| `latency` \| 492.984 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.17 pp sourcePopulation 138 stderr 1.167 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_mmlu_pro.y416pXDi.json) Source Vals AI Original model label deepseek/deepseek-v4-pro Source version 1 Observed 2026-10-22T02:10:21.194Z Evidence SHA-256 `dab025507f46c716dd8d145e90ec0a737d9c56219853f41e4b1cc3c5e8c222b4` Source locator `/tasks/psychology/deepseek~1deepseek-v4-pro` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / litigation dispute resolution](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Alitigation_dispute_resolution.md) | 0% | SE ±0 pp | 6 / 49 | $0.6 | 344.87source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.597517 \| USD \| per_test \| Lower is better \| \| `latency` \| 344.866 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 81 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label deepseek/deepseek-v4-pro Source version 1 Observed 2026-10-22T02:10:09.422Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/litigation_dispute_resolution/deepseek~1deepseek-v4-pro` | | | | | | | | | [CorpFin v2 / shared max context](https://benchize.com/benchmarks/vals-ai%3Acorp_fin_v2%3Ashared_max_context.md) | 61.89% | SE ±1.66 pp | 25 / 39 | $0.12 | 74.03source duration | [Vals AI · 2](https://www.vals.ai/_astro/benchmark_view_corp_fin_v2.CectKwl1.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 61.888 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.122195 \| USD \| per_test \| Lower is better \| \| `latency` \| 74.027 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.66 pp sourcePopulation 134 stderr 1.658 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_corp_fin_v2.CectKwl1.json) Source Vals AI Original model label deepseek/deepseek-v4-pro Source version 2 Observed 2026-10-22T02:10:13.429Z Evidence SHA-256 `7cf2217ab3728d5a84bffce82aab9fc52d7f810b0cbe8c65b387df47f3d31c9b` Source locator `/tasks/shared_max_context/deepseek~1deepseek-v4-pro` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / criteria pass rate data privacy cybersecurity](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__data_privacy_cybersecurity.md) | 85.87% | SE ±2.08 pp | 33 / 49 | $0.74 | 375.35source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 85.866 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.741361 \| USD \| per_test \| Lower is better \| \| `latency` \| 375.346 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.08 pp sourcePopulation 81 stderr 2.075 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label deepseek/deepseek-v4-pro Source version 1 Observed 2026-10-22T02:10:09.422Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__data_privacy_cybersecurity/deepseek~1deepseek-v4-pro` | | | | | | | | | [Tax Agent Bench / numbers calculations](https://benchize.com/benchmarks/vals-ai%3Atax_agent_bench%3Anumbers_calculations.md) | 33.67% | SE ±8.35 pp | 26 / 46 | $0.26 | 455.91source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 33.667 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.260482 \| USD \| per_test \| Lower is better \| \| `latency` \| 455.905 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±8.35 pp sourcePopulation 73 stderr 8.353 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) Source Vals AI Original model label deepseek/deepseek-v4-pro Source version 1 Observed 2026-10-22T02:10:11.407Z Evidence SHA-256 `446db182b67efb7923b3187a4040ec3a3ea492b8ad699a2b6d708d714cc063ac` Source locator `/tasks/numbers_calculations/deepseek~1deepseek-v4-pro` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / data privacy cybersecurity](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Adata_privacy_cybersecurity.md) | 0% | SE ±0 pp | 13 / 49 | $0.74 | 375.35source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.741361 \| USD \| per_test \| Lower is better \| \| `latency` \| 375.346 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 81 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label deepseek/deepseek-v4-pro Source version 1 Observed 2026-10-22T02:10:09.422Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/data_privacy_cybersecurity/deepseek~1deepseek-v4-pro` | | | | | | | | | [ProgramBench / partial](https://benchize.com/benchmarks/vals-ai%3Aprogrambench%3Apartial.md) | 47.78% | SE ±1.64 pp | 31 / 40 | — | 3,778.1source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_programbench.C_J7ZU4X.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 47.778 \| percent \| reported \| Higher is better \| \| `latency` \| 3778.098 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.64 pp sourcePopulation 64 stderr 1.636 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_programbench.C_J7ZU4X.json) Source Vals AI Original model label deepseek/deepseek-v4-pro Source version 1 Observed 2026-10-22T02:10:24.817Z Evidence SHA-256 `e9bbb4378c7a058441a21d456baf9316b96c2cf5b58d371a3c7c803f5000bd8c` Source locator `/tasks/partial/deepseek~1deepseek-v4-pro` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / corporate governance](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acorporate_governance.md) | 0% | SE ±0 pp | 11 / 49 | $0.59 | 355.34source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.585215 \| USD \| per_test \| Lower is better \| \| `latency` \| 355.344 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 81 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label deepseek/deepseek-v4-pro Source version 1 Observed 2026-10-22T02:10:09.422Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/corporate_governance/deepseek~1deepseek-v4-pro` | | | | | | | | | [Tax Agent Bench / controversy precedence analysis](https://benchize.com/benchmarks/vals-ai%3Atax_agent_bench%3Acontroversy_precedence_analysis.md) | 12.33% | SE ±5.91 pp | 16 / 46 | $0.7 | 1,303.85source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 12.333 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.703688 \| USD \| per_test \| Lower is better \| \| `latency` \| 1303.848 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±5.91 pp sourcePopulation 73 stderr 5.905 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) Source Vals AI Original model label deepseek/deepseek-v4-pro Source version 1 Observed 2026-10-22T02:10:11.407Z Evidence SHA-256 `446db182b67efb7923b3187a4040ec3a3ea492b8ad699a2b6d708d714cc063ac` Source locator `/tasks/controversy_precedence_analysis/deepseek~1deepseek-v4-pro` | | | | | | | | | [Code Migration / cobol](https://benchize.com/benchmarks/vals-ai%3Acode-migration%3Acobol.md) | 58.39% | SE ±15.33 pp | 33 / 47 | $0.96 | 1,712.4source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_code_migration.Dttzoqk3.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 58.393 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.959022 \| USD \| per_test \| Lower is better \| \| `latency` \| 1712.395 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±15.33 pp sourcePopulation 79 stderr 15.334 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_code_migration.Dttzoqk3.json) Source Vals AI Original model label deepseek/deepseek-v4-pro Source version 1 Observed 2026-10-22T02:10:27.307Z Evidence SHA-256 `99194597f2c76c402195be47839409046e18bd30454616da6dd40b736a088fa4` Source locator `/tasks/cobol/deepseek~1deepseek-v4-pro` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / criteria pass rate tax](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__tax.md) | 77.93% | SE ±2.4 pp | 24 / 49 | $0.77 | 381.64source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 77.926 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.767561 \| USD \| per_test \| Lower is better \| \| `latency` \| 381.644 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.4 pp sourcePopulation 81 stderr 2.403 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label deepseek/deepseek-v4-pro Source version 1 Observed 2026-10-22T02:10:09.422Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__tax/deepseek~1deepseek-v4-pro` | | | | | | | | | [GPQA Diamond / diamond zero shot cot](https://benchize.com/benchmarks/vals-ai%3Agpqa%3Adiamond_zero_shot_cot.md) | 88.38% | SE ±2.4 pp | 21 / 40 | $0.14 | 977.72source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_gpqa.ghZZBpyU.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 88.384 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.138778 \| USD \| per_test \| Lower is better \| \| `latency` \| 977.721 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.4 pp sourcePopulation 138 stderr 2.4 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_gpqa.ghZZBpyU.json) Source Vals AI Original model label deepseek/deepseek-v4-pro Source version 1 Observed 2026-10-22T02:10:20.661Z Evidence SHA-256 `951eb5224097733a9d09d06c14908ae2980e2e640e539f2c284877c5092d7077` Source locator `/tasks/diamond_zero_shot_cot/deepseek~1deepseek-v4-pro` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / antitrust competition](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Aantitrust_competition.md) | 0% | SE ±0 pp | 7 / 49 | $0.85 | 409.39source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.8503 \| USD \| per_test \| Lower is better \| \| `latency` \| 409.394 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 81 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label deepseek/deepseek-v4-pro Source version 1 Observed 2026-10-22T02:10:09.422Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/antitrust_competition/deepseek~1deepseek-v4-pro` | | | | | | | | | [LegalBench / rhetoric tasks](https://benchize.com/benchmarks/vals-ai%3Alegal_bench%3Arhetoric_tasks.md) | 75.32% | SE ±1.26 pp | 37 / 44 | $0.00301 | 209.99source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_bench.D6uTgA3b.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 75.316 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.003008 \| USD \| per_test \| Lower is better \| \| `latency` \| 209.987 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.26 pp sourcePopulation 151 stderr 1.259 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_bench.D6uTgA3b.json) Source Vals AI Original model label deepseek/deepseek-v4-pro Source version 1 Observed 2026-10-22T02:10:08.909Z Evidence SHA-256 `c8486ba756baa5cd3955d035edff67440e8db23e69d62ad1a9e18d15b8c4bc52` Source locator `/tasks/rhetoric_tasks/deepseek~1deepseek-v4-pro` | | | | | | | | | [TaxEval v2 / correctness](https://benchize.com/benchmarks/vals-ai%3Atax_eval_v2%3Acorrectness.md) | 60.59% | SE ±1.4 pp | 28 / 40 | $0.03 | 189.7source duration | [Vals AI · 2](https://www.vals.ai/_astro/benchmark_view_tax_eval_v2.B3yoffr8.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 60.589 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.033187 \| USD \| per_test \| Lower is better \| \| `latency` \| 189.695 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.4 pp sourcePopulation 145 stderr 1.397 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_tax_eval_v2.B3yoffr8.json) Source Vals AI Original model label deepseek/deepseek-v4-pro Source version 2 Observed 2026-10-22T02:10:14.414Z Evidence SHA-256 `8b354a6ae848f5bb25a2e8b061b5f576d5b9e0c11b941dc1e6bed078e6529cd6` Source locator `/tasks/correctness/deepseek~1deepseek-v4-pro` | | | | | | | | | [LiveCodeBench / hard](https://benchize.com/benchmarks/vals-ai%3Alcb%3Ahard.md) | 72.57% | SE ±2.39 pp | 15 / 43 | $0.24 | 1,487.69source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_lcb.miyRUvVa.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 72.571 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.23531 \| USD \| per_test \| Lower is better \| \| `latency` \| 1487.685 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.39 pp sourcePopulation 143 stderr 2.385 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_lcb.miyRUvVa.json) Source Vals AI Original model label deepseek/deepseek-v4-pro Source version 1 Observed 2026-10-22T02:10:28.824Z Evidence SHA-256 `1a7c7dabc9d95a0182886b1edc42623987cac2095a0d407a5e334f37acdfa2f5` Source locator `/tasks/hard/deepseek~1deepseek-v4-pro` | | | | | | | | | [Finance Agent (v2) / all pass](https://benchize.com/benchmarks/vals-ai%3Afabv2%3Aall_pass.md) | 31.45% | SE ±0.72 pp | 38 / 50 | $0.88 | 1,526.18source duration | [Vals AI · 2](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 31.448 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.879086 \| USD \| per_test \| Lower is better \| \| `latency` \| 1526.183 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.72 pp sourcePopulation 80 stderr 0.715 tokenTotals {"cache_read_tokens":284032811,"cache_write_tokens":null,"input_tokens":508078472,"output_tokens":22058372,"reasoning_tokens":17233163} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) Source Vals AI Original model label deepseek/deepseek-v4-pro Source version 2 Observed 2026-10-22T02:10:12.404Z Evidence SHA-256 `f731b20a26c126725d125519f2b5841aa45254656dba859a9570d7ce7e5d8191` Source locator `/tasks/all_pass/deepseek~1deepseek-v4-pro` | | | | | | | | | [LiveCodeBench / easy](https://benchize.com/benchmarks/vals-ai%3Alcb%3Aeasy.md) | 98.76% | SE ±0.62 pp | 14 / 43 | $0.01 | 73.49source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_lcb.miyRUvVa.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 98.758 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.010782 \| USD \| per_test \| Lower is better \| \| `latency` \| 73.491 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.62 pp sourcePopulation 143 stderr 0.617 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_lcb.miyRUvVa.json) Source Vals AI Original model label deepseek/deepseek-v4-pro Source version 1 Observed 2026-10-22T02:10:28.824Z Evidence SHA-256 `1a7c7dabc9d95a0182886b1edc42623987cac2095a0d407a5e334f37acdfa2f5` Source locator `/tasks/easy/deepseek~1deepseek-v4-pro` | | | | | | | | | [MedCode / overall](https://benchize.com/benchmarks/vals-ai%3Amedcode%3Aoverall.md) | 40.46% | SE ±2.12 pp | 36 / 49 | $0.06 | 382.67source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_medcode.DehpEDgG.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 40.455 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.06071 \| USD \| per_test \| Lower is better \| \| `latency` \| 382.667 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.12 pp sourcePopulation 107 stderr 2.122 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_medcode.DehpEDgG.json) Source Vals AI Original model label deepseek/deepseek-v4-pro Source version 1 Observed 2026-10-22T02:10:14.899Z Evidence SHA-256 `988a6ae1e07e33ce031572ed4a804be11c36ee97a1dd75cbe4740fff3b534f8c` Source locator `/tasks/overall/deepseek~1deepseek-v4-pro` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / trusts estates private client](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Atrusts_estates_private_client.md) | 0% | SE ±0 pp | 15 / 49 | $0.77 | 462.12source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.765564 \| USD \| per_test \| Lower is better \| \| `latency` \| 462.122 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 81 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label deepseek/deepseek-v4-pro Source version 1 Observed 2026-10-22T02:10:09.422Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/trusts_estates_private_client/deepseek~1deepseek-v4-pro` | | | | | | | | | [MMLU Pro / law](https://benchize.com/benchmarks/vals-ai%3Ammlu_pro%3Alaw.md) | 74.02% | SE ±1.35 pp | 21 / 40 | $0.03 | 535.64source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_mmlu_pro.y416pXDi.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 74.024 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.02826 \| USD \| per_test \| Lower is better \| \| `latency` \| 535.644 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.35 pp sourcePopulation 138 stderr 1.345 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_mmlu_pro.y416pXDi.json) Source Vals AI Original model label deepseek/deepseek-v4-pro Source version 1 Observed 2026-10-22T02:10:21.194Z Evidence SHA-256 `dab025507f46c716dd8d145e90ec0a737d9c56219853f41e4b1cc3c5e8c222b4` Source locator `/tasks/law/deepseek~1deepseek-v4-pro` | | | | | | | | | [MedScribe / overall](https://benchize.com/benchmarks/vals-ai%3Amedscribe%3Aoverall.md) | 75.14% | SE ±2 pp | 38 / 49 | $0.05 | 345.72source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_medscribe.BYf5a3C-.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 75.144 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.053954 \| USD \| per_test \| Lower is better \| \| `latency` \| 345.723 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2 pp sourcePopulation 109 stderr 2.002 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_medscribe.BYf5a3C-.json) Source Vals AI Original model label deepseek/deepseek-v4-pro Source version 1 Observed 2026-10-22T02:10:15.398Z Evidence SHA-256 `271fc122ddc2b0602bd6c7947f65a9a7f0d8cab94f6dc27d62a9380a9317f9c0` Source locator `/tasks/overall/deepseek~1deepseek-v4-pro` | | | | | | | | | [ProgramBench / strict](https://benchize.com/benchmarks/vals-ai%3Aprogrambench%3Astrict.md) | 0% | SE ±0 pp | 23 / 40 | — | 3,778.1source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_programbench.C_J7ZU4X.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `latency` \| 3778.098 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 64 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_programbench.C_J7ZU4X.json) Source Vals AI Original model label deepseek/deepseek-v4-pro Source version 1 Observed 2026-10-22T02:10:24.817Z Evidence SHA-256 `e9bbb4378c7a058441a21d456baf9316b96c2cf5b58d371a3c7c803f5000bd8c` Source locator `/tasks/strict/deepseek~1deepseek-v4-pro` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / banking finance](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Abanking_finance.md) | 0% | SE ±0 pp | 5 / 49 | $0.61 | 311.9source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.609606 \| USD \| per_test \| Lower is better \| \| `latency` \| 311.896 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 81 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label deepseek/deepseek-v4-pro Source version 1 Observed 2026-10-22T02:10:09.422Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/banking_finance/deepseek~1deepseek-v4-pro` | | | | | | | | [Next page](https://benchize.com/models/deepseek-v4-pro-max.md?cursor=7955640d1e0cbf3c9a13f692b6907f60e3bc3cf6a30af51e399e6b8f41b2db59) --- Cite as: DeepSeek V4 ProMax — Benchize, https://benchize.com/models/deepseek-v4-pro-max (retrieved 2026-10-22)