# DeepSeek V4.1 Flash 1017Max > Source: Benchize (https://benchize.com) — https://benchize.com/models/deepseek-v4-1-flash-1017-max. When citing or reusing this content, credit "Benchize" and link to https://benchize.com/models/deepseek-v4-1-flash-1017-max. DeepSeek Canonical URL: https://benchize.com/models/deepseek-v4-1-flash-1017-max Last updated: Wed, 21 Oct 2026 18:52:20 GMT Reproduction and quotation must name Benchize, link to the canonical HTML URL, and retain individual source citations. DeepSeek ## Benchmarks | Benchmark | Score | ± | Rank | Cost | Latency | Source | | | --- | --- | --- | --- | --- | --- | --- | --- | | [Finance Agent v2.1 / overall](https://benchize.com/benchmarks/vals-ai%3Afabv2-1%3Aoverall.md) | 62.93% | SE ±2.03 pp | 10 / 18 | $0.47 | 909.21source duration | [Vals AI · 2.1](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2_1.ZbMhy9xo.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 62.933 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.466261 \| USD \| per_test \| Lower is better \| \| `latency` \| 909.212 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.03 pp sourcePopulation 28 stderr 2.028 tokenTotals {"cache_read_tokens":721979776,"cache_write_tokens":null,"input_tokens":1025601275,"output_tokens":94944123,"reasoning_tokens":82978352} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2_1.ZbMhy9xo.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash-1017 Source version 2.1 Observed 2026-10-21T16:45:54.954Z Evidence SHA-256 `f1b0c2d7dca4e73b52136862a9be059f7c8481c69f15d705ce90bf3bd9f78429` Source locator `/tasks/overall/deepseek~1deepseek-v4.1-flash-1017` | | | | | | | | | [HLAB-Vals-1.1 / funds asset management](https://benchize.com/benchmarks/vals-ai%3Ahlab%3Afunds_asset_management.md) | 40% | SE ±18.71 pp | 2 / 14 | $0.65 | 1,356.48source duration | [Vals AI · 1.1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 40 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.654524 \| USD \| per_test \| Lower is better \| \| `latency` \| 1356.48 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±18.71 pp sourcePopulation 24 stderr 18.708 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash-1017 Source version 1.1 Observed 2026-10-21T16:45:52.890Z Evidence SHA-256 `14681f4a2310e0f257df9ec07c9c81e5dadc8f6f71634f02e174a02a8096d868` Source locator `/tasks/funds_asset_management/deepseek~1deepseek-v4.1-flash-1017` | | | | | | | | | [HLAB-Vals-1.1 / criteria pass rate antitrust competition](https://benchize.com/benchmarks/vals-ai%3Ahlab%3Acriteria_pass_rate__antitrust_competition.md) | 93.48% | SE ±1.49 pp | 4 / 14 | $0.53 | 1,264.71source duration | [Vals AI · 1.1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 93.478 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.534289 \| USD \| per_test \| Lower is better \| \| `latency` \| 1264.71 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.49 pp sourcePopulation 24 stderr 1.489 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash-1017 Source version 1.1 Observed 2026-10-21T16:45:52.890Z Evidence SHA-256 `14681f4a2310e0f257df9ec07c9c81e5dadc8f6f71634f02e174a02a8096d868` Source locator `/tasks/criteria_pass_rate__antitrust_competition/deepseek~1deepseek-v4.1-flash-1017` | | | | | | | | | [Finance Agent v2.1 / precedents](https://benchize.com/benchmarks/vals-ai%3Afabv2-1%3Aprecedents.md) | 29.35% | SE ±6.67 pp | 17 / 18 | $0.71 | 1,361.05source duration | [Vals AI · 2.1](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2_1.ZbMhy9xo.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 29.348 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.709077 \| USD \| per_test \| Lower is better \| \| `latency` \| 1361.053 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±6.67 pp sourcePopulation 28 stderr 6.67 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2_1.ZbMhy9xo.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash-1017 Source version 2.1 Observed 2026-10-21T16:45:54.954Z Evidence SHA-256 `f1b0c2d7dca4e73b52136862a9be059f7c8481c69f15d705ce90bf3bd9f78429` Source locator `/tasks/precedents/deepseek~1deepseek-v4.1-flash-1017` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / antitrust competition](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Aantitrust_competition.md) | 0% | SE ±0 pp | 7 / 49 | $0.53 | 1,282.79source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.527766 \| USD \| per_test \| Lower is better \| \| `latency` \| 1282.794 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 81 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash-1017 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/antitrust_competition/deepseek~1deepseek-v4.1-flash-1017` | | | | | | | | | [Finance Agent (v2) / all pass](https://benchize.com/benchmarks/vals-ai%3Afabv2%3Aall_pass.md) | 45.24% | SE ±2.16 pp | 15 / 50 | $0.46 | 879.26source duration | [Vals AI · 2](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 45.244 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.462841 \| USD \| per_test \| Lower is better \| \| `latency` \| 879.256 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.16 pp sourcePopulation 80 stderr 2.16 tokenTotals {"cache_read_tokens":898992768,"cache_write_tokens":null,"input_tokens":1221527142,"output_tokens":88436785,"reasoning_tokens":76225695} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash-1017 Source version 2 Observed 2026-10-21T16:45:57.035Z Evidence SHA-256 `f731b20a26c126725d125519f2b5841aa45254656dba859a9570d7ce7e5d8191` Source locator `/tasks/all_pass/deepseek~1deepseek-v4.1-flash-1017` | | | | | | | | | [Arena webdev / webdev-react](https://benchize.com/benchmarks/lmarena%3Awebdev%3Awebdev-react.md) | 1,666.68 | — | 14 / 81 | — | — | [Arena](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/webdev/latest) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `rating` \| 1666.679694717695 \| rating \| reported \| Higher is better \| ### Configuration category webdev-react config webdev Reasoning effort max thinkingEnabled true ### Reported statistics lower 1648.6804529046844 rank 16 upper 1684.6789365307054 variance 84.33585285308077 votes 1255 ### Source [Original source](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/webdev/latest) Source Arena Original model label DeepSeek-V4.1-Flash-20261017 (Max) Observed 2026-10-21T18:45:05.190Z Evidence SHA-256 `f82507859808110820a4ab6df15561b8864137ddefc1f6fb0c552af2e58ea738` Source locator `/rows/522/row` | | | | | | | | | [HLAB-Vals-1.1 / criteria pass rate funds asset management](https://benchize.com/benchmarks/vals-ai%3Ahlab%3Acriteria_pass_rate__funds_asset_management.md) | 89.8% | SE ±1.64 pp | 5 / 14 | $0.65 | 1,356.48source duration | [Vals AI · 1.1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 89.796 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.654524 \| USD \| per_test \| Lower is better \| \| `latency` \| 1356.48 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.64 pp sourcePopulation 24 stderr 1.637 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash-1017 Source version 1.1 Observed 2026-10-21T16:45:52.890Z Evidence SHA-256 `14681f4a2310e0f257df9ec07c9c81e5dadc8f6f71634f02e174a02a8096d868` Source locator `/tasks/criteria_pass_rate__funds_asset_management/deepseek~1deepseek-v4.1-flash-1017` | | | | | | | | | [HLAB-Vals-1.1 / overall](https://benchize.com/benchmarks/vals-ai%3Ahlab%3Aoverall.md) | 14.17% | SE ±2.79 pp | 3 / 14 | $0.61 | 1,328.11source duration | [Vals AI · 1.1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 14.167 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.612596 \| USD \| per_test \| Lower is better \| \| `latency` \| 1328.108 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.79 pp sourcePopulation 24 stderr 2.787 tokenTotals {"cache_read_tokens":5492827392,"cache_write_tokens":null,"input_tokens":5524481146,"output_tokens":25881985,"reasoning_tokens":10520362} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash-1017 Source version 1.1 Observed 2026-10-21T16:45:52.890Z Evidence SHA-256 `14681f4a2310e0f257df9ec07c9c81e5dadc8f6f71634f02e174a02a8096d868` Source locator `/tasks/overall/deepseek~1deepseek-v4.1-flash-1017` | | | | | | | | | [Finance Agent v2.1 / adjustments](https://benchize.com/benchmarks/vals-ai%3Afabv2-1%3Aadjustments.md) | 66.77% | SE ±6.59 pp | 4 / 18 | $0.51 | 954.03source duration | [Vals AI · 2.1](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2_1.ZbMhy9xo.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 66.771 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.506283 \| USD \| per_test \| Lower is better \| \| `latency` \| 954.029 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±6.59 pp sourcePopulation 28 stderr 6.591 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2_1.ZbMhy9xo.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash-1017 Source version 2.1 Observed 2026-10-21T16:45:54.954Z Evidence SHA-256 `f1b0c2d7dca4e73b52136862a9be059f7c8481c69f15d705ce90bf3bd9f78429` Source locator `/tasks/adjustments/deepseek~1deepseek-v4.1-flash-1017` | | | | | | | | | [HLAB-Vals-1.1 / bankruptcy restructuring](https://benchize.com/benchmarks/vals-ai%3Ahlab%3Abankruptcy_restructuring.md) | 10% | SE ±10 pp | 4 / 14 | $0.64 | 1,285.44source duration | [Vals AI · 1.1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 10 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.640206 \| USD \| per_test \| Lower is better \| \| `latency` \| 1285.442 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±10 pp sourcePopulation 24 stderr 10 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash-1017 Source version 1.1 Observed 2026-10-21T16:45:52.890Z Evidence SHA-256 `14681f4a2310e0f257df9ec07c9c81e5dadc8f6f71634f02e174a02a8096d868` Source locator `/tasks/bankruptcy_restructuring/deepseek~1deepseek-v4.1-flash-1017` | | | | | | | | | [Vals Index / legal agent benchmark](https://benchize.com/benchmarks/vals-ai%3Avals_index%3Alegal_agent_benchmark.md) | 14.17% | SE ±2.79 pp | 3 / 14 | $0.61 | 1,328.11source duration | [Vals AI · 2.2](https://www.vals.ai/_astro/benchmark_view_vals_index.ye6C1wH2.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 14.167 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.612596 \| USD \| per_test \| Lower is better \| \| `latency` \| 1328.108 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.79 pp sourcePopulation 23 stderr 2.787 tokenTotals {"cache_read_tokens":5492827392,"cache_write_tokens":null,"input_tokens":5524481146,"output_tokens":25881985,"reasoning_tokens":10520362} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_vals_index.ye6C1wH2.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash-1017 Source version 2.2 Observed 2026-10-21T16:45:49.430Z Evidence SHA-256 `0e95aee94cd2bf34bc4b43a8f4bb16c7817de138613c21011873fea4e0786b45` Source locator `/tasks/legal_agent_benchmark/deepseek~1deepseek-v4.1-flash-1017` | | | | | | | | | [HLAB-Vals-1.1 / criteria pass rate capital markets](https://benchize.com/benchmarks/vals-ai%3Ahlab%3Acriteria_pass_rate__capital_markets.md) | 94.12% | SE ±1.48 pp | 8 / 14 | $0.6 | 1,270.29source duration | [Vals AI · 1.1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 94.118 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.604559 \| USD \| per_test \| Lower is better \| \| `latency` \| 1270.292 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.48 pp sourcePopulation 24 stderr 1.476 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash-1017 Source version 1.1 Observed 2026-10-21T16:45:52.890Z Evidence SHA-256 `14681f4a2310e0f257df9ec07c9c81e5dadc8f6f71634f02e174a02a8096d868` Source locator `/tasks/criteria_pass_rate__capital_markets/deepseek~1deepseek-v4.1-flash-1017` | | | | | | | | | [HLAB-Vals-1.1 / criteria pass rate international trade sanctions](https://benchize.com/benchmarks/vals-ai%3Ahlab%3Acriteria_pass_rate__international_trade_sanctions.md) | 94.82% | SE ±1.4 pp | 3 / 14 | $0.61 | 1,463.82source duration | [Vals AI · 1.1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 94.821 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.612386 \| USD \| per_test \| Lower is better \| \| `latency` \| 1463.816 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.4 pp sourcePopulation 24 stderr 1.402 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash-1017 Source version 1.1 Observed 2026-10-21T16:45:52.890Z Evidence SHA-256 `14681f4a2310e0f257df9ec07c9c81e5dadc8f6f71634f02e174a02a8096d868` Source locator `/tasks/criteria_pass_rate__international_trade_sanctions/deepseek~1deepseek-v4.1-flash-1017` | | | | | | | | | [HLAB-Vals-1.1 / structured finance securitization](https://benchize.com/benchmarks/vals-ai%3Ahlab%3Astructured_finance_securitization.md) | 20% | SE ±20 pp | 3 / 14 | $0.58 | 1,214.13source duration | [Vals AI · 1.1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 20 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.5812 \| USD \| per_test \| Lower is better \| \| `latency` \| 1214.128 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±20 pp sourcePopulation 24 stderr 20 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash-1017 Source version 1.1 Observed 2026-10-21T16:45:52.890Z Evidence SHA-256 `14681f4a2310e0f257df9ec07c9c81e5dadc8f6f71634f02e174a02a8096d868` Source locator `/tasks/structured_finance_securitization/deepseek~1deepseek-v4.1-flash-1017` | | | | | | | | | [Finance Agent v2.1 / comparables](https://benchize.com/benchmarks/vals-ai%3Afabv2-1%3Acomparables.md) | 59.75% | SE ±6.38 pp | 2 / 18 | $0.52 | 1,064.8source duration | [Vals AI · 2.1](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2_1.ZbMhy9xo.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 59.751 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.51929 \| USD \| per_test \| Lower is better \| \| `latency` \| 1064.803 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±6.38 pp sourcePopulation 28 stderr 6.375 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2_1.ZbMhy9xo.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash-1017 Source version 2.1 Observed 2026-10-21T16:45:54.954Z Evidence SHA-256 `f1b0c2d7dca4e73b52136862a9be059f7c8481c69f15d705ce90bf3bd9f78429` Source locator `/tasks/comparables/deepseek~1deepseek-v4.1-flash-1017` | | | | | | | | | [HLAB-Vals-1.1 / criteria pass rate corporate ma](https://benchize.com/benchmarks/vals-ai%3Ahlab%3Acriteria_pass_rate__corporate_ma.md) | 95.34% | SE ±1.2 pp | 9 / 14 | $0.64 | 1,348.01source duration | [Vals AI · 1.1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 95.338 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.64235 \| USD \| per_test \| Lower is better \| \| `latency` \| 1348.014 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.2 pp sourcePopulation 24 stderr 1.197 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash-1017 Source version 1.1 Observed 2026-10-21T16:45:52.890Z Evidence SHA-256 `14681f4a2310e0f257df9ec07c9c81e5dadc8f6f71634f02e174a02a8096d868` Source locator `/tasks/criteria_pass_rate__corporate_ma/deepseek~1deepseek-v4.1-flash-1017` | | | | | | | | | [HLAB-Vals-1.1 / antitrust competition](https://benchize.com/benchmarks/vals-ai%3Ahlab%3Aantitrust_competition.md) | 30% | SE ±20 pp | 2 / 14 | $0.53 | 1,264.71source duration | [Vals AI · 1.1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 30 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.534289 \| USD \| per_test \| Lower is better \| \| `latency` \| 1264.71 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±20 pp sourcePopulation 24 stderr 20 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash-1017 Source version 1.1 Observed 2026-10-21T16:45:52.890Z Evidence SHA-256 `14681f4a2310e0f257df9ec07c9c81e5dadc8f6f71634f02e174a02a8096d868` Source locator `/tasks/antitrust_competition/deepseek~1deepseek-v4.1-flash-1017` | | | | | | | | | [Vals Index / emb](https://benchize.com/benchmarks/vals-ai%3Avals_index%3Aemb.md) | 69.17% | SE ±2.4 pp | 9 / 14 | $0.33 | 1,971.69source duration | [Vals AI · 2.2](https://www.vals.ai/_astro/benchmark_view_vals_index.ye6C1wH2.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 69.173 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.330922 \| USD \| per_test \| Lower is better \| \| `latency` \| 1971.687 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.4 pp sourcePopulation 23 stderr 2.395 tokenTotals {"cache_read_tokens":1797102837,"cache_write_tokens":null,"input_tokens":1809205546,"output_tokens":16117195,"reasoning_tokens":11510477} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_vals_index.ye6C1wH2.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash-1017 Source version 2.2 Observed 2026-10-21T16:45:49.430Z Evidence SHA-256 `0e95aee94cd2bf34bc4b43a8f4bb16c7817de138613c21011873fea4e0786b45` Source locator `/tasks/emb/deepseek~1deepseek-v4.1-flash-1017` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / banking finance](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Abanking_finance.md) | 0% | SE ±0 pp | 5 / 49 | $0.51 | 1,164.46source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.505282 \| USD \| per_test \| Lower is better \| \| `latency` \| 1164.462 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 81 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash-1017 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/banking_finance/deepseek~1deepseek-v4.1-flash-1017` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / criteria pass rate emerging companies venture capital](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__emerging_companies_venture_capital.md) | 92.15% | SE ±1.57 pp | 9 / 49 | $0.58 | 1,429.54source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 92.15 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.581828 \| USD \| per_test \| Lower is better \| \| `latency` \| 1429.538 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.57 pp sourcePopulation 81 stderr 1.574 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash-1017 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__emerging_companies_venture_capital/deepseek~1deepseek-v4.1-flash-1017` | | | | | | | | | [Code Migration v1.1 / cli](https://benchize.com/benchmarks/vals-ai%3Acode-migration-v1-1%3Acli.md) | 59.2% | SE ±2.86 pp | 7 / 17 | $2.19 | 10,029.73source duration | [Vals AI · 1.1](https://www.vals.ai/_astro/benchmark_view_code_migration_v1_1.bJHk6HEY.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 59.199 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 2.193325 \| USD \| per_test \| Lower is better \| \| `latency` \| 10029.729 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.86 pp sourcePopulation 26 stderr 2.861 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_code_migration_v1_1.bJHk6HEY.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash-1017 Source version 1.1 Observed 2026-10-21T16:46:09.631Z Evidence SHA-256 `352faac603c73f05cc01e89214fb4adcafa05ea7432ea71ea9903a6b5fe36b08` Source locator `/tasks/cli/deepseek~1deepseek-v4.1-flash-1017` | | | | | | | | | [Finance Agent v2.1 / disclosure analysis](https://benchize.com/benchmarks/vals-ai%3Afabv2-1%3Adisclosure_analysis.md) | 70.26% | SE ±6.15 pp | 12 / 18 | $0.65 | 1,178.03source duration | [Vals AI · 2.1](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2_1.ZbMhy9xo.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 70.262 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.654078 \| USD \| per_test \| Lower is better \| \| `latency` \| 1178.026 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±6.15 pp sourcePopulation 28 stderr 6.153 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2_1.ZbMhy9xo.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash-1017 Source version 2.1 Observed 2026-10-21T16:45:54.954Z Evidence SHA-256 `f1b0c2d7dca4e73b52136862a9be059f7c8481c69f15d705ce90bf3bd9f78429` Source locator `/tasks/disclosure_analysis/deepseek~1deepseek-v4.1-flash-1017` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / criteria pass rate](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate.md) | 89.74% | SE ±0.37 pp | 10 / 49 | $0.54 | 1,298.99source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 89.738 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.543702 \| USD \| per_test \| Lower is better \| \| `latency` \| 1298.986 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.37 pp sourcePopulation 81 stderr 0.365 tokenTotals {"cache_read_tokens":4425635584,"cache_write_tokens":null,"input_tokens":4452935434,"output_tokens":25417039,"reasoning_tokens":10306539} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash-1017 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate/deepseek~1deepseek-v4.1-flash-1017` | | | | | | | | | [Finance Agent v2.1 / all pass](https://benchize.com/benchmarks/vals-ai%3Afabv2-1%3Aall_pass.md) | 51.35% | SE ±2.2 pp | 8 / 18 | $0.47 | 909.21source duration | [Vals AI · 2.1](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2_1.ZbMhy9xo.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 51.345 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.466261 \| USD \| per_test \| Lower is better \| \| `latency` \| 909.212 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.2 pp sourcePopulation 28 stderr 2.198 tokenTotals {"cache_read_tokens":721979776,"cache_write_tokens":null,"input_tokens":1025601275,"output_tokens":94944123,"reasoning_tokens":82978352} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2_1.ZbMhy9xo.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash-1017 Source version 2.1 Observed 2026-10-21T16:45:54.954Z Evidence SHA-256 `f1b0c2d7dca4e73b52136862a9be059f7c8481c69f15d705ce90bf3bd9f78429` Source locator `/tasks/all_pass/deepseek~1deepseek-v4.1-flash-1017` | | | | | | | | | [HLAB-Vals-1.1 / criteria pass rate data privacy cybersecurity](https://benchize.com/benchmarks/vals-ai%3Ahlab%3Acriteria_pass_rate__data_privacy_cybersecurity.md) | 97.53% | SE ±0.93 pp | 4 / 14 | $0.53 | 1,228.25source duration | [Vals AI · 1.1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 97.527 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.52666 \| USD \| per_test \| Lower is better \| \| `latency` \| 1228.252 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.93 pp sourcePopulation 24 stderr 0.925 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash-1017 Source version 1.1 Observed 2026-10-21T16:45:52.890Z Evidence SHA-256 `14681f4a2310e0f257df9ec07c9c81e5dadc8f6f71634f02e174a02a8096d868` Source locator `/tasks/criteria_pass_rate__data_privacy_cybersecurity/deepseek~1deepseek-v4.1-flash-1017` | | | | | | | | | [Legal Research Bench / family](https://benchize.com/benchmarks/vals-ai%3Alegal_research%3Afamily.md) | 27.27% | SE ±9.72 pp | 8 / 50 | $0.83 | 2,445.95source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 27.273 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.826784 \| USD \| per_test \| Lower is better \| \| `latency` \| 2445.946 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±9.72 pp sourcePopulation 82 stderr 9.719 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash-1017 Source version 1 Observed 2026-10-21T16:45:52.389Z Evidence SHA-256 `26c58dfd47d0d2dc86f33c6d056f30309e0ee63570e7cd752c593b385f9d862e` Source locator `/tasks/family/deepseek~1deepseek-v4.1-flash-1017` | | | | | | | | | [HLAB-Vals-1.1 / criteria pass rate corporate governance](https://benchize.com/benchmarks/vals-ai%3Ahlab%3Acriteria_pass_rate__corporate_governance.md) | 90.58% | SE ±1.71 pp | 10 / 14 | $0.7 | 1,497.44source duration | [Vals AI · 1.1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 90.582 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.70205 \| USD \| per_test \| Lower is better \| \| `latency` \| 1497.44 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.71 pp sourcePopulation 24 stderr 1.712 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash-1017 Source version 1.1 Observed 2026-10-21T16:45:52.890Z Evidence SHA-256 `14681f4a2310e0f257df9ec07c9c81e5dadc8f6f71634f02e174a02a8096d868` Source locator `/tasks/criteria_pass_rate__corporate_governance/deepseek~1deepseek-v4.1-flash-1017` | | | | | | | | | [HLAB-Vals-1.1 / criteria pass rate banking finance](https://benchize.com/benchmarks/vals-ai%3Ahlab%3Acriteria_pass_rate__banking_finance.md) | 74.81% | SE ±2.66 pp | 13 / 14 | $0.6 | 1,273.96source duration | [Vals AI · 1.1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 74.813 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.604563 \| USD \| per_test \| Lower is better \| \| `latency` \| 1273.962 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.66 pp sourcePopulation 24 stderr 2.657 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash-1017 Source version 1.1 Observed 2026-10-21T16:45:52.890Z Evidence SHA-256 `14681f4a2310e0f257df9ec07c9c81e5dadc8f6f71634f02e174a02a8096d868` Source locator `/tasks/criteria_pass_rate__banking_finance/deepseek~1deepseek-v4.1-flash-1017` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / criteria pass rate antitrust competition](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__antitrust_competition.md) | 90.94% | SE ±1.73 pp | 10 / 49 | $0.53 | 1,282.79source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 90.942 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.527766 \| USD \| per_test \| Lower is better \| \| `latency` \| 1282.794 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.73 pp sourcePopulation 81 stderr 1.731 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash-1017 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__antitrust_competition/deepseek~1deepseek-v4.1-flash-1017` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / capital markets](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acapital_markets.md) | 0% | SE ±0 pp | 16 / 49 | $0.54 | 1,169.73source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.539779 \| USD \| per_test \| Lower is better \| \| `latency` \| 1169.734 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 81 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash-1017 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/capital_markets/deepseek~1deepseek-v4.1-flash-1017` | | | | | | | | | [HLAB-Vals-1.1 / criteria pass rate healthcare life sciences](https://benchize.com/benchmarks/vals-ai%3Ahlab%3Acriteria_pass_rate__healthcare_life_sciences.md) | 95.97% | SE ±1.14 pp | 2 / 14 | $0.66 | 1,457.21source duration | [Vals AI · 1.1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 95.973 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.656838 \| USD \| per_test \| Lower is better \| \| `latency` \| 1457.208 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.14 pp sourcePopulation 24 stderr 1.141 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash-1017 Source version 1.1 Observed 2026-10-21T16:45:52.890Z Evidence SHA-256 `14681f4a2310e0f257df9ec07c9c81e5dadc8f6f71634f02e174a02a8096d868` Source locator `/tasks/criteria_pass_rate__healthcare_life_sciences/deepseek~1deepseek-v4.1-flash-1017` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / criteria pass rate litigation dispute resolution](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__litigation_dispute_resolution.md) | 87.64% | SE ±2.02 pp | 16 / 49 | $0.55 | 1,236.26source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 87.64 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.551954 \| USD \| per_test \| Lower is better \| \| `latency` \| 1236.258 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.02 pp sourcePopulation 81 stderr 2.018 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash-1017 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__litigation_dispute_resolution/deepseek~1deepseek-v4.1-flash-1017` | | | | | | | | | [HLAB-Vals-1.1 / criteria pass rate intellectual property](https://benchize.com/benchmarks/vals-ai%3Ahlab%3Acriteria_pass_rate__intellectual_property.md) | 94.84% | SE ±1.32 pp | 8 / 14 | $0.6 | 1,311.01source duration | [Vals AI · 1.1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 94.84 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.601227 \| USD \| per_test \| Lower is better \| \| `latency` \| 1311.01 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.32 pp sourcePopulation 24 stderr 1.322 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash-1017 Source version 1.1 Observed 2026-10-21T16:45:52.890Z Evidence SHA-256 `14681f4a2310e0f257df9ec07c9c81e5dadc8f6f71634f02e174a02a8096d868` Source locator `/tasks/criteria_pass_rate__intellectual_property/deepseek~1deepseek-v4.1-flash-1017` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / criteria pass rate banking finance](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__banking_finance.md) | 89.59% | SE ±1.87 pp | 17 / 49 | $0.51 | 1,164.46source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 89.591 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.505282 \| USD \| per_test \| Lower is better \| \| `latency` \| 1164.462 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.87 pp sourcePopulation 81 stderr 1.865 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash-1017 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__banking_finance/deepseek~1deepseek-v4.1-flash-1017` | | | | | | | | | [HLAB-Vals-1.1 / criteria pass rate arbitration international dispute resolution](https://benchize.com/benchmarks/vals-ai%3Ahlab%3Acriteria_pass_rate__arbitration_international_dispute_resolution.md) | 91.76% | SE ±1.65 pp | 7 / 14 | $0.67 | 1,273.8source duration | [Vals AI · 1.1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 91.756 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.66721 \| USD \| per_test \| Lower is better \| \| `latency` \| 1273.796 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.65 pp sourcePopulation 24 stderr 1.65 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash-1017 Source version 1.1 Observed 2026-10-21T16:45:52.890Z Evidence SHA-256 `14681f4a2310e0f257df9ec07c9c81e5dadc8f6f71634f02e174a02a8096d868` Source locator `/tasks/criteria_pass_rate__arbitration_international_dispute_resolution/deepseek~1deepseek-v4.1-flash-1017` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / criteria pass rate arbitration international dispute resolution](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__arbitration_international_dispute_resolution.md) | 88.89% | SE ±1.89 pp | 16 / 49 | $0.54 | 1,158.84source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 88.889 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.541479 \| USD \| per_test \| Lower is better \| \| `latency` \| 1158.842 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.89 pp sourcePopulation 81 stderr 1.885 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash-1017 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__arbitration_international_dispute_resolution/deepseek~1deepseek-v4.1-flash-1017` | | | | | | | | | [Tax Agent Bench / numbers calculations](https://benchize.com/benchmarks/vals-ai%3Atax_agent_bench%3Anumbers_calculations.md) | 45.67% | SE ±9.12 pp | 15 / 46 | $0.24 | 677.79source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 45.667 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.241521 \| USD \| per_test \| Lower is better \| \| `latency` \| 677.794 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±9.12 pp sourcePopulation 73 stderr 9.118 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash-1017 Source version 1 Observed 2026-10-21T16:45:55.958Z Evidence SHA-256 `446db182b67efb7923b3187a4040ec3a3ea492b8ad699a2b6d708d714cc063ac` Source locator `/tasks/numbers_calculations/deepseek~1deepseek-v4.1-flash-1017` | | | | | | | | | [Finance Agent v2.1 / general qualitative analysis](https://benchize.com/benchmarks/vals-ai%3Afabv2-1%3Ageneral_qualitative_analysis.md) | 71% | SE ±5.82 pp | 9 / 18 | $0.57 | 1,143.48source duration | [Vals AI · 2.1](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2_1.ZbMhy9xo.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 70.995 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.573931 \| USD \| per_test \| Lower is better \| \| `latency` \| 1143.475 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±5.82 pp sourcePopulation 28 stderr 5.818 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2_1.ZbMhy9xo.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash-1017 Source version 2.1 Observed 2026-10-21T16:45:54.954Z Evidence SHA-256 `f1b0c2d7dca4e73b52136862a9be059f7c8481c69f15d705ce90bf3bd9f78429` Source locator `/tasks/general_qualitative_analysis/deepseek~1deepseek-v4.1-flash-1017` | | | | | | | | | [EMB / comps](https://benchize.com/benchmarks/vals-ai%3Aemb%3Acomps.md) | 49.77% | SE ±8.07 pp | 33 / 47 | $0.34 | 815.93source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 49.774 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.343239 \| USD \| per_test \| Lower is better \| \| `latency` \| 815.93 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±8.07 pp sourcePopulation 76 stderr 8.069 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash-1017 Source version 1 Observed 2026-10-21T16:45:57.495Z Evidence SHA-256 `d807aee620ea47152a1fca0257a8dd12a1be4512c76cc1744c1ae9e285180918` Source locator `/tasks/comps/deepseek~1deepseek-v4.1-flash-1017` | | | | | | | | | [HLAB-Vals-1.1 / criteria pass rate environmental esg](https://benchize.com/benchmarks/vals-ai%3Ahlab%3Acriteria_pass_rate__environmental_esg.md) | 93.01% | SE ±1.53 pp | 3 / 14 | $0.64 | 1,433.79source duration | [Vals AI · 1.1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 93.011 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.635298 \| USD \| per_test \| Lower is better \| \| `latency` \| 1433.79 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.53 pp sourcePopulation 24 stderr 1.529 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash-1017 Source version 1.1 Observed 2026-10-21T16:45:52.890Z Evidence SHA-256 `14681f4a2310e0f257df9ec07c9c81e5dadc8f6f71634f02e174a02a8096d868` Source locator `/tasks/criteria_pass_rate__environmental_esg/deepseek~1deepseek-v4.1-flash-1017` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / corporate ma](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acorporate_ma.md) | 20% | SE ±20 pp | 1 / 49 | $0.52 | 1,268.74source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 20 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.520056 \| USD \| per_test \| Lower is better \| \| `latency` \| 1268.744 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±20 pp sourcePopulation 81 stderr 20 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash-1017 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/corporate_ma/deepseek~1deepseek-v4.1-flash-1017` | | | | | | | | | [HLAB-Vals-1.1 / criteria pass rate immigration](https://benchize.com/benchmarks/vals-ai%3Ahlab%3Acriteria_pass_rate__immigration.md) | 92.55% | SE ±1.59 pp | 4 / 14 | $0.47 | 1,218.93source duration | [Vals AI · 1.1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 92.545 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.468313 \| USD \| per_test \| Lower is better \| \| `latency` \| 1218.928 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.59 pp sourcePopulation 24 stderr 1.587 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash-1017 Source version 1.1 Observed 2026-10-21T16:45:52.890Z Evidence SHA-256 `14681f4a2310e0f257df9ec07c9c81e5dadc8f6f71634f02e174a02a8096d868` Source locator `/tasks/criteria_pass_rate__immigration/deepseek~1deepseek-v4.1-flash-1017` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / employment labor](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Aemployment_labor.md) | 0% | SE ±0 pp | 1 / 49 | $0.49 | 1,222.02source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.486142 \| USD \| per_test \| Lower is better \| \| `latency` \| 1222.022 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 81 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash-1017 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/employment_labor/deepseek~1deepseek-v4.1-flash-1017` | | | | | | | | | [EMB v1.1 / operating](https://benchize.com/benchmarks/vals-ai%3Aemb-v1-1%3Aoperating.md) | 65.97% | SE ±4.69 pp | 17 / 19 | $0.29 | 842.33source duration | [Vals AI · 1.1](https://www.vals.ai/_astro/benchmark_view_emb_v1_1.11L06J7L.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 65.965 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.28596 \| USD \| per_test \| Lower is better \| \| `latency` \| 842.327 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±4.69 pp sourcePopulation 30 stderr 4.694 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb_v1_1.11L06J7L.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash-1017 Source version 1.1 Observed 2026-10-21T16:45:55.456Z Evidence SHA-256 `eea2ef2c83a6dc7d553c7c6531c287710ea7ffbc56ee1ccec06c85b104b2408e` Source locator `/tasks/operating/deepseek~1deepseek-v4.1-flash-1017` | | | | | | | | | [EMB v1.1 / comps](https://benchize.com/benchmarks/vals-ai%3Aemb-v1-1%3Acomps.md) | 70.77% | SE ±4.34 pp | 10 / 19 | $0.33 | 790.73source duration | [Vals AI · 1.1](https://www.vals.ai/_astro/benchmark_view_emb_v1_1.11L06J7L.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 70.771 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.331686 \| USD \| per_test \| Lower is better \| \| `latency` \| 790.726 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±4.34 pp sourcePopulation 30 stderr 4.335 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb_v1_1.11L06J7L.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash-1017 Source version 1.1 Observed 2026-10-21T16:45:55.456Z Evidence SHA-256 `eea2ef2c83a6dc7d553c7c6531c287710ea7ffbc56ee1ccec06c85b104b2408e` Source locator `/tasks/comps/deepseek~1deepseek-v4.1-flash-1017` | | | | | | | | | [HLAB-Vals-1.1 / corporate governance](https://benchize.com/benchmarks/vals-ai%3Ahlab%3Acorporate_governance.md) | 20% | SE ±20 pp | 3 / 14 | $0.7 | 1,497.44source duration | [Vals AI · 1.1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 20 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.70205 \| USD \| per_test \| Lower is better \| \| `latency` \| 1497.44 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±20 pp sourcePopulation 24 stderr 20 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash-1017 Source version 1.1 Observed 2026-10-21T16:45:52.890Z Evidence SHA-256 `14681f4a2310e0f257df9ec07c9c81e5dadc8f6f71634f02e174a02a8096d868` Source locator `/tasks/corporate_governance/deepseek~1deepseek-v4.1-flash-1017` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / intellectual property](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Aintellectual_property.md) | 20% | SE ±20 pp | 1 / 49 | $0.52 | 1,340.16source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 20 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.524267 \| USD \| per_test \| Lower is better \| \| `latency` \| 1340.156 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±20 pp sourcePopulation 81 stderr 20 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash-1017 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/intellectual_property/deepseek~1deepseek-v4.1-flash-1017` | | | | | | | | | [Tax Agent Bench / weighted pass controversy precedence analysis](https://benchize.com/benchmarks/vals-ai%3Atax_agent_bench%3Aweighted_pass__controversy_precedence_analysis.md) | 73.24% | SE ±7.54 pp | 1 / 46 | $0.47 | 1,323.03source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 73.236 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.472427 \| USD \| per_test \| Lower is better \| \| `latency` \| 1323.029 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±7.54 pp sourcePopulation 73 stderr 7.536 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash-1017 Source version 1 Observed 2026-10-21T16:45:55.958Z Evidence SHA-256 `446db182b67efb7923b3187a4040ec3a3ea492b8ad699a2b6d708d714cc063ac` Source locator `/tasks/weighted_pass__controversy_precedence_analysis/deepseek~1deepseek-v4.1-flash-1017` | | | | | | | | | [Arena webdev / overall](https://benchize.com/benchmarks/lmarena%3Awebdev%3Aoverall.md) | 1,664.33 | — | 12 / 90 | — | — | [Arena](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/webdev/latest) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `rating` \| 1664.3348298599165 \| rating \| reported \| Higher is better \| ### Configuration category overall config webdev Reasoning effort max thinkingEnabled true ### Reported statistics lower 1650.0115684885711 rank 13 upper 1678.6580912312618 variance 53.40570493862625 votes 2008 ### Source [Original source](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/webdev/latest) Source Arena Original model label DeepSeek-V4.1-Flash-20261017 (Max) Observed 2026-10-21T18:45:05.190Z Evidence SHA-256 `f82507859808110820a4ab6df15561b8864137ddefc1f6fb0c552af2e58ea738` Source locator `/rows/12/row` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / criteria pass rate funds asset management](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__funds_asset_management.md) | 88.05% | SE ±1.75 pp | 8 / 49 | $0.65 | 1,493.22source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 88.047 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.652496 \| USD \| per_test \| Lower is better \| \| `latency` \| 1493.224 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.75 pp sourcePopulation 81 stderr 1.754 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash-1017 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__funds_asset_management/deepseek~1deepseek-v4.1-flash-1017` | | | | | | | | | [EMB v1.1 / ma](https://benchize.com/benchmarks/vals-ai%3Aemb-v1-1%3Ama.md) | 73.08% | SE ±5.04 pp | 15 / 19 | $0.29 | 2,347.8source duration | [Vals AI · 1.1](https://www.vals.ai/_astro/benchmark_view_emb_v1_1.11L06J7L.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 73.081 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.286688 \| USD \| per_test \| Lower is better \| \| `latency` \| 2347.799 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±5.04 pp sourcePopulation 30 stderr 5.038 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb_v1_1.11L06J7L.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash-1017 Source version 1.1 Observed 2026-10-21T16:45:55.456Z Evidence SHA-256 `eea2ef2c83a6dc7d553c7c6531c287710ea7ffbc56ee1ccec06c85b104b2408e` Source locator `/tasks/ma/deepseek~1deepseek-v4.1-flash-1017` | | | | | | | | | [Tax Agent Bench / overall](https://benchize.com/benchmarks/vals-ai%3Atax_agent_bench%3Aoverall.md) | 44.35% | SE ±3.57 pp | 5 / 46 | $0.46 | 1,264.43source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 44.35 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.459454 \| USD \| per_test \| Lower is better \| \| `latency` \| 1264.434 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±3.57 pp sourcePopulation 73 stderr 3.574 tokenTotals {"cache_read_tokens":712066688,"cache_write_tokens":null,"input_tokens":779160448,"output_tokens":53561725,"reasoning_tokens":45716077} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash-1017 Source version 1 Observed 2026-10-21T16:45:55.958Z Evidence SHA-256 `446db182b67efb7923b3187a4040ec3a3ea492b8ad699a2b6d708d714cc063ac` Source locator `/tasks/overall/deepseek~1deepseek-v4.1-flash-1017` | | | | | | | | | [Arena webdev / webdev](https://benchize.com/benchmarks/lmarena%3Awebdev%3Awebdev.md) | 1,664.33 | — | 12 / 90 | — | — | [Arena](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/webdev/latest) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `rating` \| 1664.3348298599165 \| rating \| reported \| Higher is better \| ### Configuration category webdev config webdev Reasoning effort max thinkingEnabled true ### Reported statistics lower 1650.0115684885711 rank 13 upper 1678.6580912312618 variance 53.40570493862625 votes 2008 ### Source [Original source](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/webdev/latest) Source Arena Original model label DeepSeek-V4.1-Flash-20261017 (Max) Observed 2026-10-21T18:45:05.190Z Evidence SHA-256 `f82507859808110820a4ab6df15561b8864137ddefc1f6fb0c552af2e58ea738` Source locator `/rows/224/row` | | | | | | | | | [Legal Research Bench / weighted pass family](https://benchize.com/benchmarks/vals-ai%3Alegal_research%3Aweighted_pass__family.md) | 74.19% | SE ±5.8 pp | 22 / 50 | $0.83 | 2,445.95source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 74.191 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.826784 \| USD \| per_test \| Lower is better \| \| `latency` \| 2445.946 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±5.8 pp sourcePopulation 82 stderr 5.801 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash-1017 Source version 1 Observed 2026-10-21T16:45:52.389Z Evidence SHA-256 `26c58dfd47d0d2dc86f33c6d056f30309e0ee63570e7cd752c593b385f9d862e` Source locator `/tasks/weighted_pass__family/deepseek~1deepseek-v4.1-flash-1017` | | | | | | | | | [EMB v1.1 / overall](https://benchize.com/benchmarks/vals-ai%3Aemb-v1-1%3Aoverall.md) | 69.17% | SE ±2.4 pp | 11 / 19 | $0.33 | 1,971.69source duration | [Vals AI · 1.1](https://www.vals.ai/_astro/benchmark_view_emb_v1_1.11L06J7L.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 69.173 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.330922 \| USD \| per_test \| Lower is better \| \| `latency` \| 1971.687 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.4 pp sourcePopulation 30 stderr 2.395 tokenTotals {"cache_read_tokens":1797102837,"cache_write_tokens":null,"input_tokens":1809205546,"output_tokens":16117195,"reasoning_tokens":11510477} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb_v1_1.11L06J7L.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash-1017 Source version 1.1 Observed 2026-10-21T16:45:55.456Z Evidence SHA-256 `eea2ef2c83a6dc7d553c7c6531c287710ea7ffbc56ee1ccec06c85b104b2408e` Source locator `/tasks/overall/deepseek~1deepseek-v4.1-flash-1017` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / criteria pass rate trusts estates private client](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__trusts_estates_private_client.md) | 87.79% | SE ±2.03 pp | 17 / 49 | $0.47 | 1,186.37source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 87.786 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.469121 \| USD \| per_test \| Lower is better \| \| `latency` \| 1186.37 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.03 pp sourcePopulation 81 stderr 2.027 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash-1017 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__trusts_estates_private_client/deepseek~1deepseek-v4.1-flash-1017` | | | | | | | | | [HLAB-Vals-1.1 / criteria pass rate emerging companies venture capital](https://benchize.com/benchmarks/vals-ai%3Ahlab%3Acriteria_pass_rate__emerging_companies_venture_capital.md) | 94.88% | SE ±1.29 pp | 4 / 14 | $0.63 | 1,539.01source duration | [Vals AI · 1.1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 94.881 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.633521 \| USD \| per_test \| Lower is better \| \| `latency` \| 1539.008 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.29 pp sourcePopulation 24 stderr 1.29 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash-1017 Source version 1.1 Observed 2026-10-21T16:45:52.890Z Evidence SHA-256 `14681f4a2310e0f257df9ec07c9c81e5dadc8f6f71634f02e174a02a8096d868` Source locator `/tasks/criteria_pass_rate__emerging_companies_venture_capital/deepseek~1deepseek-v4.1-flash-1017` | | | | | | | | | [Finance Agent (v2) / disclosure analysis](https://benchize.com/benchmarks/vals-ai%3Afabv2%3Adisclosure_analysis.md) | 66.26% | SE ±6.53 pp | 16 / 50 | $0.64 | 1,257.12source duration | [Vals AI · 2](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 66.264 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.636847 \| USD \| per_test \| Lower is better \| \| `latency` \| 1257.118 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±6.53 pp sourcePopulation 80 stderr 6.532 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash-1017 Source version 2 Observed 2026-10-21T16:45:57.035Z Evidence SHA-256 `f731b20a26c126725d125519f2b5841aa45254656dba859a9570d7ce7e5d8191` Source locator `/tasks/disclosure_analysis/deepseek~1deepseek-v4.1-flash-1017` | | | | | | | | | [HLAB-Vals-1.1 / criteria pass rate](https://benchize.com/benchmarks/vals-ai%3Ahlab%3Acriteria_pass_rate.md) | 92.67% | SE ±0.31 pp | 5 / 14 | $0.61 | 1,328.11source duration | [Vals AI · 1.1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 92.665 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.612596 \| USD \| per_test \| Lower is better \| \| `latency` \| 1328.108 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.31 pp sourcePopulation 24 stderr 0.314 tokenTotals {"cache_read_tokens":5492827392,"cache_write_tokens":null,"input_tokens":5524481146,"output_tokens":25881985,"reasoning_tokens":10520362} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash-1017 Source version 1.1 Observed 2026-10-21T16:45:52.890Z Evidence SHA-256 `14681f4a2310e0f257df9ec07c9c81e5dadc8f6f71634f02e174a02a8096d868` Source locator `/tasks/criteria_pass_rate/deepseek~1deepseek-v4.1-flash-1017` | | | | | | | | | [Legal Research Bench / weighted pass civil litigation](https://benchize.com/benchmarks/vals-ai%3Alegal_research%3Aweighted_pass__civil_litigation.md) | 84.85% | SE ±4.59 pp | 31 / 50 | $0.66 | 2,088.43source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 84.852 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.658074 \| USD \| per_test \| Lower is better \| \| `latency` \| 2088.431 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±4.59 pp sourcePopulation 82 stderr 4.591 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash-1017 Source version 1 Observed 2026-10-21T16:45:52.389Z Evidence SHA-256 `26c58dfd47d0d2dc86f33c6d056f30309e0ee63570e7cd752c593b385f9d862e` Source locator `/tasks/weighted_pass__civil_litigation/deepseek~1deepseek-v4.1-flash-1017` | | | | | | | | | [Finance Agent (v2) / earnings analysis](https://benchize.com/benchmarks/vals-ai%3Afabv2%3Aearnings_analysis.md) | 75.26% | SE ±6.07 pp | 17 / 50 | $0.21 | 507.27source duration | [Vals AI · 2](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 75.263 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.206276 \| USD \| per_test \| Lower is better \| \| `latency` \| 507.274 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±6.07 pp sourcePopulation 80 stderr 6.068 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash-1017 Source version 2 Observed 2026-10-21T16:45:57.035Z Evidence SHA-256 `f731b20a26c126725d125519f2b5841aa45254656dba859a9570d7ce7e5d8191` Source locator `/tasks/earnings_analysis/deepseek~1deepseek-v4.1-flash-1017` | | | | | | | | | [TapTap Maker public](https://benchize.com/benchmarks/taptap-maker%3Apublic.md) | 80.34% | Range 80.34–80.34% | 8 / 26 | $0.08 | — | [taptap maker](https://maker.taptap.cn/leaderboard/data/editions/2026-10-18-v6-voxel.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `l2_pass_rate` \| 0.8034 \| score_fraction \| reported \| Higher is better \| \| `cost_per_case` \| 0.0772 \| USD \| per_case \| Lower is better \| ### Configuration datasetVersion 0.2.3 edition 2026-10-18-v6-voxel harness claude Reasoning effort max reasoningMode Not reported ### Reported statistics Uncertainty Reported interval: 80.34–80.34% interval [0.8034,0.8034] intervalMethod bootstrap-case rank 8 rankInterval [8,10] ### Source [Original source](https://maker.taptap.cn/leaderboard/data/editions/2026-10-18-v6-voxel.json) Source taptap maker Original model label deepseek-v4.1-flash-1017 Observed 2026-10-21T16:33:19.620Z Evidence SHA-256 `ad2d7f1adeae0548a4edc0b5e75d11449b443449ff76533d04bc1fb1b6d7035e` Source locator `/public/7` | | | | | | | | | [EMB v1.1 / misc](https://benchize.com/benchmarks/vals-ai%3Aemb-v1-1%3Amisc.md) | 65.72% | SE ±3.32 pp | 9 / 19 | $0.33 | 977.25source duration | [Vals AI · 1.1](https://www.vals.ai/_astro/benchmark_view_emb_v1_1.11L06J7L.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 65.723 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.334964 \| USD \| per_test \| Lower is better \| \| `latency` \| 977.252 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±3.32 pp sourcePopulation 30 stderr 3.32 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb_v1_1.11L06J7L.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash-1017 Source version 1.1 Observed 2026-10-21T16:45:55.456Z Evidence SHA-256 `eea2ef2c83a6dc7d553c7c6531c287710ea7ffbc56ee1ccec06c85b104b2408e` Source locator `/tasks/misc/deepseek~1deepseek-v4.1-flash-1017` | | | | | | | | | [Legal Research Bench / weighted pass](https://benchize.com/benchmarks/vals-ai%3Alegal_research%3Aweighted_pass.md) | 86.28% | SE ±1.38 pp | 8 / 50 | $0.7 | 2,106.88source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 86.28 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.69856 \| USD \| per_test \| Lower is better \| \| `latency` \| 2106.881 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.38 pp sourcePopulation 82 stderr 1.376 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash-1017 Source version 1 Observed 2026-10-21T16:45:52.389Z Evidence SHA-256 `26c58dfd47d0d2dc86f33c6d056f30309e0ee63570e7cd752c593b385f9d862e` Source locator `/tasks/weighted_pass/deepseek~1deepseek-v4.1-flash-1017` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / environmental esg](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Aenvironmental_esg.md) | 0% | SE ±0 pp | 1 / 49 | $0.62 | 1,443.94source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.617056 \| USD \| per_test \| Lower is better \| \| `latency` \| 1443.936 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 81 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash-1017 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/environmental_esg/deepseek~1deepseek-v4.1-flash-1017` | | | | | | | | | [Tax Agent Bench / rule source lookup](https://benchize.com/benchmarks/vals-ai%3Atax_agent_bench%3Arule_source_lookup.md) | 46.67% | SE ±9.26 pp | 5 / 46 | $0.49 | 1,361.74source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 46.667 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.492773 \| USD \| per_test \| Lower is better \| \| `latency` \| 1361.742 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±9.26 pp sourcePopulation 73 stderr 9.264 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash-1017 Source version 1 Observed 2026-10-21T16:45:55.958Z Evidence SHA-256 `446db182b67efb7923b3187a4040ec3a3ea492b8ad699a2b6d708d714cc063ac` Source locator `/tasks/rule_source_lookup/deepseek~1deepseek-v4.1-flash-1017` | | | | | | | | | [Finance Agent (v2) / financial modeling](https://benchize.com/benchmarks/vals-ai%3Afabv2%3Afinancial_modeling.md) | 22.75% | SE ±5.57 pp | 23 / 50 | $0.39 | 792.43source duration | [Vals AI · 2](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 22.751 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.385189 \| USD \| per_test \| Lower is better \| \| `latency` \| 792.429 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±5.57 pp sourcePopulation 80 stderr 5.574 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash-1017 Source version 2 Observed 2026-10-21T16:45:57.035Z Evidence SHA-256 `f731b20a26c126725d125519f2b5841aa45254656dba859a9570d7ce7e5d8191` Source locator `/tasks/financial_modeling/deepseek~1deepseek-v4.1-flash-1017` | | | | | | | | | [Tax Agent Bench / controversy precedence analysis](https://benchize.com/benchmarks/vals-ai%3Atax_agent_bench%3Acontroversy_precedence_analysis.md) | 23.33% | SE ±7.85 pp | 3 / 46 | $0.47 | 1,323.03source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 23.333 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.472427 \| USD \| per_test \| Lower is better \| \| `latency` \| 1323.029 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±7.85 pp sourcePopulation 73 stderr 7.854 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash-1017 Source version 1 Observed 2026-10-21T16:45:55.958Z Evidence SHA-256 `446db182b67efb7923b3187a4040ec3a3ea492b8ad699a2b6d708d714cc063ac` Source locator `/tasks/controversy_precedence_analysis/deepseek~1deepseek-v4.1-flash-1017` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / funds asset management](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Afunds_asset_management.md) | 20% | SE ±20 pp | 3 / 49 | $0.65 | 1,493.22source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 20 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.652496 \| USD \| per_test \| Lower is better \| \| `latency` \| 1493.224 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±20 pp sourcePopulation 81 stderr 20 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash-1017 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/funds_asset_management/deepseek~1deepseek-v4.1-flash-1017` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / criteria pass rate tax](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__tax.md) | 84.95% | SE ±2.07 pp | 9 / 49 | $0.64 | 1,401.31source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 84.95 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.636777 \| USD \| per_test \| Lower is better \| \| `latency` \| 1401.314 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.07 pp sourcePopulation 81 stderr 2.071 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash-1017 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__tax/deepseek~1deepseek-v4.1-flash-1017` | | | | | | | | | [EMB / dcf](https://benchize.com/benchmarks/vals-ai%3Aemb%3Adcf.md) | 53% | SE ±9.43 pp | 27 / 47 | $0.35 | 1,032.07source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 53 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.350055 \| USD \| per_test \| Lower is better \| \| `latency` \| 1032.065 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±9.43 pp sourcePopulation 76 stderr 9.434 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash-1017 Source version 1 Observed 2026-10-21T16:45:57.495Z Evidence SHA-256 `d807aee620ea47152a1fca0257a8dd12a1be4512c76cc1744c1ae9e285180918` Source locator `/tasks/dcf/deepseek~1deepseek-v4.1-flash-1017` | | | | | | | | | [Legal Research Bench / immigration](https://benchize.com/benchmarks/vals-ai%3Alegal_research%3Aimmigration.md) | 36.36% | SE ±15.21 pp | 9 / 50 | $0.9 | 2,363.89source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 36.364 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.899309 \| USD \| per_test \| Lower is better \| \| `latency` \| 2363.885 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±15.21 pp sourcePopulation 82 stderr 15.212 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash-1017 Source version 1 Observed 2026-10-21T16:45:52.389Z Evidence SHA-256 `26c58dfd47d0d2dc86f33c6d056f30309e0ee63570e7cd752c593b385f9d862e` Source locator `/tasks/immigration/deepseek~1deepseek-v4.1-flash-1017` | | | | | | | | | [HLAB-Vals-1.1 / international trade sanctions](https://benchize.com/benchmarks/vals-ai%3Ahlab%3Ainternational_trade_sanctions.md) | 0% | SE ±0 pp | 5 / 14 | $0.61 | 1,463.82source duration | [Vals AI · 1.1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.612386 \| USD \| per_test \| Lower is better \| \| `latency` \| 1463.816 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 24 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash-1017 Source version 1.1 Observed 2026-10-21T16:45:52.890Z Evidence SHA-256 `14681f4a2310e0f257df9ec07c9c81e5dadc8f6f71634f02e174a02a8096d868` Source locator `/tasks/international_trade_sanctions/deepseek~1deepseek-v4.1-flash-1017` | | | | | | | | | [Legal Research Bench / civil litigation](https://benchize.com/benchmarks/vals-ai%3Alegal_research%3Acivil_litigation.md) | 37.04% | SE ±9.47 pp | 21 / 50 | $0.66 | 2,088.43source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 37.037 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.658074 \| USD \| per_test \| Lower is better \| \| `latency` \| 2088.431 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±9.47 pp sourcePopulation 82 stderr 9.471 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash-1017 Source version 1 Observed 2026-10-21T16:45:52.389Z Evidence SHA-256 `26c58dfd47d0d2dc86f33c6d056f30309e0ee63570e7cd752c593b385f9d862e` Source locator `/tasks/civil_litigation/deepseek~1deepseek-v4.1-flash-1017` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / corporate governance](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acorporate_governance.md) | 0% | SE ±0 pp | 11 / 49 | $0.49 | 1,222.26source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.48714 \| USD \| per_test \| Lower is better \| \| `latency` \| 1222.26 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 81 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash-1017 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/corporate_governance/deepseek~1deepseek-v4.1-flash-1017` | | | | | | | | | [ToneBench Scriptwriting](https://benchize.com/benchmarks/tonebench%3Ascripts.md) | 2,085 | Range 2,038.5–2,130.7 | 24 / 126 | — | — | [tonebench](https://benchmark.towardsai.com/data/leaderboard.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `elo` \| 2085 \| rating \| reported \| Higher is better \| \| `overall` \| 81.61 \| percent \| reported \| Higher is better \| \| `anti_slop_numbers` \| 73.09 \| source_score \| rubric \| Higher is better \| \| `continuity_emotion` \| 81.03 \| source_score \| rubric \| Higher is better \| \| `cue_quality` \| 75.31 \| source_score \| rubric \| Higher is better \| \| `hook_strength` \| 81.13 \| source_score \| rubric \| Higher is better \| \| `length_adherence` \| 94.8 \| source_score \| rubric \| Higher is better \| \| `slop` \| 84.1 \| source_score \| rubric \| Higher is better \| \| `slop_score` \| 95.11 \| source_score \| rubric \| Higher is better \| \| `substance_accuracy` \| 82.28 \| source_score \| rubric \| Higher is better \| \| `tone_voice` \| 79.24 \| source_score \| rubric \| Higher is better \| \| `writing_quality` \| 79.51 \| source_score \| rubric \| Higher is better \| \| `youtube_best_practices` \| 80.13 \| source_score \| rubric \| Higher is better \| ### Configuration provider openrouter Reasoning effort max scoreScope 10_task_current sourceConfiguration deepseek-v4-1-flash-1017-max-or thinkingEnabled true ### Reported statistics Uncertainty Reported interval: 2,038.5–2,130.7 interval [2038.5,2130.7] rank 24 runCount 5 ### Source [Original source](https://benchmark.towardsai.com/data/leaderboard.json) Source tonebench Original model label DeepSeek V4.1 Flash 1017 (max) Evaluated 2026-10-17 Observed 2026-10-21T16:32:48.862Z Evidence SHA-256 `4264338337f418d3ded7ace250cad64fe12cacbd4b79a312bfaf0ae9268a01c3` Source locator `/rows/23` | | | | | | | | | [Tax Agent Bench / weighted pass fact pattern analysis](https://benchize.com/benchmarks/vals-ai%3Atax_agent_bench%3Aweighted_pass__fact_pattern_analysis.md) | 72.52% | SE ±6.62 pp | 8 / 46 | $0.46 | 1,324.47source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 72.524 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.460654 \| USD \| per_test \| Lower is better \| \| `latency` \| 1324.466 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±6.62 pp sourcePopulation 73 stderr 6.621 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash-1017 Source version 1 Observed 2026-10-21T16:45:55.958Z Evidence SHA-256 `446db182b67efb7923b3187a4040ec3a3ea492b8ad699a2b6d708d714cc063ac` Source locator `/tasks/weighted_pass__fact_pattern_analysis/deepseek~1deepseek-v4.1-flash-1017` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / criteria pass rate healthcare life sciences](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__healthcare_life_sciences.md) | 89.44% | SE ±1.77 pp | 18 / 49 | $0.51 | 1,310.75source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 89.439 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.506684 \| USD \| per_test \| Lower is better \| \| `latency` \| 1310.752 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.77 pp sourcePopulation 81 stderr 1.769 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash-1017 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__healthcare_life_sciences/deepseek~1deepseek-v4.1-flash-1017` | | | | | | | | | [HLAB-Vals-1.1 / arbitration international dispute resolution](https://benchize.com/benchmarks/vals-ai%3Ahlab%3Aarbitration_international_dispute_resolution.md) | 10% | SE ±10 pp | 4 / 14 | $0.67 | 1,273.8source duration | [Vals AI · 1.1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 10 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.66721 \| USD \| per_test \| Lower is better \| \| `latency` \| 1273.796 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±10 pp sourcePopulation 24 stderr 10 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash-1017 Source version 1.1 Observed 2026-10-21T16:45:52.890Z Evidence SHA-256 `14681f4a2310e0f257df9ec07c9c81e5dadc8f6f71634f02e174a02a8096d868` Source locator `/tasks/arbitration_international_dispute_resolution/deepseek~1deepseek-v4.1-flash-1017` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / litigation dispute resolution](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Alitigation_dispute_resolution.md) | 20% | SE ±20 pp | 2 / 49 | $0.55 | 1,236.26source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 20 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.551954 \| USD \| per_test \| Lower is better \| \| `latency` \| 1236.258 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±20 pp sourcePopulation 81 stderr 20 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash-1017 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/litigation_dispute_resolution/deepseek~1deepseek-v4.1-flash-1017` | | | | | | | | | [Tax Agent Bench / fact pattern analysis](https://benchize.com/benchmarks/vals-ai%3Atax_agent_bench%3Afact_pattern_analysis.md) | 55.77% | SE ±7.66 pp | 1 / 46 | $0.46 | 1,324.47source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 55.767 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.460654 \| USD \| per_test \| Lower is better \| \| `latency` \| 1324.466 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±7.66 pp sourcePopulation 73 stderr 7.657 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash-1017 Source version 1 Observed 2026-10-21T16:45:55.958Z Evidence SHA-256 `446db182b67efb7923b3187a4040ec3a3ea492b8ad699a2b6d708d714cc063ac` Source locator `/tasks/fact_pattern_analysis/deepseek~1deepseek-v4.1-flash-1017` | | | | | | | | | [Code Migration v1.1 / code quality](https://benchize.com/benchmarks/vals-ai%3Acode-migration-v1-1%3Acode_quality.md) | 3.48% | SE ±0.03 pp | 8 / 17 | $1.76 | 8,475.47source duration | [Vals AI · 1.1](https://www.vals.ai/_astro/benchmark_view_code_migration_v1_1.bJHk6HEY.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 3.476 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 1.764297 \| USD \| per_test \| Lower is better \| \| `latency` \| 8475.469 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.03 pp sourcePopulation 26 stderr 0.033 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_code_migration_v1_1.bJHk6HEY.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash-1017 Source version 1.1 Observed 2026-10-21T16:46:09.631Z Evidence SHA-256 `352faac603c73f05cc01e89214fb4adcafa05ea7432ea71ea9903a6b5fe36b08` Source locator `/tasks/code_quality/deepseek~1deepseek-v4.1-flash-1017` | | | | | | | | | [Code Migration v1.1 / cobol](https://benchize.com/benchmarks/vals-ai%3Acode-migration-v1-1%3Acobol.md) | 92.5% | SE ±6.96 pp | 4 / 17 | $0.48 | 3,812.69source duration | [Vals AI · 1.1](https://www.vals.ai/_astro/benchmark_view_code_migration_v1_1.bJHk6HEY.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 92.5 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.477214 \| USD \| per_test \| Lower is better \| \| `latency` \| 3812.688 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±6.96 pp sourcePopulation 26 stderr 6.962 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_code_migration_v1_1.bJHk6HEY.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash-1017 Source version 1.1 Observed 2026-10-21T16:46:09.631Z Evidence SHA-256 `352faac603c73f05cc01e89214fb4adcafa05ea7432ea71ea9903a6b5fe36b08` Source locator `/tasks/cobol/deepseek~1deepseek-v4.1-flash-1017` | | | | | | | | | [Legal Research Bench / weighted pass health](https://benchize.com/benchmarks/vals-ai%3Alegal_research%3Aweighted_pass__health.md) | 96.18% | SE ±1.4 pp | 5 / 50 | $0.71 | 1,923.98source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 96.178 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.708547 \| USD \| per_test \| Lower is better \| \| `latency` \| 1923.984 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.4 pp sourcePopulation 82 stderr 1.399 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash-1017 Source version 1 Observed 2026-10-21T16:45:52.389Z Evidence SHA-256 `26c58dfd47d0d2dc86f33c6d056f30309e0ee63570e7cd752c593b385f9d862e` Source locator `/tasks/weighted_pass__health/deepseek~1deepseek-v4.1-flash-1017` | | | | | | | | | [Legal Research Bench / weighted pass administrative regulatory](https://benchize.com/benchmarks/vals-ai%3Alegal_research%3Aweighted_pass__administrative_regulatory.md) | 91.28% | SE ±1.98 pp | 5 / 50 | $0.65 | 1,931.91source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 91.277 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.652538 \| USD \| per_test \| Lower is better \| \| `latency` \| 1931.905 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.98 pp sourcePopulation 82 stderr 1.979 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash-1017 Source version 1 Observed 2026-10-21T16:45:52.389Z Evidence SHA-256 `26c58dfd47d0d2dc86f33c6d056f30309e0ee63570e7cd752c593b385f9d862e` Source locator `/tasks/weighted_pass__administrative_regulatory/deepseek~1deepseek-v4.1-flash-1017` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / criteria pass rate structured finance securitization](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__structured_finance_securitization.md) | 86.67% | SE ±1.92 pp | 9 / 49 | $0.48 | 1,156.54source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 86.667 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.478566 \| USD \| per_test \| Lower is better \| \| `latency` \| 1156.54 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.92 pp sourcePopulation 81 stderr 1.918 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash-1017 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__structured_finance_securitization/deepseek~1deepseek-v4.1-flash-1017` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / trusts estates private client](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Atrusts_estates_private_client.md) | 20% | SE ±20 pp | 2 / 49 | $0.47 | 1,186.37source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 20 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.469121 \| USD \| per_test \| Lower is better \| \| `latency` \| 1186.37 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±20 pp sourcePopulation 81 stderr 20 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash-1017 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/trusts_estates_private_client/deepseek~1deepseek-v4.1-flash-1017` | | | | | | | | | [Code Migration / overall](https://benchize.com/benchmarks/vals-ai%3Acode-migration%3Aoverall.md) | 44.21% | SE ±4.46 pp | 19 / 47 | $1.86 | 7,843.51source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_code_migration.Dttzoqk3.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 44.21 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 1.858025 \| USD \| per_test \| Lower is better \| \| `latency` \| 7843.507 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±4.46 pp sourcePopulation 79 stderr 4.455 tokenTotals {"cache_read_tokens":31797847932,"cache_write_tokens":null,"input_tokens":31908355564,"output_tokens":52632882,"reasoning_tokens":25529336} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_code_migration.Dttzoqk3.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash-1017 Source version 1 Observed 2026-10-21T16:46:12.712Z Evidence SHA-256 `99194597f2c76c402195be47839409046e18bd30454616da6dd40b736a088fa4` Source locator `/tasks/overall/deepseek~1deepseek-v4.1-flash-1017` | | | | | | | | | [Finance Agent (v2) / overall](https://benchize.com/benchmarks/vals-ai%3Afabv2%3Aoverall.md) | 56.96% | SE ±2.05 pp | 14 / 50 | $0.46 | 879.26source duration | [Vals AI · 2](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 56.96 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.462841 \| USD \| per_test \| Lower is better \| \| `latency` \| 879.256 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.05 pp sourcePopulation 80 stderr 2.049 tokenTotals {"cache_read_tokens":898992768,"cache_write_tokens":null,"input_tokens":1221527142,"output_tokens":88436785,"reasoning_tokens":76225695} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash-1017 Source version 2 Observed 2026-10-21T16:45:57.035Z Evidence SHA-256 `f731b20a26c126725d125519f2b5841aa45254656dba859a9570d7ce7e5d8191` Source locator `/tasks/overall/deepseek~1deepseek-v4.1-flash-1017` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / criteria pass rate corporate ma](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__corporate_ma.md) | 92.95% | SE ±1.45 pp | 12 / 49 | $0.52 | 1,268.74source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 92.949 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.520056 \| USD \| per_test \| Lower is better \| \| `latency` \| 1268.744 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.45 pp sourcePopulation 81 stderr 1.452 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash-1017 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__corporate_ma/deepseek~1deepseek-v4.1-flash-1017` | | | | | | | | | [HLAB-Vals-1.1 / real estate](https://benchize.com/benchmarks/vals-ai%3Ahlab%3Areal_estate.md) | 10% | SE ±10 pp | 7 / 14 | $0.67 | 1,407.42source duration | [Vals AI · 1.1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 10 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.669758 \| USD \| per_test \| Lower is better \| \| `latency` \| 1407.424 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±10 pp sourcePopulation 24 stderr 10 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash-1017 Source version 1.1 Observed 2026-10-21T16:45:52.890Z Evidence SHA-256 `14681f4a2310e0f257df9ec07c9c81e5dadc8f6f71634f02e174a02a8096d868` Source locator `/tasks/real_estate/deepseek~1deepseek-v4.1-flash-1017` | | | | | | | | | [HLAB-Vals-1.1 / environmental esg](https://benchize.com/benchmarks/vals-ai%3Ahlab%3Aenvironmental_esg.md) | 0% | SE ±0 pp | 3 / 14 | $0.64 | 1,433.79source duration | [Vals AI · 1.1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.635298 \| USD \| per_test \| Lower is better \| \| `latency` \| 1433.79 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 24 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash-1017 Source version 1.1 Observed 2026-10-21T16:45:52.890Z Evidence SHA-256 `14681f4a2310e0f257df9ec07c9c81e5dadc8f6f71634f02e174a02a8096d868` Source locator `/tasks/environmental_esg/deepseek~1deepseek-v4.1-flash-1017` | | | | | | | | | [Vibe Code Bench v1.1 / overall](https://benchize.com/benchmarks/vals-ai%3Avibe-code%3Aoverall.md) | 87.73% | SE ±2.75 pp | 11 / 59 | $0.58 | 973.83source duration | [Vals AI · 1.1](https://www.vals.ai/_astro/benchmark_view_vibe_code_bench.DcEBMbCA.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 87.73 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.579984 \| USD \| per_test \| Lower is better \| \| `latency` \| 973.834 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness OpenHands maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.75 pp sourcePopulation 116 stderr 2.752 tokenTotals {"cache_read_tokens":2156782720,"cache_write_tokens":null,"input_tokens":2173326299,"output_tokens":9246207,"reasoning_tokens":3107112} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_vibe_code_bench.DcEBMbCA.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash-1017 Source version 1.1 Observed 2026-10-21T16:46:08.625Z Evidence SHA-256 `0e8c23f521a0f751c5592952620e197b376cade41c78578461b7aa5740a173e7` Source locator `/tasks/overall/deepseek~1deepseek-v4.1-flash-1017` | | | | | | | | | [HLAB-Vals-1.1 / white collar defense investigations](https://benchize.com/benchmarks/vals-ai%3Ahlab%3Awhite_collar_defense_investigations.md) | 0% | SE ±0 pp | 2 / 14 | $0.69 | 1,431.65source duration | [Vals AI · 1.1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.688859 \| USD \| per_test \| Lower is better \| \| `latency` \| 1431.654 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 24 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash-1017 Source version 1.1 Observed 2026-10-21T16:45:52.890Z Evidence SHA-256 `14681f4a2310e0f257df9ec07c9c81e5dadc8f6f71634f02e174a02a8096d868` Source locator `/tasks/white_collar_defense_investigations/deepseek~1deepseek-v4.1-flash-1017` | | | | | | | | | [EMB v1.1 / lbo](https://benchize.com/benchmarks/vals-ai%3Aemb-v1-1%3Albo.md) | 62.46% | SE ±4.42 pp | 7 / 19 | $0.42 | 4,170.03source duration | [Vals AI · 1.1](https://www.vals.ai/_astro/benchmark_view_emb_v1_1.11L06J7L.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 62.463 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.419172 \| USD \| per_test \| Lower is better \| \| `latency` \| 4170.026 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±4.42 pp sourcePopulation 30 stderr 4.415 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb_v1_1.11L06J7L.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash-1017 Source version 1.1 Observed 2026-10-21T16:45:55.456Z Evidence SHA-256 `eea2ef2c83a6dc7d553c7c6531c287710ea7ffbc56ee1ccec06c85b104b2408e` Source locator `/tasks/lbo/deepseek~1deepseek-v4.1-flash-1017` | | | | | | | | | [Arena webdev / webdev-html](https://benchize.com/benchmarks/lmarena%3Awebdev%3Awebdev-html.md) | 1,686.92 | — | 7 / 89 | — | — | [Arena](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/webdev/latest) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `rating` \| 1686.9166747474715 \| rating \| reported \| Higher is better \| ### Configuration category webdev-html config webdev Reasoning effort max thinkingEnabled true ### Reported statistics lower 1648.4395993877395 rank 8 upper 1725.3937501072035 variance 385.39664157352206 votes 295 ### Source [Original source](https://datasets-server.huggingface.co/parquet/lmarena-ai%2Fleaderboard-dataset/webdev/latest) Source Arena Original model label DeepSeek-V4.1-Flash-20261017 (Max) Observed 2026-10-21T18:45:05.190Z Evidence SHA-256 `f82507859808110820a4ab6df15561b8864137ddefc1f6fb0c552af2e58ea738` Source locator `/rows/367/row` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / criteria pass rate immigration](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__immigration.md) | 89.82% | SE ±1.83 pp | 9 / 49 | $0.46 | 1,267.69source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 89.818 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.463623 \| USD \| per_test \| Lower is better \| \| `latency` \| 1267.69 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.83 pp sourcePopulation 81 stderr 1.827 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash-1017 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__immigration/deepseek~1deepseek-v4.1-flash-1017` | | | | | | | | | [HLAB-Vals-1.1 / criteria pass rate employment labor](https://benchize.com/benchmarks/vals-ai%3Ahlab%3Acriteria_pass_rate__employment_labor.md) | 90.64% | SE ±1.79 pp | 3 / 14 | $0.52 | 1,291.33source duration | [Vals AI · 1.1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 90.637 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.520216 \| USD \| per_test \| Lower is better \| \| `latency` \| 1291.326 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.79 pp sourcePopulation 24 stderr 1.786 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash-1017 Source version 1.1 Observed 2026-10-21T16:45:52.890Z Evidence SHA-256 `14681f4a2310e0f257df9ec07c9c81e5dadc8f6f71634f02e174a02a8096d868` Source locator `/tasks/criteria_pass_rate__employment_labor/deepseek~1deepseek-v4.1-flash-1017` | | | | | | | | | [HLAB-Vals-1.1 / litigation dispute resolution](https://benchize.com/benchmarks/vals-ai%3Ahlab%3Alitigation_dispute_resolution.md) | 20% | SE ±12.25 pp | 1 / 14 | $0.58 | 1,129.72source duration | [Vals AI · 1.1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 20 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.577367 \| USD \| per_test \| Lower is better \| \| `latency` \| 1129.724 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±12.25 pp sourcePopulation 24 stderr 12.247 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash-1017 Source version 1.1 Observed 2026-10-21T16:45:52.890Z Evidence SHA-256 `14681f4a2310e0f257df9ec07c9c81e5dadc8f6f71634f02e174a02a8096d868` Source locator `/tasks/litigation_dispute_resolution/deepseek~1deepseek-v4.1-flash-1017` | | | | | | | | [Next page](https://benchize.com/models/deepseek-v4-1-flash-1017-max.md?cursor=6ab722920c9fa9cdcfa8b9c58aa8967630afc62c9d66a8019d6c5a11a21e4d67) --- Cite as: DeepSeek V4.1 Flash 1017Max — Benchize, https://benchize.com/models/deepseek-v4-1-flash-1017-max (retrieved 2026-10-21)