# DeepSeek V4 Flash 0731High > Source: Benchize (https://benchize.com) — https://benchize.com/models/deepseek-v4-flash-0731-high. When citing or reusing this content, credit "Benchize" and link to https://benchize.com/models/deepseek-v4-flash-0731-high. DeepSeek Canonical URL: https://benchize.com/models/deepseek-v4-flash-0731-high Last updated: Wed, 21 Oct 2026 18:52:20 GMT Reproduction and quotation must name Benchize, link to the canonical HTML URL, and retain individual source citations. DeepSeek ## Benchmarks | Benchmark | Score | ± | Rank | Cost | Latency | Source | | | --- | --- | --- | --- | --- | --- | --- | --- | | [Harvey's Legal Agent Benchmark v1 / criteria pass rate tax](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__tax.md) | 82.94% | SE ±2.18 pp | 17 / 49 | $0.07 | 961.29source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 82.943 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.065423 \| USD \| per_test \| Lower is better \| \| `latency` \| 961.292 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.18 pp sourcePopulation 81 stderr 2.179 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label deepseek/deepseek-v4-flash-0731 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__tax/deepseek~1deepseek-v4-flash-0731` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / criteria pass rate healthcare life sciences](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__healthcare_life_sciences.md) | 91.09% | SE ±1.64 pp | 12 / 49 | $0.06 | 1,043.86source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 91.089 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.056365 \| USD \| per_test \| Lower is better \| \| `latency` \| 1043.858 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.64 pp sourcePopulation 81 stderr 1.639 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label deepseek/deepseek-v4-flash-0731 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__healthcare_life_sciences/deepseek~1deepseek-v4-flash-0731` | | | | | | | | | [MMLU Pro / math](https://benchize.com/benchmarks/vals-ai%3Ammlu_pro%3Amath.md) | 94% | SE ±0.65 pp | 25 / 40 | $0.00204 | 12.35source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_mmlu_pro.y416pXDi.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 94.004 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.002039 \| USD \| per_test \| Lower is better \| \| `latency` \| 12.347 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.65 pp sourcePopulation 138 stderr 0.646 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_mmlu_pro.y416pXDi.json) Source Vals AI Original model label deepseek/deepseek-v4-flash-0731 Source version 1 Observed 2026-10-21T16:46:06.503Z Evidence SHA-256 `dab025507f46c716dd8d145e90ec0a737d9c56219853f41e4b1cc3c5e8c222b4` Source locator `/tasks/math/deepseek~1deepseek-v4-flash-0731` | | | | | | | | | [Legal Research Bench / administrative regulatory](https://benchize.com/benchmarks/vals-ai%3Alegal_research%3Aadministrative_regulatory.md) | 36.36% | SE ±5.97 pp | 35 / 50 | $0.3 | 1,185.77source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 36.364 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.295871 \| USD \| per_test \| Lower is better \| \| `latency` \| 1185.766 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±5.97 pp sourcePopulation 82 stderr 5.967 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) Source Vals AI Original model label deepseek/deepseek-v4-flash-0731 Source version 1 Observed 2026-10-21T16:45:52.389Z Evidence SHA-256 `26c58dfd47d0d2dc86f33c6d056f30309e0ee63570e7cd752c593b385f9d862e` Source locator `/tasks/administrative_regulatory/deepseek~1deepseek-v4-flash-0731` | | | | | | | | | [CorpFin v2 / max fitting context](https://benchize.com/benchmarks/vals-ai%3Acorp_fin_v2%3Amax_fitting_context.md) | 62.7% | SE ±1.65 pp | 22 / 39 | $0.03 | 22.53source duration | [Vals AI · 2](https://www.vals.ai/_astro/benchmark_view_corp_fin_v2.CectKwl1.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 62.704 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.02548 \| USD \| per_test \| Lower is better \| \| `latency` \| 22.531 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.65 pp sourcePopulation 134 stderr 1.651 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_corp_fin_v2.CectKwl1.json) Source Vals AI Original model label deepseek/deepseek-v4-flash-0731 Source version 2 Observed 2026-10-21T16:45:57.998Z Evidence SHA-256 `7cf2217ab3728d5a84bffce82aab9fc52d7f810b0cbe8c65b387df47f3d31c9b` Source locator `/tasks/max_fitting_context/deepseek~1deepseek-v4-flash-0731` | | | | | | | | | [SkillsBench / no skills](https://benchize.com/benchmarks/vals-ai%3Askillsbench%3Ano_skills.md) | 38.25% | SE ±4.24 pp | 15 / 20 | $0.15 | 495.44source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_skillsbench.BE2TXB-M.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 38.252 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.151888 \| USD \| per_test \| Lower is better \| \| `latency` \| 495.441 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness OpenHands maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±4.24 pp sourcePopulation 35 stderr 4.242 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_skillsbench.BE2TXB-M.json) Source Vals AI Original model label deepseek/deepseek-v4-flash-0731 Source version 1 Observed 2026-10-21T16:46:11.147Z Evidence SHA-256 `aaad242949d81b7c7c61c36d4895d4460e9146760c1e6b4b38d235a16043bc61` Source locator `/tasks/no_skills/deepseek~1deepseek-v4-flash-0731` | | | | | | | | | [LegalBench / issue tasks](https://benchize.com/benchmarks/vals-ai%3Alegal_bench%3Aissue_tasks.md) | 81.96% | SE ±0.73 pp | 35 / 44 | $0.000484 | 2.83source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_bench.D6uTgA3b.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 81.956 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.000484 \| USD \| per_test \| Lower is better \| \| `latency` \| 2.829 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.73 pp sourcePopulation 151 stderr 0.727 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_bench.D6uTgA3b.json) Source Vals AI Original model label deepseek/deepseek-v4-flash-0731 Source version 1 Observed 2026-10-21T16:45:53.390Z Evidence SHA-256 `c8486ba756baa5cd3955d035edff67440e8db23e69d62ad1a9e18d15b8c4bc52` Source locator `/tasks/issue_tasks/deepseek~1deepseek-v4-flash-0731` | | | | | | | | | [FrontierCode 1.1 main](https://benchize.com/benchmarks/frontiercode%3Av1_1%3Amain.md) | 18.8% | — | 118 / 125 | $1.53 | — | [frontiercode](https://cognition.com/data/frontiercode-leaderboard/data.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 0.188 \| score_fraction \| blocking_criteria_adjusted \| Higher is better \| \| `correct` \| 0.2108 \| score_fraction \| raw_rubric \| Higher is better \| \| `cost` \| 1.5337 \| USD \| per_rollout \| Lower is better \| ### Configuration harness chisel Reasoning effort high sourceVersion v1_1 subset main ### Reported statistics flaggedRate 0.2547 sampleSize 100 ### Source [Original source](https://cognition.com/data/frontiercode-leaderboard/data.json) Source frontiercode Original model label DeepSeek V4 Flash 0731 Observed 2026-10-21T16:31:00.878Z Evidence SHA-256 `ff0040778b7bdf69c705ad7474f771fb5e53cfaf5cf8573b3a4614153543d75f` Source locator `/v1_1/data/DeepSeek%20V4%20Flash%200731/high/main` | | | | | | | | | [ProgramBench / overall](https://benchize.com/benchmarks/vals-ai%3Aprogrambench%3Aoverall.md) | 0% | SE ±0 pp | 23 / 40 | $1.29 | 20,327.6source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_programbench.C_J7ZU4X.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 1.285061 \| USD \| per_test \| Lower is better \| \| `latency` \| 20327.596 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 64 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_programbench.C_J7ZU4X.json) Source Vals AI Original model label deepseek/deepseek-v4-flash-0731 Source version 1 Observed 2026-10-21T16:46:10.137Z Evidence SHA-256 `e9bbb4378c7a058441a21d456baf9316b96c2cf5b58d371a3c7c803f5000bd8c` Source locator `/tasks/overall/deepseek~1deepseek-v4-flash-0731` | | | | | | | | | [LegalBench / interpretation tasks](https://benchize.com/benchmarks/vals-ai%3Alegal_bench%3Ainterpretation_tasks.md) | 70.99% | SE ±0.24 pp | 41 / 44 | $0.000592 | 3.94source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_bench.D6uTgA3b.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 70.993 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.000592 \| USD \| per_test \| Lower is better \| \| `latency` \| 3.943 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.24 pp sourcePopulation 151 stderr 0.236 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_bench.D6uTgA3b.json) Source Vals AI Original model label deepseek/deepseek-v4-flash-0731 Source version 1 Observed 2026-10-21T16:45:53.390Z Evidence SHA-256 `c8486ba756baa5cd3955d035edff67440e8db23e69d62ad1a9e18d15b8c4bc52` Source locator `/tasks/interpretation_tasks/deepseek~1deepseek-v4-flash-0731` | | | | | | | | | [Tax Agent Bench / numbers calculations](https://benchize.com/benchmarks/vals-ai%3Atax_agent_bench%3Anumbers_calculations.md) | 41.33% | SE ±8.88 pp | 18 / 46 | $0.07 | 178.13source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 41.333 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.071997 \| USD \| per_test \| Lower is better \| \| `latency` \| 178.13 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±8.88 pp sourcePopulation 73 stderr 8.876 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) Source Vals AI Original model label deepseek/deepseek-v4-flash-0731 Source version 1 Observed 2026-10-21T16:45:55.958Z Evidence SHA-256 `446db182b67efb7923b3187a4040ec3a3ea492b8ad699a2b6d708d714cc063ac` Source locator `/tasks/numbers_calculations/deepseek~1deepseek-v4-flash-0731` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / criteria pass rate banking finance](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__banking_finance.md) | 87.36% | SE ±2.03 pp | 25 / 49 | $0.05 | 638.05source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 87.361 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.053598 \| USD \| per_test \| Lower is better \| \| `latency` \| 638.052 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.03 pp sourcePopulation 81 stderr 2.03 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label deepseek/deepseek-v4-flash-0731 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__banking_finance/deepseek~1deepseek-v4-flash-0731` | | | | | | | | | [MMLU Pro / health](https://benchize.com/benchmarks/vals-ai%3Ammlu_pro%3Ahealth.md) | 80.69% | SE ±1.38 pp | 23 / 40 | $0.00177 | 13.27source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_mmlu_pro.y416pXDi.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 80.685 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.001767 \| USD \| per_test \| Lower is better \| \| `latency` \| 13.266 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.38 pp sourcePopulation 138 stderr 1.38 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_mmlu_pro.y416pXDi.json) Source Vals AI Original model label deepseek/deepseek-v4-flash-0731 Source version 1 Observed 2026-10-21T16:46:06.503Z Evidence SHA-256 `dab025507f46c716dd8d145e90ec0a737d9c56219853f41e4b1cc3c5e8c222b4` Source locator `/tasks/health/deepseek~1deepseek-v4-flash-0731` | | | | | | | | | [CorpFin v2 / overall](https://benchize.com/benchmarks/vals-ai%3Acorp_fin_v2%3Aoverall.md) | 61.85% | SE ±0.96 pp | 20 / 39 | $0.01 | 16.48source duration | [Vals AI · 2](https://www.vals.ai/_astro/benchmark_view_corp_fin_v2.CectKwl1.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 61.849 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.011801 \| USD \| per_test \| Lower is better \| \| `latency` \| 16.477 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.96 pp sourcePopulation 134 stderr 0.957 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_corp_fin_v2.CectKwl1.json) Source Vals AI Original model label deepseek/deepseek-v4-flash-0731 Source version 2 Observed 2026-10-21T16:45:57.998Z Evidence SHA-256 `7cf2217ab3728d5a84bffce82aab9fc52d7f810b0cbe8c65b387df47f3d31c9b` Source locator `/tasks/overall/deepseek~1deepseek-v4-flash-0731` | | | | | | | | | [EMB / dcf](https://benchize.com/benchmarks/vals-ai%3Aemb%3Adcf.md) | 51.29% | SE ±7.34 pp | 29 / 47 | $0.38 | 1,432.3source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 51.288 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.38317 \| USD \| per_test \| Lower is better \| \| `latency` \| 1432.295 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±7.34 pp sourcePopulation 76 stderr 7.335 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) Source Vals AI Original model label deepseek/deepseek-v4-flash-0731 Source version 1 Observed 2026-10-21T16:45:57.495Z Evidence SHA-256 `d807aee620ea47152a1fca0257a8dd12a1be4512c76cc1744c1ae9e285180918` Source locator `/tasks/dcf/deepseek~1deepseek-v4-flash-0731` | | | | | | | | | [Terminal-Bench 4.0 / operations](https://benchize.com/benchmarks/vals-ai%3Aterminal-bench-4%3Aoperations.md) | 18.52% | SE ±3.7 pp | 13 / 36 | — | — | [Vals AI · 4.0](https://www.vals.ai/_astro/benchmark_view_terminal_bench_4.CAnGyzC_.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 18.519 \| percent \| reported \| Higher is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±3.7 pp sourcePopulation 52 stderr 3.704 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_terminal_bench_4.CAnGyzC_.json) Source Vals AI Original model label deepseek/deepseek-v4-flash-0731 Source version 4.0 Observed 2026-10-21T16:46:10.642Z Evidence SHA-256 `98f6964580e901f1aa143db664bd92aa558415f643569492545d67c47d7f519b` Source locator `/tasks/operations/deepseek~1deepseek-v4-flash-0731` | | | | | | | | | [IOI / ioi2024](https://benchize.com/benchmarks/vals-ai%3Aioi%3Aioi2024.md) | 34% | SE ±14.89 pp | 26 / 29 | $0.35 | 1,373.25source duration | [Vals AI · 2](https://www.vals.ai/_astro/benchmark_view_ioi.CuUolFNv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 34 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.346944 \| USD \| per_test \| Lower is better \| \| `latency` \| 1373.252 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±14.89 pp sourcePopulation 44 stderr 14.887 tokenTotals {"cache_read_tokens":56395904,"cache_write_tokens":0,"input_tokens":56595542,"output_tokens":905221,"reasoning_tokens":766588} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_ioi.CuUolFNv.json) Source Vals AI Original model label deepseek/deepseek-v4-flash-0731 Source version 2 Observed 2026-10-21T16:46:11.757Z Evidence SHA-256 `e1545b21b5b5c98d037aefa9d441b65859d07a10550f242d0e16162a49b8e8ab` Source locator `/tasks/ioi2024/deepseek~1deepseek-v4-flash-0731` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / criteria pass rate intellectual property](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__intellectual_property.md) | 91.87% | SE ±1.63 pp | 16 / 49 | $0.06 | 1,121.5source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 91.873 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.055806 \| USD \| per_test \| Lower is better \| \| `latency` \| 1121.5 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.63 pp sourcePopulation 81 stderr 1.627 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label deepseek/deepseek-v4-flash-0731 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__intellectual_property/deepseek~1deepseek-v4-flash-0731` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / criteria pass rate funds asset management](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__funds_asset_management.md) | 86.3% | SE ±1.86 pp | 10 / 49 | $0.08 | 985.74source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 86.297 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.077692 \| USD \| per_test \| Lower is better \| \| `latency` \| 985.742 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.86 pp sourcePopulation 81 stderr 1.859 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label deepseek/deepseek-v4-flash-0731 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__funds_asset_management/deepseek~1deepseek-v4-flash-0731` | | | | | | | | | [SWE-bench / <15 min fix](https://benchize.com/benchmarks/vals-ai%3Aswebench%3A%3C15%20min%20fix.md) | 90.72% | SE ±2.09 pp | 10 / 39 | $0.0065 | 139.44source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_swebench.Ae8ni7h0.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 90.722 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.006499 \| USD \| per_test \| Lower is better \| \| `latency` \| 139.441 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.09 pp sourcePopulation 88 stderr 2.088 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_swebench.Ae8ni7h0.json) Source Vals AI Original model label deepseek/deepseek-v4-flash-0731 Source version 1 Observed 2026-10-21T16:46:13.711Z Evidence SHA-256 `7a4e2e8485283caac500127419f68ec4bd5414c837ade114fc6afa5b2d34ccaa` Source locator `/tasks/<15 min fix/deepseek~1deepseek-v4-flash-0731` | | | | | | | | | [Code Migration / cli](https://benchize.com/benchmarks/vals-ai%3Acode-migration%3Acli.md) | 31.99% | SE ±2.39 pp | 24 / 47 | $4.63 | 9,130.86source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_code_migration.Dttzoqk3.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 31.985 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 4.627483 \| USD \| per_test \| Lower is better \| \| `latency` \| 9130.856 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.39 pp sourcePopulation 79 stderr 2.386 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_code_migration.Dttzoqk3.json) Source Vals AI Original model label deepseek/deepseek-v4-flash-0731 Source version 1 Observed 2026-10-21T16:46:12.712Z Evidence SHA-256 `99194597f2c76c402195be47839409046e18bd30454616da6dd40b736a088fa4` Source locator `/tasks/cli/deepseek~1deepseek-v4-flash-0731` | | | | | | | | | [Legal Research Bench / business commercial](https://benchize.com/benchmarks/vals-ai%3Alegal_research%3Abusiness_commercial.md) | 23.26% | SE ±6.52 pp | 35 / 50 | $0.22 | 1,087.93source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 23.256 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.215579 \| USD \| per_test \| Lower is better \| \| `latency` \| 1087.928 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±6.52 pp sourcePopulation 82 stderr 6.519 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) Source Vals AI Original model label deepseek/deepseek-v4-flash-0731 Source version 1 Observed 2026-10-21T16:45:52.389Z Evidence SHA-256 `26c58dfd47d0d2dc86f33c6d056f30309e0ee63570e7cd752c593b385f9d862e` Source locator `/tasks/business_commercial/deepseek~1deepseek-v4-flash-0731` | | | | | | | | | [Finance Agent (v2) / financial modeling](https://benchize.com/benchmarks/vals-ai%3Afabv2%3Afinancial_modeling.md) | 19.84% | SE ±1.17 pp | 30 / 50 | $0.23 | 392.6source duration | [Vals AI · 2](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 19.84 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.231681 \| USD \| per_test \| Lower is better \| \| `latency` \| 392.596 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.17 pp sourcePopulation 80 stderr 1.171 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) Source Vals AI Original model label deepseek/deepseek-v4-flash-0731 Source version 2 Observed 2026-10-21T16:45:57.035Z Evidence SHA-256 `f731b20a26c126725d125519f2b5841aa45254656dba859a9570d7ce7e5d8191` Source locator `/tasks/financial_modeling/deepseek~1deepseek-v4-flash-0731` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / litigation dispute resolution](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Alitigation_dispute_resolution.md) | 0% | SE ±0 pp | 6 / 49 | $0.06 | 1,415.96source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.055896 \| USD \| per_test \| Lower is better \| \| `latency` \| 1415.958 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 81 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label deepseek/deepseek-v4-flash-0731 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/litigation_dispute_resolution/deepseek~1deepseek-v4-flash-0731` | | | | | | | | | [TaxEval v2 / stepwise reasoning](https://benchize.com/benchmarks/vals-ai%3Atax_eval_v2%3Astepwise_reasoning.md) | 80.87% | SE ±1.13 pp | 31 / 40 | $0.00976 | 72.2source duration | [Vals AI · 2](https://www.vals.ai/_astro/benchmark_view_tax_eval_v2.B3yoffr8.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 80.867 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.009755 \| USD \| per_test \| Lower is better \| \| `latency` \| 72.203 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.13 pp sourcePopulation 145 stderr 1.125 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_tax_eval_v2.B3yoffr8.json) Source Vals AI Original model label deepseek/deepseek-v4-flash-0731 Source version 2 Observed 2026-10-21T16:45:59.000Z Evidence SHA-256 `8b354a6ae848f5bb25a2e8b061b5f576d5b9e0c11b941dc1e6bed078e6529cd6` Source locator `/tasks/stepwise_reasoning/deepseek~1deepseek-v4-flash-0731` | | | | | | | | | [MMLU Pro / psychology](https://benchize.com/benchmarks/vals-ai%3Ammlu_pro%3Apsychology.md) | 85.59% | SE ±1.24 pp | 24 / 40 | $0.00106 | 8.21source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_mmlu_pro.y416pXDi.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 85.589 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.001056 \| USD \| per_test \| Lower is better \| \| `latency` \| 8.214 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.24 pp sourcePopulation 138 stderr 1.243 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_mmlu_pro.y416pXDi.json) Source Vals AI Original model label deepseek/deepseek-v4-flash-0731 Source version 1 Observed 2026-10-21T16:46:06.503Z Evidence SHA-256 `dab025507f46c716dd8d145e90ec0a737d9c56219853f41e4b1cc3c5e8c222b4` Source locator `/tasks/psychology/deepseek~1deepseek-v4-flash-0731` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / criteria pass rate litigation dispute resolution](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__litigation_dispute_resolution.md) | 86.89% | SE ±2.07 pp | 18 / 49 | $0.06 | 1,415.96source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 86.891 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.055896 \| USD \| per_test \| Lower is better \| \| `latency` \| 1415.958 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.07 pp sourcePopulation 81 stderr 2.069 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label deepseek/deepseek-v4-flash-0731 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__litigation_dispute_resolution/deepseek~1deepseek-v4-flash-0731` | | | | | | | | | [EMB / misc](https://benchize.com/benchmarks/vals-ai%3Aemb%3Amisc.md) | 57.62% | SE ±6.14 pp | 25 / 47 | $0.44 | 2,071.46source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 57.615 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.439265 \| USD \| per_test \| Lower is better \| \| `latency` \| 2071.462 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±6.14 pp sourcePopulation 76 stderr 6.136 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) Source Vals AI Original model label deepseek/deepseek-v4-flash-0731 Source version 1 Observed 2026-10-21T16:45:57.495Z Evidence SHA-256 `d807aee620ea47152a1fca0257a8dd12a1be4512c76cc1744c1ae9e285180918` Source locator `/tasks/misc/deepseek~1deepseek-v4-flash-0731` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / overall](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Aoverall.md) | 8.33% | SE ±2.15 pp | 15 / 49 | $0.06 | 1,014.03source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 8.333 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.059087 \| USD \| per_test \| Lower is better \| \| `latency` \| 1014.028 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.15 pp sourcePopulation 81 stderr 2.148 tokenTotals {"cache_read_tokens":723421312,"cache_write_tokens":null,"input_tokens":744084543,"output_tokens":7335131,"reasoning_tokens":2960038} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label deepseek/deepseek-v4-flash-0731 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/overall/deepseek~1deepseek-v4-flash-0731` | | | | | | | | | [SkillsBench / overall](https://benchize.com/benchmarks/vals-ai%3Askillsbench%3Aoverall.md) | 50.67% | SE ±4.43 pp | 17 / 20 | $0.13 | 411.96source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_skillsbench.BE2TXB-M.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 50.668 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.12607 \| USD \| per_test \| Lower is better \| \| `latency` \| 411.963 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness OpenHands maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±4.43 pp sourcePopulation 35 stderr 4.429 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_skillsbench.BE2TXB-M.json) Source Vals AI Original model label deepseek/deepseek-v4-flash-0731 Source version 1 Observed 2026-10-21T16:46:11.147Z Evidence SHA-256 `aaad242949d81b7c7c61c36d4895d4460e9146760c1e6b4b38d235a16043bc61` Source locator `/tasks/overall/deepseek~1deepseek-v4-flash-0731` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / banking finance](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Abanking_finance.md) | 0% | SE ±0 pp | 5 / 49 | $0.05 | 638.05source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.053598 \| USD \| per_test \| Lower is better \| \| `latency` \| 638.052 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 81 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label deepseek/deepseek-v4-flash-0731 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/banking_finance/deepseek~1deepseek-v4-flash-0731` | | | | | | | | | [Tax Agent Bench / current temporal analysis](https://benchize.com/benchmarks/vals-ai%3Atax_agent_bench%3Acurrent_temporal_analysis.md) | 42.99% | SE ±9.13 pp | 15 / 46 | $0.21 | 446.1source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 42.99 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.208159 \| USD \| per_test \| Lower is better \| \| `latency` \| 446.097 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±9.13 pp sourcePopulation 73 stderr 9.132 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) Source Vals AI Original model label deepseek/deepseek-v4-flash-0731 Source version 1 Observed 2026-10-21T16:45:55.958Z Evidence SHA-256 `446db182b67efb7923b3187a4040ec3a3ea492b8ad699a2b6d708d714cc063ac` Source locator `/tasks/current_temporal_analysis/deepseek~1deepseek-v4-flash-0731` | | | | | | | | | [MMLU Pro / history](https://benchize.com/benchmarks/vals-ai%3Ammlu_pro%3Ahistory.md) | 77.17% | SE ±2.15 pp | 24 / 40 | $0.00139 | 10.59source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_mmlu_pro.y416pXDi.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 77.165 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.001394 \| USD \| per_test \| Lower is better \| \| `latency` \| 10.586 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.15 pp sourcePopulation 138 stderr 2.151 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_mmlu_pro.y416pXDi.json) Source Vals AI Original model label deepseek/deepseek-v4-flash-0731 Source version 1 Observed 2026-10-21T16:46:06.503Z Evidence SHA-256 `dab025507f46c716dd8d145e90ec0a737d9c56219853f41e4b1cc3c5e8c222b4` Source locator `/tasks/history/deepseek~1deepseek-v4-flash-0731` | | | | | | | | | [Terminal-Bench 4.0 / hardware](https://benchize.com/benchmarks/vals-ai%3Aterminal-bench-4%3Ahardware.md) | 0% | SE ±0 pp | 21 / 36 | — | — | [Vals AI · 4.0](https://www.vals.ai/_astro/benchmark_view_terminal_bench_4.CAnGyzC_.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 52 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_terminal_bench_4.CAnGyzC_.json) Source Vals AI Original model label deepseek/deepseek-v4-flash-0731 Source version 4.0 Observed 2026-10-21T16:46:10.642Z Evidence SHA-256 `98f6964580e901f1aa143db664bd92aa558415f643569492545d67c47d7f519b` Source locator `/tasks/hardware/deepseek~1deepseek-v4-flash-0731` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / arbitration international dispute resolution](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Aarbitration_international_dispute_resolution.md) | 0% | SE ±0 pp | 4 / 49 | $0.08 | 926.49source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.076998 \| USD \| per_test \| Lower is better \| \| `latency` \| 926.49 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 81 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label deepseek/deepseek-v4-flash-0731 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/arbitration_international_dispute_resolution/deepseek~1deepseek-v4-flash-0731` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / real estate](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Areal_estate.md) | 0% | SE ±0 pp | 6 / 49 | $0.07 | 939.14source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.066263 \| USD \| per_test \| Lower is better \| \| `latency` \| 939.142 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 81 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label deepseek/deepseek-v4-flash-0731 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/real_estate/deepseek~1deepseek-v4-flash-0731` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / energy natural resources](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Aenergy_natural_resources.md) | 20% | SE ±20 pp | 5 / 49 | $0.1 | 1,183.83source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 20 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.100514 \| USD \| per_test \| Lower is better \| \| `latency` \| 1183.832 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±20 pp sourcePopulation 81 stderr 20 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label deepseek/deepseek-v4-flash-0731 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/energy_natural_resources/deepseek~1deepseek-v4-flash-0731` | | | | | | | | | [Tax Agent Bench / overall](https://benchize.com/benchmarks/vals-ai%3Atax_agent_bench%3Aoverall.md) | 28.07% | SE ±3.16 pp | 20 / 46 | $0.16 | 337.34source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 28.069 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.156388 \| USD \| per_test \| Lower is better \| \| `latency` \| 337.343 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±3.16 pp sourcePopulation 73 stderr 3.158 tokenTotals {"cache_read_tokens":97366102,"cache_write_tokens":null,"input_tokens":118557354,"output_tokens":14769391,"reasoning_tokens":11893686} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) Source Vals AI Original model label deepseek/deepseek-v4-flash-0731 Source version 1 Observed 2026-10-21T16:45:55.958Z Evidence SHA-256 `446db182b67efb7923b3187a4040ec3a3ea492b8ad699a2b6d708d714cc063ac` Source locator `/tasks/overall/deepseek~1deepseek-v4-flash-0731` | | | | | | | | | [Tax Agent Bench / weighted pass rule source lookup](https://benchize.com/benchmarks/vals-ai%3Atax_agent_bench%3Aweighted_pass__rule_source_lookup.md) | 70.18% | SE ±7.41 pp | 5 / 46 | $0.16 | 339.09source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 70.181 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.163641 \| USD \| per_test \| Lower is better \| \| `latency` \| 339.091 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±7.41 pp sourcePopulation 73 stderr 7.405 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) Source Vals AI Original model label deepseek/deepseek-v4-flash-0731 Source version 1 Observed 2026-10-21T16:45:55.958Z Evidence SHA-256 `446db182b67efb7923b3187a4040ec3a3ea492b8ad699a2b6d708d714cc063ac` Source locator `/tasks/weighted_pass__rule_source_lookup/deepseek~1deepseek-v4-flash-0731` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / criteria pass rate white collar defense investigations](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__white_collar_defense_investigations.md) | 86.2% | SE ±2.01 pp | 14 / 49 | $0.05 | 654.63source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 86.195 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.053956 \| USD \| per_test \| Lower is better \| \| `latency` \| 654.63 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.01 pp sourcePopulation 81 stderr 2.005 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label deepseek/deepseek-v4-flash-0731 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__white_collar_defense_investigations/deepseek~1deepseek-v4-flash-0731` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / data privacy cybersecurity](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Adata_privacy_cybersecurity.md) | 0% | SE ±0 pp | 13 / 49 | $0.07 | 821.75source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.067318 \| USD \| per_test \| Lower is better \| \| `latency` \| 821.752 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 81 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label deepseek/deepseek-v4-flash-0731 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/data_privacy_cybersecurity/deepseek~1deepseek-v4-flash-0731` | | | | | | | | | [Legal Research Bench / weighted pass](https://benchize.com/benchmarks/vals-ai%3Alegal_research%3Aweighted_pass.md) | 81.22% | SE ±1.51 pp | 31 / 50 | $0.28 | 1,266.74source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 81.223 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.275172 \| USD \| per_test \| Lower is better \| \| `latency` \| 1266.744 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.51 pp sourcePopulation 82 stderr 1.513 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) Source Vals AI Original model label deepseek/deepseek-v4-flash-0731 Source version 1 Observed 2026-10-21T16:45:52.389Z Evidence SHA-256 `26c58dfd47d0d2dc86f33c6d056f30309e0ee63570e7cd752c593b385f9d862e` Source locator `/tasks/weighted_pass/deepseek~1deepseek-v4-flash-0731` | | | | | | | | | [Terminal-Bench 4.0 / science](https://benchize.com/benchmarks/vals-ai%3Aterminal-bench-4%3Ascience.md) | 26.19% | SE ±8.59 pp | 16 / 36 | — | — | [Vals AI · 4.0](https://www.vals.ai/_astro/benchmark_view_terminal_bench_4.CAnGyzC_.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 26.19 \| percent \| reported \| Higher is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±8.59 pp sourcePopulation 52 stderr 8.585 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_terminal_bench_4.CAnGyzC_.json) Source Vals AI Original model label deepseek/deepseek-v4-flash-0731 Source version 4.0 Observed 2026-10-21T16:46:10.642Z Evidence SHA-256 `98f6964580e901f1aa143db664bd92aa558415f643569492545d67c47d7f519b` Source locator `/tasks/science/deepseek~1deepseek-v4-flash-0731` | | | | | | | | | [EMB / dataroom](https://benchize.com/benchmarks/vals-ai%3Aemb%3Adataroom.md) | 72.8% | SE ±7.81 pp | 35 / 47 | $0.3 | 1,440.12source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 72.803 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.298467 \| USD \| per_test \| Lower is better \| \| `latency` \| 1440.119 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±7.81 pp sourcePopulation 76 stderr 7.812 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) Source Vals AI Original model label deepseek/deepseek-v4-flash-0731 Source version 1 Observed 2026-10-21T16:45:57.495Z Evidence SHA-256 `d807aee620ea47152a1fca0257a8dd12a1be4512c76cc1744c1ae9e285180918` Source locator `/tasks/dataroom/deepseek~1deepseek-v4-flash-0731` | | | | | | | | | [MedCode / overall](https://benchize.com/benchmarks/vals-ai%3Amedcode%3Aoverall.md) | 41.42% | SE ±2.15 pp | 33 / 49 | $0.02 | 127source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_medcode.DehpEDgG.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 41.415 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.019659 \| USD \| per_test \| Lower is better \| \| `latency` \| 127 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 30000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.15 pp sourcePopulation 107 stderr 2.15 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_medcode.DehpEDgG.json) Source Vals AI Original model label deepseek/deepseek-v4-flash-0731 Source version 1 Observed 2026-10-21T16:45:59.516Z Evidence SHA-256 `988a6ae1e07e33ce031572ed4a804be11c36ee97a1dd75cbe4740fff3b534f8c` Source locator `/tasks/overall/deepseek~1deepseek-v4-flash-0731` | | | | | | | | | [SWE-bench / 1-4 hours](https://benchize.com/benchmarks/vals-ai%3Aswebench%3A1-4%20hours.md) | 90.48% | SE ±4.58 pp | 5 / 39 | $0.02 | 355.53source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_swebench.Ae8ni7h0.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 90.476 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.019578 \| USD \| per_test \| Lower is better \| \| `latency` \| 355.526 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±4.58 pp sourcePopulation 88 stderr 4.584 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_swebench.Ae8ni7h0.json) Source Vals AI Original model label deepseek/deepseek-v4-flash-0731 Source version 1 Observed 2026-10-21T16:46:13.711Z Evidence SHA-256 `7a4e2e8485283caac500127419f68ec4bd5414c837ade114fc6afa5b2d34ccaa` Source locator `/tasks/1-4 hours/deepseek~1deepseek-v4-flash-0731` | | | | | | | | | [EMB / operating](https://benchize.com/benchmarks/vals-ai%3Aemb%3Aoperating.md) | 55.98% | SE ±7.06 pp | 35 / 47 | $0.32 | 1,316.34source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 55.976 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.32347 \| USD \| per_test \| Lower is better \| \| `latency` \| 1316.344 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±7.06 pp sourcePopulation 76 stderr 7.061 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) Source Vals AI Original model label deepseek/deepseek-v4-flash-0731 Source version 1 Observed 2026-10-21T16:45:57.495Z Evidence SHA-256 `d807aee620ea47152a1fca0257a8dd12a1be4512c76cc1744c1ae9e285180918` Source locator `/tasks/operating/deepseek~1deepseek-v4-flash-0731` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / healthcare life sciences](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Ahealthcare_life_sciences.md) | 40% | SE ±24.5 pp | 2 / 49 | $0.06 | 1,043.86source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 40 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.056365 \| USD \| per_test \| Lower is better \| \| `latency` \| 1043.858 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±24.5 pp sourcePopulation 81 stderr 24.495 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label deepseek/deepseek-v4-flash-0731 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/healthcare_life_sciences/deepseek~1deepseek-v4-flash-0731` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / structured finance securitization](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Astructured_finance_securitization.md) | 20% | SE ±20 pp | 1 / 49 | $0.06 | 783.04source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 20 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.055487 \| USD \| per_test \| Lower is better \| \| `latency` \| 783.044 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±20 pp sourcePopulation 81 stderr 20 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label deepseek/deepseek-v4-flash-0731 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/structured_finance_securitization/deepseek~1deepseek-v4-flash-0731` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / insurance](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Ainsurance.md) | 0% | SE ±0 pp | 2 / 49 | $0.04 | 607.28source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.042314 \| USD \| per_test \| Lower is better \| \| `latency` \| 607.276 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 81 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label deepseek/deepseek-v4-flash-0731 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/insurance/deepseek~1deepseek-v4-flash-0731` | | | | | | | | | [Legal Research Bench / constitutional civil rights](https://benchize.com/benchmarks/vals-ai%3Alegal_research%3Aconstitutional_civil_rights.md) | 31.58% | SE ±7.64 pp | 27 / 50 | $0.2 | 1,025.06source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 31.579 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.204391 \| USD \| per_test \| Lower is better \| \| `latency` \| 1025.06 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±7.64 pp sourcePopulation 82 stderr 7.642 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) Source Vals AI Original model label deepseek/deepseek-v4-flash-0731 Source version 1 Observed 2026-10-21T16:45:52.389Z Evidence SHA-256 `26c58dfd47d0d2dc86f33c6d056f30309e0ee63570e7cd752c593b385f9d862e` Source locator `/tasks/constitutional_civil_rights/deepseek~1deepseek-v4-flash-0731` | | | | | | | | | [LegalBench / conclusion tasks](https://benchize.com/benchmarks/vals-ai%3Alegal_bench%3Aconclusion_tasks.md) | 88.89% | SE ±0.74 pp | 21 / 44 | $0.00036 | 3.2source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_bench.D6uTgA3b.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 88.892 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.00036 \| USD \| per_test \| Lower is better \| \| `latency` \| 3.197 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.74 pp sourcePopulation 151 stderr 0.742 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_bench.D6uTgA3b.json) Source Vals AI Original model label deepseek/deepseek-v4-flash-0731 Source version 1 Observed 2026-10-21T16:45:53.390Z Evidence SHA-256 `c8486ba756baa5cd3955d035edff67440e8db23e69d62ad1a9e18d15b8c4bc52` Source locator `/tasks/conclusion_tasks/deepseek~1deepseek-v4-flash-0731` | | | | | | | | | [Terminal-Bench 4.0 / media](https://benchize.com/benchmarks/vals-ai%3Aterminal-bench-4%3Amedia.md) | 16.67% | SE ±8.33 pp | 17 / 36 | — | — | [Vals AI · 4.0](https://www.vals.ai/_astro/benchmark_view_terminal_bench_4.CAnGyzC_.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 16.667 \| percent \| reported \| Higher is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±8.33 pp sourcePopulation 52 stderr 8.333 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_terminal_bench_4.CAnGyzC_.json) Source Vals AI Original model label deepseek/deepseek-v4-flash-0731 Source version 4.0 Observed 2026-10-21T16:46:10.642Z Evidence SHA-256 `98f6964580e901f1aa143db664bd92aa558415f643569492545d67c47d7f519b` Source locator `/tasks/media/deepseek~1deepseek-v4-flash-0731` | | | | | | | | | [Legal Research Bench / health](https://benchize.com/benchmarks/vals-ai%3Alegal_research%3Ahealth.md) | 36.67% | SE ±8.95 pp | 36 / 50 | $0.4 | 1,384.62source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 36.667 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.400672 \| USD \| per_test \| Lower is better \| \| `latency` \| 1384.623 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±8.95 pp sourcePopulation 82 stderr 8.949 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) Source Vals AI Original model label deepseek/deepseek-v4-flash-0731 Source version 1 Observed 2026-10-21T16:45:52.389Z Evidence SHA-256 `26c58dfd47d0d2dc86f33c6d056f30309e0ee63570e7cd752c593b385f9d862e` Source locator `/tasks/health/deepseek~1deepseek-v4-flash-0731` | | | | | | | | | [Legal Research Bench / criminal](https://benchize.com/benchmarks/vals-ai%3Alegal_research%3Acriminal.md) | 24.56% | SE ±5.75 pp | 36 / 50 | $0.25 | 1,264.86source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 24.561 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.25224 \| USD \| per_test \| Lower is better \| \| `latency` \| 1264.855 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±5.75 pp sourcePopulation 82 stderr 5.752 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) Source Vals AI Original model label deepseek/deepseek-v4-flash-0731 Source version 1 Observed 2026-10-21T16:45:52.389Z Evidence SHA-256 `26c58dfd47d0d2dc86f33c6d056f30309e0ee63570e7cd752c593b385f9d862e` Source locator `/tasks/criminal/deepseek~1deepseek-v4-flash-0731` | | | | | | | | | [LegalBench / rhetoric tasks](https://benchize.com/benchmarks/vals-ai%3Alegal_bench%3Arhetoric_tasks.md) | 70.44% | SE ±1.48 pp | 40 / 44 | $0.000351 | 2.9source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_bench.D6uTgA3b.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 70.442 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.000351 \| USD \| per_test \| Lower is better \| \| `latency` \| 2.898 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.48 pp sourcePopulation 151 stderr 1.479 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_bench.D6uTgA3b.json) Source Vals AI Original model label deepseek/deepseek-v4-flash-0731 Source version 1 Observed 2026-10-21T16:45:53.390Z Evidence SHA-256 `c8486ba756baa5cd3955d035edff67440e8db23e69d62ad1a9e18d15b8c4bc52` Source locator `/tasks/rhetoric_tasks/deepseek~1deepseek-v4-flash-0731` | | | | | | | | | [ProgramBench / strict](https://benchize.com/benchmarks/vals-ai%3Aprogrambench%3Astrict.md) | 0% | SE ±0 pp | 23 / 40 | $1.29 | 20,327.6source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_programbench.C_J7ZU4X.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 1.285061 \| USD \| per_test \| Lower is better \| \| `latency` \| 20327.596 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 64 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_programbench.C_J7ZU4X.json) Source Vals AI Original model label deepseek/deepseek-v4-flash-0731 Source version 1 Observed 2026-10-21T16:46:10.137Z Evidence SHA-256 `e9bbb4378c7a058441a21d456baf9316b96c2cf5b58d371a3c7c803f5000bd8c` Source locator `/tasks/strict/deepseek~1deepseek-v4-flash-0731` | | | | | | | | | [Finance Agent (v2) / general qualitative analysis](https://benchize.com/benchmarks/vals-ai%3Afabv2%3Ageneral_qualitative_analysis.md) | 60.45% | SE ±0.93 pp | 33 / 50 | $0.34 | 584.89source duration | [Vals AI · 2](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 60.449 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.337187 \| USD \| per_test \| Lower is better \| \| `latency` \| 584.887 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.93 pp sourcePopulation 80 stderr 0.925 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) Source Vals AI Original model label deepseek/deepseek-v4-flash-0731 Source version 2 Observed 2026-10-21T16:45:57.035Z Evidence SHA-256 `f731b20a26c126725d125519f2b5841aa45254656dba859a9570d7ce7e5d8191` Source locator `/tasks/general_qualitative_analysis/deepseek~1deepseek-v4-flash-0731` | | | | | | | | | [GPQA Diamond](https://benchize.com/benchmarks/gpqa-diamond.md) | 89.9% | SE ±1.69 pp | 18 / 40 | $0.01 | 79.96source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_gpqa.ghZZBpyU.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 89.899 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.012875 \| USD \| per_test \| Lower is better \| \| `latency` \| 79.956 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.69 pp sourcePopulation 138 stderr 1.694 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_gpqa.ghZZBpyU.json) Source Vals AI Original model label deepseek/deepseek-v4-flash-0731 Source version 1 Observed 2026-10-21T16:46:06.040Z Evidence SHA-256 `951eb5224097733a9d09d06c14908ae2980e2e640e539f2c284877c5092d7077` Source locator `/tasks/overall/deepseek~1deepseek-v4-flash-0731` | | | | | | | | | [EMB / lbo](https://benchize.com/benchmarks/vals-ai%3Aemb%3Albo.md) | 46.48% | SE ±6.14 pp | 33 / 47 | $0.6 | 2,285.28source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 46.475 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.604921 \| USD \| per_test \| Lower is better \| \| `latency` \| 2285.283 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±6.14 pp sourcePopulation 76 stderr 6.136 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) Source Vals AI Original model label deepseek/deepseek-v4-flash-0731 Source version 1 Observed 2026-10-21T16:45:57.495Z Evidence SHA-256 `d807aee620ea47152a1fca0257a8dd12a1be4512c76cc1744c1ae9e285180918` Source locator `/tasks/lbo/deepseek~1deepseek-v4-flash-0731` | | | | | | | | | [Legal Research Bench / overall](https://benchize.com/benchmarks/vals-ai%3Alegal_research%3Aoverall.md) | 30.29% | SE ±3.19 pp | 33 / 50 | $0.28 | 1,266.74source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 30.288 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.275172 \| USD \| per_test \| Lower is better \| \| `latency` \| 1266.744 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±3.19 pp sourcePopulation 82 stderr 3.194 tokenTotals {"cache_read_tokens":261666816,"cache_write_tokens":0,"input_tokens":310561183,"output_tokens":24287114,"reasoning_tokens":20969113} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) Source Vals AI Original model label deepseek/deepseek-v4-flash-0731 Source version 1 Observed 2026-10-21T16:45:52.389Z Evidence SHA-256 `26c58dfd47d0d2dc86f33c6d056f30309e0ee63570e7cd752c593b385f9d862e` Source locator `/tasks/overall/deepseek~1deepseek-v4-flash-0731` | | | | | | | | | [Finance Agent (v2) / all pass](https://benchize.com/benchmarks/vals-ai%3Afabv2%3Aall_pass.md) | 36.75% | SE ±0.57 pp | 34 / 50 | $0.28 | 490.25source duration | [Vals AI · 2](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 36.75 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.276154 \| USD \| per_test \| Lower is better \| \| `latency` \| 490.245 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.57 pp sourcePopulation 80 stderr 0.572 tokenTotals {"cache_read_tokens":213584768,"cache_write_tokens":null,"input_tokens":406112640,"output_tokens":27423276,"reasoning_tokens":22726163} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) Source Vals AI Original model label deepseek/deepseek-v4-flash-0731 Source version 2 Observed 2026-10-21T16:45:57.035Z Evidence SHA-256 `f731b20a26c126725d125519f2b5841aa45254656dba859a9570d7ce7e5d8191` Source locator `/tasks/all_pass/deepseek~1deepseek-v4-flash-0731` | | | | | | | | | [SkillsBench / with skills](https://benchize.com/benchmarks/vals-ai%3Askillsbench%3Awith_skills.md) | 50.67% | SE ±4.43 pp | 17 / 20 | $0.13 | 411.96source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_skillsbench.BE2TXB-M.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 50.668 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.12607 \| USD \| per_test \| Lower is better \| \| `latency` \| 411.963 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness OpenHands maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±4.43 pp sourcePopulation 35 stderr 4.429 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_skillsbench.BE2TXB-M.json) Source Vals AI Original model label deepseek/deepseek-v4-flash-0731 Source version 1 Observed 2026-10-21T16:46:11.147Z Evidence SHA-256 `aaad242949d81b7c7c61c36d4895d4460e9146760c1e6b4b38d235a16043bc61` Source locator `/tasks/with_skills/deepseek~1deepseek-v4-flash-0731` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / criteria pass rate trusts estates private client](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__trusts_estates_private_client.md) | 89.86% | SE ±2.1 pp | 8 / 49 | $0.06 | 1,181.33source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 89.855 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.059982 \| USD \| per_test \| Lower is better \| \| `latency` \| 1181.327 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.1 pp sourcePopulation 81 stderr 2.104 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label deepseek/deepseek-v4-flash-0731 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__trusts_estates_private_client/deepseek~1deepseek-v4-flash-0731` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / tax](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Atax.md) | 0% | SE ±0 pp | 3 / 49 | $0.07 | 961.29source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.065423 \| USD \| per_test \| Lower is better \| \| `latency` \| 961.292 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 81 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label deepseek/deepseek-v4-flash-0731 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/tax/deepseek~1deepseek-v4-flash-0731` | | | | | | | | | [MMLU Pro / physics](https://benchize.com/benchmarks/vals-ai%3Ammlu_pro%3Aphysics.md) | 92.38% | SE ±0.74 pp | 18 / 40 | $0.0028 | 17.44source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_mmlu_pro.y416pXDi.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 92.379 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.002804 \| USD \| per_test \| Lower is better \| \| `latency` \| 17.441 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.74 pp sourcePopulation 138 stderr 0.736 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_mmlu_pro.y416pXDi.json) Source Vals AI Original model label deepseek/deepseek-v4-flash-0731 Source version 1 Observed 2026-10-21T16:46:06.503Z Evidence SHA-256 `dab025507f46c716dd8d145e90ec0a737d9c56219853f41e4b1cc3c5e8c222b4` Source locator `/tasks/physics/deepseek~1deepseek-v4-flash-0731` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / criteria pass rate arbitration international dispute resolution](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__arbitration_international_dispute_resolution.md) | 90.68% | SE ±1.74 pp | 9 / 49 | $0.08 | 926.49source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 90.681 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.076998 \| USD \| per_test \| Lower is better \| \| `latency` \| 926.49 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.74 pp sourcePopulation 81 stderr 1.743 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label deepseek/deepseek-v4-flash-0731 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__arbitration_international_dispute_resolution/deepseek~1deepseek-v4-flash-0731` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / criteria pass rate capital markets](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__capital_markets.md) | 90.98% | SE ±1.8 pp | 17 / 49 | $0.05 | 1,121.47source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 90.98 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.048956 \| USD \| per_test \| Lower is better \| \| `latency` \| 1121.472 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.8 pp sourcePopulation 81 stderr 1.797 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label deepseek/deepseek-v4-flash-0731 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__capital_markets/deepseek~1deepseek-v4-flash-0731` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / criteria pass rate](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate.md) | 88.82% | SE ±0.38 pp | 12 / 49 | $0.06 | 1,014.03source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 88.815 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.059087 \| USD \| per_test \| Lower is better \| \| `latency` \| 1014.028 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.38 pp sourcePopulation 81 stderr 0.382 tokenTotals {"cache_read_tokens":723421312,"cache_write_tokens":null,"input_tokens":744084543,"output_tokens":7335131,"reasoning_tokens":2960038} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label deepseek/deepseek-v4-flash-0731 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate/deepseek~1deepseek-v4-flash-0731` | | | | | | | | | [LiveCodeBench / medium](https://benchize.com/benchmarks/vals-ai%3Alcb%3Amedium.md) | 89.03% | SE ±1.6 pp | 16 / 43 | $0.01 | 82.77source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_lcb.miyRUvVa.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 89.034 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.014546 \| USD \| per_test \| Lower is better \| \| `latency` \| 82.766 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.6 pp sourcePopulation 143 stderr 1.597 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_lcb.miyRUvVa.json) Source Vals AI Original model label deepseek/deepseek-v4-flash-0731 Source version 1 Observed 2026-10-21T16:46:14.220Z Evidence SHA-256 `1a7c7dabc9d95a0182886b1edc42623987cac2095a0d407a5e334f37acdfa2f5` Source locator `/tasks/medium/deepseek~1deepseek-v4-flash-0731` | | | | | | | | | [Terminal-Bench 4.0 / security](https://benchize.com/benchmarks/vals-ai%3Aterminal-bench-4%3Asecurity.md) | 6.67% | SE ±6.67 pp | 24 / 36 | — | — | [Vals AI · 4.0](https://www.vals.ai/_astro/benchmark_view_terminal_bench_4.CAnGyzC_.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 6.667 \| percent \| reported \| Higher is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±6.67 pp sourcePopulation 52 stderr 6.667 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_terminal_bench_4.CAnGyzC_.json) Source Vals AI Original model label deepseek/deepseek-v4-flash-0731 Source version 4.0 Observed 2026-10-21T16:46:10.642Z Evidence SHA-256 `98f6964580e901f1aa143db664bd92aa558415f643569492545d67c47d7f519b` Source locator `/tasks/security/deepseek~1deepseek-v4-flash-0731` | | | | | | | | | [LiveCodeBench / overall](https://benchize.com/benchmarks/vals-ai%3Alcb%3Aoverall.md) | 87.26% | SE ±0.97 pp | 14 / 43 | $0.02 | 126.61source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_lcb.miyRUvVa.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 87.264 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.020251 \| USD \| per_test \| Lower is better \| \| `latency` \| 126.61 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.97 pp sourcePopulation 143 stderr 0.968 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_lcb.miyRUvVa.json) Source Vals AI Original model label deepseek/deepseek-v4-flash-0731 Source version 1 Observed 2026-10-21T16:46:14.220Z Evidence SHA-256 `1a7c7dabc9d95a0182886b1edc42623987cac2095a0d407a5e334f37acdfa2f5` Source locator `/tasks/overall/deepseek~1deepseek-v4-flash-0731` | | | | | | | | | [Terminal-Bench-Science / mathematical sciences](https://benchize.com/benchmarks/vals-ai%3Aterminal-bench-science%3Amathematical_sciences.md) | 5.88% | SE ±5.88 pp | 15 / 30 | — | — | [Vals AI · 0.1](https://www.vals.ai/_astro/benchmark_view_terminal_bench_science.o-HR8Nyt.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 5.882 \| percent \| reported \| Higher is better \| ### Configuration computeEffort Not reported harness Mini-SWE-agent maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±5.88 pp sourcePopulation 41 stderr 5.882 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_terminal_bench_science.o-HR8Nyt.json) Source Vals AI Original model label deepseek/deepseek-v4-flash-0731 Source version 0.1 Observed 2026-10-21T16:46:03.529Z Evidence SHA-256 `a30a8618deaecf018ea1c8c9f675576cdd9ab79742d05788067db49afcc5f91b` Source locator `/tasks/mathematical_sciences/deepseek~1deepseek-v4-flash-0731` | | | | | | | | | [EMB / ma](https://benchize.com/benchmarks/vals-ai%3Aemb%3Ama.md) | 54.76% | SE ±9.21 pp | 36 / 47 | $0.28 | 1,184.38source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 54.761 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.283682 \| USD \| per_test \| Lower is better \| \| `latency` \| 1184.383 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±9.21 pp sourcePopulation 76 stderr 9.208 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) Source Vals AI Original model label deepseek/deepseek-v4-flash-0731 Source version 1 Observed 2026-10-21T16:45:57.495Z Evidence SHA-256 `d807aee620ea47152a1fca0257a8dd12a1be4512c76cc1744c1ae9e285180918` Source locator `/tasks/ma/deepseek~1deepseek-v4-flash-0731` | | | | | | | | | [ToneBench Scriptwriting](https://benchize.com/benchmarks/tonebench%3Ascripts.md) | 1,680.5 | Range 1,640.7–1,718.6 | 59 / 126 | — | — | [tonebench](https://benchmark.towardsai.com/data/leaderboard.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `elo` \| 1680.5 \| rating \| reported \| Higher is better \| \| `overall` \| 77.47 \| percent \| reported \| Higher is better \| \| `anti_slop_numbers` \| 62.03 \| source_score \| rubric \| Higher is better \| \| `continuity_emotion` \| 75.95 \| source_score \| rubric \| Higher is better \| \| `cue_quality` \| 73.57 \| source_score \| rubric \| Higher is better \| \| `hook_strength` \| 80.51 \| source_score \| rubric \| Higher is better \| \| `length_adherence` \| 95.84 \| source_score \| rubric \| Higher is better \| \| `slop` \| 78.17 \| source_score \| rubric \| Higher is better \| \| `slop_score` \| 94.3 \| source_score \| rubric \| Higher is better \| \| `substance_accuracy` \| 80.41 \| source_score \| rubric \| Higher is better \| \| `tone_voice` \| 72.77 \| source_score \| rubric \| Higher is better \| \| `writing_quality` \| 71.44 \| source_score \| rubric \| Higher is better \| \| `youtube_best_practices` \| 75.81 \| source_score \| rubric \| Higher is better \| ### Configuration provider openrouter Reasoning effort unknown scoreScope 10_task_current sourceConfiguration deepseek-v4-flash-0731-or thinkingEnabled true ### Reported statistics Uncertainty Reported interval: 1,640.7–1,718.6 interval [1640.7,1718.6] rank 59 runCount 5 ### Source [Original source](https://benchmark.towardsai.com/data/leaderboard.json) Source tonebench Original model label DeepSeek V4 Flash 0731 (high) Evaluated 2026-10-08 Observed 2026-10-21T16:32:48.862Z Evidence SHA-256 `4264338337f418d3ded7ace250cad64fe12cacbd4b79a312bfaf0ae9268a01c3` Source locator `/rows/58` | | | | | | | | | [Tax Agent Bench / weighted pass forms filings](https://benchize.com/benchmarks/vals-ai%3Atax_agent_bench%3Aweighted_pass__forms_filings.md) | 59.42% | SE ±7.95 pp | 22 / 46 | $0.18 | 400.47source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 59.417 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.181934 \| USD \| per_test \| Lower is better \| \| `latency` \| 400.472 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±7.95 pp sourcePopulation 73 stderr 7.946 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) Source Vals AI Original model label deepseek/deepseek-v4-flash-0731 Source version 1 Observed 2026-10-21T16:45:55.958Z Evidence SHA-256 `446db182b67efb7923b3187a4040ec3a3ea492b8ad699a2b6d708d714cc063ac` Source locator `/tasks/weighted_pass__forms_filings/deepseek~1deepseek-v4-flash-0731` | | | | | | | | | [Tax Agent Bench / forms filings](https://benchize.com/benchmarks/vals-ai%3Atax_agent_bench%3Aforms_filings.md) | 15.67% | SE ±6.57 pp | 21 / 46 | $0.18 | 400.47source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 15.667 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.181934 \| USD \| per_test \| Lower is better \| \| `latency` \| 400.472 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±6.57 pp sourcePopulation 73 stderr 6.569 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) Source Vals AI Original model label deepseek/deepseek-v4-flash-0731 Source version 1 Observed 2026-10-21T16:45:55.958Z Evidence SHA-256 `446db182b67efb7923b3187a4040ec3a3ea492b8ad699a2b6d708d714cc063ac` Source locator `/tasks/forms_filings/deepseek~1deepseek-v4-flash-0731` | | | | | | | | | [GPQA Diamond / diamond few shot cot](https://benchize.com/benchmarks/vals-ai%3Agpqa%3Adiamond_few_shot_cot.md) | 89.9% | SE ±2.23 pp | 18 / 40 | $0.01 | 80.11source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_gpqa.ghZZBpyU.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 89.899 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.012966 \| USD \| per_test \| Lower is better \| \| `latency` \| 80.111 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.23 pp sourcePopulation 138 stderr 2.233 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_gpqa.ghZZBpyU.json) Source Vals AI Original model label deepseek/deepseek-v4-flash-0731 Source version 1 Observed 2026-10-21T16:46:06.040Z Evidence SHA-256 `951eb5224097733a9d09d06c14908ae2980e2e640e539f2c284877c5092d7077` Source locator `/tasks/diamond_few_shot_cot/deepseek~1deepseek-v4-flash-0731` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / criteria pass rate insurance](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__insurance.md) | 87.55% | SE ±2 pp | 19 / 49 | $0.04 | 607.28source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 87.546 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.042314 \| USD \| per_test \| Lower is better \| \| `latency` \| 607.276 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2 pp sourcePopulation 81 stderr 2.002 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label deepseek/deepseek-v4-flash-0731 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__insurance/deepseek~1deepseek-v4-flash-0731` | | | | | | | | | [SWE-bench / 15 min - 1 hour](https://benchize.com/benchmarks/vals-ai%3Aswebench%3A15%20min%20-%201%20hour.md) | 87.36% | SE ±2.06 pp | 8 / 39 | $0.01 | 222.84source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_swebench.Ae8ni7h0.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 87.356 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.010774 \| USD \| per_test \| Lower is better \| \| `latency` \| 222.837 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.06 pp sourcePopulation 88 stderr 2.061 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_swebench.Ae8ni7h0.json) Source Vals AI Original model label deepseek/deepseek-v4-flash-0731 Source version 1 Observed 2026-10-21T16:46:13.711Z Evidence SHA-256 `7a4e2e8485283caac500127419f68ec4bd5414c837ade114fc6afa5b2d34ccaa` Source locator `/tasks/15 min - 1 hour/deepseek~1deepseek-v4-flash-0731` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / criteria pass rate antitrust competition](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__antitrust_competition.md) | 91.3% | SE ±1.7 pp | 8 / 49 | $0.05 | 723.78source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 91.304 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.052986 \| USD \| per_test \| Lower is better \| \| `latency` \| 723.782 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.7 pp sourcePopulation 81 stderr 1.699 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label deepseek/deepseek-v4-flash-0731 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__antitrust_competition/deepseek~1deepseek-v4-flash-0731` | | | | | | | | | [ARC-AGI-2](https://benchize.com/benchmarks/arc-prize%3Av2_Semi_Private.md) | 55.97% | — | 77 / 212 | 0.05usd per task | — | [arc prize · v2](https://arcprize.org/media/data/leaderboard/v2.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 0.5597222222222221 \| score_fraction \| v2_Semi_Private \| Higher is better \| \| `cost_per_task` \| 0.0453381305 \| usd_per_task \| v2_Semi_Private \| Lower is better \| ### Configuration evaluationSplit v2_Semi_Private harness Not reported modelGroup deepseek-v4-flash-0731 modelType CoT Reasoning effort high sourceModelVersion deepseek-v4-flash-0731-high thinkingEnabled true ### Source [Original source](https://arcprize.org/media/data/leaderboard/v2.json) Source arc prize Original model label DeepSeek V4 Flash 0731 (High) Source version v2 Observed 2026-10-21T16:34:20.750Z Evidence SHA-256 `bca68e12cc9a01954ae21b641844c1ecbabe146cb24b87f0f8e9b1db27433c29` Source locator `/evaluations/193` | | | | | | | | | [LiveCodeBench / hard](https://benchize.com/benchmarks/vals-ai%3Alcb%3Ahard.md) | 74% | SE ±2.35 pp | 11 / 43 | $0.04 | 286.14source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_lcb.miyRUvVa.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 74 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.044236 \| USD \| per_test \| Lower is better \| \| `latency` \| 286.143 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.35 pp sourcePopulation 143 stderr 2.345 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_lcb.miyRUvVa.json) Source Vals AI Original model label deepseek/deepseek-v4-flash-0731 Source version 1 Observed 2026-10-21T16:46:14.220Z Evidence SHA-256 `1a7c7dabc9d95a0182886b1edc42623987cac2095a0d407a5e334f37acdfa2f5` Source locator `/tasks/hard/deepseek~1deepseek-v4-flash-0731` | | | | | | | | | [Finance Agent (v2) / earnings analysis](https://benchize.com/benchmarks/vals-ai%3Afabv2%3Aearnings_analysis.md) | 69.78% | SE ±1.77 pp | 32 / 50 | $0.09 | 263.45source duration | [Vals AI · 2](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 69.777 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.087508 \| USD \| per_test \| Lower is better \| \| `latency` \| 263.446 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.77 pp sourcePopulation 80 stderr 1.773 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) Source Vals AI Original model label deepseek/deepseek-v4-flash-0731 Source version 2 Observed 2026-10-21T16:45:57.035Z Evidence SHA-256 `f731b20a26c126725d125519f2b5841aa45254656dba859a9570d7ce7e5d8191` Source locator `/tasks/earnings_analysis/deepseek~1deepseek-v4-flash-0731` | | | | | | | | | [ProgramBench / partial](https://benchize.com/benchmarks/vals-ai%3Aprogrambench%3Apartial.md) | 57.29% | SE ±2.39 pp | 29 / 40 | $1.29 | 20,327.6source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_programbench.C_J7ZU4X.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 57.291 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 1.285061 \| USD \| per_test \| Lower is better \| \| `latency` \| 20327.596 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.39 pp sourcePopulation 64 stderr 2.386 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_programbench.C_J7ZU4X.json) Source Vals AI Original model label deepseek/deepseek-v4-flash-0731 Source version 1 Observed 2026-10-21T16:46:10.137Z Evidence SHA-256 `e9bbb4378c7a058441a21d456baf9316b96c2cf5b58d371a3c7c803f5000bd8c` Source locator `/tasks/partial/deepseek~1deepseek-v4-flash-0731` | | | | | | | | | [EMB / overall](https://benchize.com/benchmarks/vals-ai%3Aemb%3Aoverall.md) | 56.98% | SE ±3.39 pp | 32 / 47 | $0.38 | 1,574.1source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 56.977 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.383709 \| USD \| per_test \| Lower is better \| \| `latency` \| 1574.097 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±3.39 pp sourcePopulation 76 stderr 3.391 tokenTotals {"cache_read_tokens":1008679680,"cache_write_tokens":null,"input_tokens":1018385162,"output_tokens":15135599,"reasoning_tokens":11153857} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) Source Vals AI Original model label deepseek/deepseek-v4-flash-0731 Source version 1 Observed 2026-10-21T16:45:57.495Z Evidence SHA-256 `d807aee620ea47152a1fca0257a8dd12a1be4512c76cc1744c1ae9e285180918` Source locator `/tasks/overall/deepseek~1deepseek-v4-flash-0731` | | | | | | | | | [IOI / overall](https://benchize.com/benchmarks/vals-ai%3Aioi%3Aoverall.md) | 32.72% | SE ±2.35 pp | 27 / 29 | $0.37 | 1,663.09source duration | [Vals AI · 2](https://www.vals.ai/_astro/benchmark_view_ioi.CuUolFNv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 32.722 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.373112 \| USD \| per_test \| Lower is better \| \| `latency` \| 1663.091 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.35 pp sourcePopulation 44 stderr 2.35 tokenTotals {"cache_read_tokens":53385216,"cache_write_tokens":0,"input_tokens":53634013,"output_tokens":1034888,"reasoning_tokens":889275} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_ioi.CuUolFNv.json) Source Vals AI Original model label deepseek/deepseek-v4-flash-0731 Source version 2 Observed 2026-10-21T16:46:11.757Z Evidence SHA-256 `e1545b21b5b5c98d037aefa9d441b65859d07a10550f242d0e16162a49b8e8ab` Source locator `/tasks/overall/deepseek~1deepseek-v4-flash-0731` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / criteria pass rate bankruptcy restructuring](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__bankruptcy_restructuring.md) | 87.89% | SE ±1.92 pp | 8 / 49 | $0.07 | 2,251.58source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 87.889 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.069982 \| USD \| per_test \| Lower is better \| \| `latency` \| 2251.578 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.92 pp sourcePopulation 81 stderr 1.922 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label deepseek/deepseek-v4-flash-0731 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__bankruptcy_restructuring/deepseek~1deepseek-v4-flash-0731` | | | | | | | | | [CorpFin v2 / shared max context](https://benchize.com/benchmarks/vals-ai%3Acorp_fin_v2%3Ashared_max_context.md) | 62.01% | SE ±1.66 pp | 24 / 39 | $0.00943 | 17.43source duration | [Vals AI · 2](https://www.vals.ai/_astro/benchmark_view_corp_fin_v2.CectKwl1.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 62.005 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.009434 \| USD \| per_test \| Lower is better \| \| `latency` \| 17.433 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.66 pp sourcePopulation 134 stderr 1.657 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_corp_fin_v2.CectKwl1.json) Source Vals AI Original model label deepseek/deepseek-v4-flash-0731 Source version 2 Observed 2026-10-21T16:45:57.998Z Evidence SHA-256 `7cf2217ab3728d5a84bffce82aab9fc52d7f810b0cbe8c65b387df47f3d31c9b` Source locator `/tasks/shared_max_context/deepseek~1deepseek-v4-flash-0731` | | | | | | | | | [Tax Agent Bench / rule source lookup](https://benchize.com/benchmarks/vals-ai%3Atax_agent_bench%3Arule_source_lookup.md) | 42.59% | SE ±8.63 pp | 13 / 46 | $0.16 | 339.09source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 42.59 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.163641 \| USD \| per_test \| Lower is better \| \| `latency` \| 339.091 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±8.63 pp sourcePopulation 73 stderr 8.625 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) Source Vals AI Original model label deepseek/deepseek-v4-flash-0731 Source version 1 Observed 2026-10-21T16:45:55.958Z Evidence SHA-256 `446db182b67efb7923b3187a4040ec3a3ea492b8ad699a2b6d708d714cc063ac` Source locator `/tasks/rule_source_lookup/deepseek~1deepseek-v4-flash-0731` | | | | | | | | | [Tax Agent Bench / weighted pass fact pattern analysis](https://benchize.com/benchmarks/vals-ai%3Atax_agent_bench%3Aweighted_pass__fact_pattern_analysis.md) | 54.74% | SE ±6.99 pp | 30 / 46 | $0.15 | 348.29source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 54.742 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.147096 \| USD \| per_test \| Lower is better \| \| `latency` \| 348.287 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±6.99 pp sourcePopulation 73 stderr 6.993 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) Source Vals AI Original model label deepseek/deepseek-v4-flash-0731 Source version 1 Observed 2026-10-21T16:45:55.958Z Evidence SHA-256 `446db182b67efb7923b3187a4040ec3a3ea492b8ad699a2b6d708d714cc063ac` Source locator `/tasks/weighted_pass__fact_pattern_analysis/deepseek~1deepseek-v4-flash-0731` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / international trade sanctions](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Ainternational_trade_sanctions.md) | 0% | SE ±0 pp | 4 / 49 | $0.04 | 643.49source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.03909 \| USD \| per_test \| Lower is better \| \| `latency` \| 643.488 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 81 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label deepseek/deepseek-v4-flash-0731 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/international_trade_sanctions/deepseek~1deepseek-v4-flash-0731` | | | | | | | | | [Terminal-Bench-Science / overall](https://benchize.com/benchmarks/vals-ai%3Aterminal-bench-science%3Aoverall.md) | 4.29% | SE ±2.44 pp | 18 / 30 | $1.1 | 5,653.64source duration | [Vals AI · 0.1](https://www.vals.ai/_astro/benchmark_view_terminal_bench_science.o-HR8Nyt.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 4.286 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 1.095543 \| USD \| per_test \| Lower is better \| \| `latency` \| 5653.642 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Mini-SWE-agent maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.44 pp sourcePopulation 41 stderr 2.438 tokenTotals {"cache_read_tokens":3620612352,"cache_write_tokens":0,"input_tokens":7249217395,"output_tokens":29172191,"reasoning_tokens":12139861} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_terminal_bench_science.o-HR8Nyt.json) Source Vals AI Original model label deepseek/deepseek-v4-flash-0731 Source version 0.1 Observed 2026-10-21T16:46:03.529Z Evidence SHA-256 `a30a8618deaecf018ea1c8c9f675576cdd9ab79742d05788067db49afcc5f91b` Source locator `/tasks/overall/deepseek~1deepseek-v4-flash-0731` | | | | | | | | | [MMLU Pro / law](https://benchize.com/benchmarks/vals-ai%3Ammlu_pro%3Alaw.md) | 72.21% | SE ±1.35 pp | 26 / 40 | $0.00377 | 27.8source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_mmlu_pro.y416pXDi.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 72.207 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.003767 \| USD \| per_test \| Lower is better \| \| `latency` \| 27.801 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.35 pp sourcePopulation 138 stderr 1.35 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_mmlu_pro.y416pXDi.json) Source Vals AI Original model label deepseek/deepseek-v4-flash-0731 Source version 1 Observed 2026-10-21T16:46:06.503Z Evidence SHA-256 `dab025507f46c716dd8d145e90ec0a737d9c56219853f41e4b1cc3c5e8c222b4` Source locator `/tasks/law/deepseek~1deepseek-v4-flash-0731` | | | | | | | | | [GPQA Diamond / diamond zero shot cot](https://benchize.com/benchmarks/vals-ai%3Agpqa%3Adiamond_zero_shot_cot.md) | 89.9% | SE ±2.55 pp | 18 / 40 | $0.01 | 79.8source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_gpqa.ghZZBpyU.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 89.899 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.012785 \| USD \| per_test \| Lower is better \| \| `latency` \| 79.801 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.55 pp sourcePopulation 138 stderr 2.548 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_gpqa.ghZZBpyU.json) Source Vals AI Original model label deepseek/deepseek-v4-flash-0731 Source version 1 Observed 2026-10-21T16:46:06.040Z Evidence SHA-256 `951eb5224097733a9d09d06c14908ae2980e2e640e539f2c284877c5092d7077` Source locator `/tasks/diamond_zero_shot_cot/deepseek~1deepseek-v4-flash-0731` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / criteria pass rate environmental esg](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__environmental_esg.md) | 85.31% | SE ±2.12 pp | 15 / 49 | $0.05 | 723.79source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 85.305 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.048721 \| USD \| per_test \| Lower is better \| \| `latency` \| 723.794 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.12 pp sourcePopulation 81 stderr 2.124 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label deepseek/deepseek-v4-flash-0731 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__environmental_esg/deepseek~1deepseek-v4-flash-0731` | | | | | | | | | [Terminal-Bench 4.0 / overall](https://benchize.com/benchmarks/vals-ai%3Aterminal-bench-4%3Aoverall.md) | 18.69% | SE ±3.54 pp | 22 / 36 | $0.77 | 6,368.24source duration | [Vals AI · 4.0](https://www.vals.ai/_astro/benchmark_view_terminal_bench_4.CAnGyzC_.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 18.687 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.766472 \| USD \| per_test \| Lower is better \| \| `latency` \| 6368.24 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±3.54 pp sourcePopulation 52 stderr 3.535 tokenTotals {"cache_read_tokens":1966243579,"cache_write_tokens":0,"input_tokens":1973117830,"output_tokens":15178104,"reasoning_tokens":11818307} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_terminal_bench_4.CAnGyzC_.json) Source Vals AI Original model label deepseek/deepseek-v4-flash-0731 Source version 4.0 Observed 2026-10-21T16:46:10.642Z Evidence SHA-256 `98f6964580e901f1aa143db664bd92aa558415f643569492545d67c47d7f519b` Source locator `/tasks/overall/deepseek~1deepseek-v4-flash-0731` | | | | | | | | | [Terminal-Bench-Science / physical sciences](https://benchize.com/benchmarks/vals-ai%3Aterminal-bench-science%3Aphysical_sciences.md) | 11.77% | SE ±8.06 pp | 13 / 30 | — | — | [Vals AI · 0.1](https://www.vals.ai/_astro/benchmark_view_terminal_bench_science.o-HR8Nyt.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 11.765 \| percent \| reported \| Higher is better \| ### Configuration computeEffort Not reported harness Mini-SWE-agent maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±8.06 pp sourcePopulation 41 stderr 8.055 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_terminal_bench_science.o-HR8Nyt.json) Source Vals AI Original model label deepseek/deepseek-v4-flash-0731 Source version 0.1 Observed 2026-10-21T16:46:03.529Z Evidence SHA-256 `a30a8618deaecf018ea1c8c9f675576cdd9ab79742d05788067db49afcc5f91b` Source locator `/tasks/physical_sciences/deepseek~1deepseek-v4-flash-0731` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / funds asset management](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Afunds_asset_management.md) | 20% | SE ±20 pp | 3 / 49 | $0.08 | 985.74source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 20 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.077692 \| USD \| per_test \| Lower is better \| \| `latency` \| 985.742 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±20 pp sourcePopulation 81 stderr 20 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label deepseek/deepseek-v4-flash-0731 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/funds_asset_management/deepseek~1deepseek-v4-flash-0731` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / environmental esg](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Aenvironmental_esg.md) | 0% | SE ±0 pp | 1 / 49 | $0.05 | 723.79source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.048721 \| USD \| per_test \| Lower is better \| \| `latency` \| 723.794 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 81 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label deepseek/deepseek-v4-flash-0731 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/environmental_esg/deepseek~1deepseek-v4-flash-0731` | | | | | | | | [Next page](https://benchize.com/models/deepseek-v4-flash-0731-high.md?cursor=852e4612e11a730633603b0aa5ebc45ad3227bf62f9117480b445b6d11974349) --- Cite as: DeepSeek V4 Flash 0731High — Benchize, https://benchize.com/models/deepseek-v4-flash-0731-high (retrieved 2026-10-21)