# DeepSeek V4.1 FlashHigh > Source: Benchize (https://benchize.com) — https://benchize.com/models/deepseek-v4-1-flash-high. When citing or reusing this content, credit "Benchize" and link to https://benchize.com/models/deepseek-v4-1-flash-high. DeepSeek Canonical URL: https://benchize.com/models/deepseek-v4-1-flash-high Last updated: Wed, 21 Oct 2026 18:52:20 GMT Reproduction and quotation must name Benchize, link to the canonical HTML URL, and retain individual source citations. DeepSeek ## Benchmarks | Benchmark | Score | ± | Rank | Cost | Latency | Source | | | --- | --- | --- | --- | --- | --- | --- | --- | | [Terminal-Bench 4.0 / hardware](https://benchize.com/benchmarks/vals-ai%3Aterminal-bench-4%3Ahardware.md) | 0% | SE ±0 pp | 21 / 36 | — | — | [Vals AI · 4.0](https://www.vals.ai/_astro/benchmark_view_terminal_bench_4.CAnGyzC_.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 52 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_terminal_bench_4.CAnGyzC_.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash Source version 4.0 Observed 2026-10-21T16:46:10.642Z Evidence SHA-256 `98f6964580e901f1aa143db664bd92aa558415f643569492545d67c47d7f519b` Source locator `/tasks/hardware/deepseek~1deepseek-v4.1-flash` | | | | | | | | | [Vibe Code Bench 1-100 / overall](https://benchize.com/benchmarks/vals-ai%3Avcb-1-100%3Aoverall.md) | 16.38% | SE ±3.19 pp | 12 / 16 | $0.69 | 1,701.01source duration | [Vals AI · 1.0](https://www.vals.ai/_astro/benchmark_view_vcb_1_100.ZqWqHxGl.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 16.378 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.688342 \| USD \| per_test \| Lower is better \| \| `latency` \| 1701.009 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness OpenHands maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±3.19 pp sourcePopulation 25 stderr 3.194 tokenTotals {"cache_read_tokens":1518694655,"cache_write_tokens":null,"input_tokens":1618986589,"output_tokens":22945175,"reasoning_tokens":12528036} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_vcb_1_100.ZqWqHxGl.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash Source version 1.0 Observed 2026-10-21T16:46:09.131Z Evidence SHA-256 `4a3ebdcd9364ec3a21193d00de0bf723384d85187244abff859527fd3cc51891` Source locator `/tasks/overall/deepseek~1deepseek-v4.1-flash` | | | | | | | | | [EMB / overall](https://benchize.com/benchmarks/vals-ai%3Aemb%3Aoverall.md) | 57.21% | SE ±3.23 pp | 29 / 47 | $0.23 | 780.46source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 57.208 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.234152 \| USD \| per_test \| Lower is better \| \| `latency` \| 780.458 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±3.23 pp sourcePopulation 76 stderr 3.227 tokenTotals {"cache_read_tokens":884618493,"cache_write_tokens":null,"input_tokens":893420650,"output_tokens":13279281,"reasoning_tokens":10184478} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash Source version 1 Observed 2026-10-21T16:45:57.495Z Evidence SHA-256 `d807aee620ea47152a1fca0257a8dd12a1be4512c76cc1744c1ae9e285180918` Source locator `/tasks/overall/deepseek~1deepseek-v4.1-flash` | | | | | | | | | [SAGE / overall](https://benchize.com/benchmarks/vals-ai%3Asage%3Aoverall.md) | 47.88% | SE ±3.44 pp | 20 / 41 | $0.01 | 35.78source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_sage.RRwIIzO5.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 47.88 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.01219 \| USD \| per_test \| Lower is better \| \| `latency` \| 35.776 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 30000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±3.44 pp sourcePopulation 91 stderr 3.438 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_sage.RRwIIzO5.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash Source version 1 Observed 2026-10-21T16:46:07.515Z Evidence SHA-256 `cc5fb3fecdb0382d492ff167a42ad84b9c5c3883373c78ed87baaa972b8b6f7c` Source locator `/tasks/overall/deepseek~1deepseek-v4.1-flash` | | | | | | | | | [LegalBench / overall](https://benchize.com/benchmarks/vals-ai%3Alegal_bench%3Aoverall.md) | 83.28% | SE ±0.46 pp | 24 / 44 | $0.00142 | 6.53source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_bench.D6uTgA3b.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 83.277 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.001419 \| USD \| per_test \| Lower is better \| \| `latency` \| 6.533 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.46 pp sourcePopulation 151 stderr 0.458 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_bench.D6uTgA3b.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash Source version 1 Observed 2026-10-21T16:45:53.390Z Evidence SHA-256 `c8486ba756baa5cd3955d035edff67440e8db23e69d62ad1a9e18d15b8c4bc52` Source locator `/tasks/overall/deepseek~1deepseek-v4.1-flash` | | | | | | | | | [Code Migration / code quality](https://benchize.com/benchmarks/vals-ai%3Acode-migration%3Acode_quality.md) | 4.41% | SE ±0.04 pp | 12 / 47 | $0.94 | 6,325.87source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_code_migration.Dttzoqk3.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 4.411 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.944694 \| USD \| per_test \| Lower is better \| \| `latency` \| 6325.865 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.04 pp sourcePopulation 79 stderr 0.035 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_code_migration.Dttzoqk3.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash Source version 1 Observed 2026-10-21T16:46:12.712Z Evidence SHA-256 `99194597f2c76c402195be47839409046e18bd30454616da6dd40b736a088fa4` Source locator `/tasks/code_quality/deepseek~1deepseek-v4.1-flash` | | | | | | | | | [MedScribe / overall](https://benchize.com/benchmarks/vals-ai%3Amedscribe%3Aoverall.md) | 85.5% | SE ±1.92 pp | 17 / 49 | $0.02 | 53.83source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_medscribe.BYf5a3C-.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 85.5 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.015397 \| USD \| per_test \| Lower is better \| \| `latency` \| 53.826 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.92 pp sourcePopulation 109 stderr 1.918 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_medscribe.BYf5a3C-.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash Source version 1 Observed 2026-10-21T16:46:00.004Z Evidence SHA-256 `271fc122ddc2b0602bd6c7947f65a9a7f0d8cab94f6dc27d62a9380a9317f9c0` Source locator `/tasks/overall/deepseek~1deepseek-v4.1-flash` | | | | | | | | | [Terminal-Bench 4.0 / operations](https://benchize.com/benchmarks/vals-ai%3Aterminal-bench-4%3Aoperations.md) | 18.52% | SE ±3.7 pp | 13 / 36 | — | — | [Vals AI · 4.0](https://www.vals.ai/_astro/benchmark_view_terminal_bench_4.CAnGyzC_.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 18.519 \| percent \| reported \| Higher is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±3.7 pp sourcePopulation 52 stderr 3.704 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_terminal_bench_4.CAnGyzC_.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash Source version 4.0 Observed 2026-10-21T16:46:10.642Z Evidence SHA-256 `98f6964580e901f1aa143db664bd92aa558415f643569492545d67c47d7f519b` Source locator `/tasks/operations/deepseek~1deepseek-v4.1-flash` | | | | | | | | | [LegalBench / conclusion tasks](https://benchize.com/benchmarks/vals-ai%3Alegal_bench%3Aconclusion_tasks.md) | 89.08% | SE ±0.78 pp | 19 / 44 | $0.00115 | 5.7source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_bench.D6uTgA3b.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 89.075 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.001151 \| USD \| per_test \| Lower is better \| \| `latency` \| 5.698 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.78 pp sourcePopulation 151 stderr 0.782 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_bench.D6uTgA3b.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash Source version 1 Observed 2026-10-21T16:45:53.390Z Evidence SHA-256 `c8486ba756baa5cd3955d035edff67440e8db23e69d62ad1a9e18d15b8c4bc52` Source locator `/tasks/conclusion_tasks/deepseek~1deepseek-v4.1-flash` | | | | | | | | | [SAGE / linear algebra](https://benchize.com/benchmarks/vals-ai%3Asage%3Alinear_algebra.md) | 62.29% | SE ±5.46 pp | 29 / 41 | $0.01 | 32.3source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_sage.RRwIIzO5.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 62.29 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.012613 \| USD \| per_test \| Lower is better \| \| `latency` \| 32.298 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 30000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±5.46 pp sourcePopulation 91 stderr 5.464 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_sage.RRwIIzO5.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash Source version 1 Observed 2026-10-21T16:46:07.515Z Evidence SHA-256 `cc5fb3fecdb0382d492ff167a42ad84b9c5c3883373c78ed87baaa972b8b6f7c` Source locator `/tasks/linear_algebra/deepseek~1deepseek-v4.1-flash` | | | | | | | | | [IOI / overall](https://benchize.com/benchmarks/vals-ai%3Aioi%3Aoverall.md) | 40.28% | SE ±2.56 pp | 23 / 29 | $0.24 | 1,109.58source duration | [Vals AI · 2](https://www.vals.ai/_astro/benchmark_view_ioi.CuUolFNv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 40.278 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.24255 \| USD \| per_test \| Lower is better \| \| `latency` \| 1109.577 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.56 pp sourcePopulation 44 stderr 2.557 tokenTotals {"cache_read_tokens":39069653,"cache_write_tokens":0,"input_tokens":39221134,"output_tokens":970177,"reasoning_tokens":853932} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_ioi.CuUolFNv.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash Source version 2 Observed 2026-10-21T16:46:11.757Z Evidence SHA-256 `e1545b21b5b5c98d037aefa9d441b65859d07a10550f242d0e16162a49b8e8ab` Source locator `/tasks/overall/deepseek~1deepseek-v4.1-flash` | | | | | | | | | [Code Migration / cobol](https://benchize.com/benchmarks/vals-ai%3Acode-migration%3Acobol.md) | 70% | SE ±15.28 pp | 1 / 47 | $0.33 | 3,456.84source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_code_migration.Dttzoqk3.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 70 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.328209 \| USD \| per_test \| Lower is better \| \| `latency` \| 3456.844 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±15.28 pp sourcePopulation 79 stderr 15.275 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_code_migration.Dttzoqk3.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash Source version 1 Observed 2026-10-21T16:46:12.712Z Evidence SHA-256 `99194597f2c76c402195be47839409046e18bd30454616da6dd40b736a088fa4` Source locator `/tasks/cobol/deepseek~1deepseek-v4.1-flash` | | | | | | | | | [SAGE / PDEs](https://benchize.com/benchmarks/vals-ai%3Asage%3APDEs.md) | 48.41% | SE ±6.27 pp | 3 / 41 | $0.01 | 43.48source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_sage.RRwIIzO5.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 48.413 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.014128 \| USD \| per_test \| Lower is better \| \| `latency` \| 43.479 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 30000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±6.27 pp sourcePopulation 91 stderr 6.271 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_sage.RRwIIzO5.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash Source version 1 Observed 2026-10-21T16:46:07.515Z Evidence SHA-256 `cc5fb3fecdb0382d492ff167a42ad84b9c5c3883373c78ed87baaa972b8b6f7c` Source locator `/tasks/PDEs/deepseek~1deepseek-v4.1-flash` | | | | | | | | | [ARC-AGI-1](https://benchize.com/benchmarks/arc-prize%3Av1_Semi_Private.md) | 88.5% | — | 69 / 213 | 0.06usd per task | — | [arc prize · v1](https://arcprize.org/media/data/leaderboard/v1.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 0.885 \| score_fraction \| v1_Semi_Private \| Higher is better \| \| `cost_per_task` \| 0.06328819800000002 \| usd_per_task \| v1_Semi_Private \| Lower is better \| ### Configuration evaluationSplit v1_Semi_Private harness Not reported modelGroup deepseek-v4-1-flash modelType CoT Reasoning effort high sourceModelVersion deepseek-v4-1-flash-high thinkingEnabled true ### Source [Original source](https://arcprize.org/media/data/leaderboard/v1.json) Source arc prize Original model label DeepSeek V4.1 Flash (High) Source version v1 Observed 2026-10-21T16:34:20.378Z Evidence SHA-256 `302d8267c0522bf450ec1deeac4d060e2cf2fb444ba0a4269014d9535de0bd1f` Source locator `/evaluations/263` | | | | | | | | | [Finance Agent (v2) / all pass](https://benchize.com/benchmarks/vals-ai%3Afabv2%3Aall_pass.md) | 41.05% | SE ±0.45 pp | 25 / 50 | $0.21 | 364.38source duration | [Vals AI · 2](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 41.052 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.207465 \| USD \| per_test \| Lower is better \| \| `latency` \| 364.376 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.45 pp sourcePopulation 80 stderr 0.446 tokenTotals {"cache_read_tokens":182250763,"cache_write_tokens":null,"input_tokens":354713236,"output_tokens":33772547,"reasoning_tokens":28648435} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash Source version 2 Observed 2026-10-21T16:45:57.035Z Evidence SHA-256 `f731b20a26c126725d125519f2b5841aa45254656dba859a9570d7ce7e5d8191` Source locator `/tasks/all_pass/deepseek~1deepseek-v4.1-flash` | | | | | | | | | [Finance Agent (v2) / earnings analysis](https://benchize.com/benchmarks/vals-ai%3Afabv2%3Aearnings_analysis.md) | 72.88% | SE ±2.85 pp | 25 / 50 | $0.08 | 244.53source duration | [Vals AI · 2](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 72.881 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.080051 \| USD \| per_test \| Lower is better \| \| `latency` \| 244.529 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.85 pp sourcePopulation 80 stderr 2.846 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash Source version 2 Observed 2026-10-21T16:45:57.035Z Evidence SHA-256 `f731b20a26c126725d125519f2b5841aa45254656dba859a9570d7ce7e5d8191` Source locator `/tasks/earnings_analysis/deepseek~1deepseek-v4.1-flash` | | | | | | | | | [EMB / scratch](https://benchize.com/benchmarks/vals-ai%3Aemb%3Ascratch.md) | 66.08% | SE ±2.63 pp | 23 / 47 | $0.21 | 714.92source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 66.083 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.211272 \| USD \| per_test \| Lower is better \| \| `latency` \| 714.918 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.63 pp sourcePopulation 76 stderr 2.627 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash Source version 1 Observed 2026-10-21T16:45:57.495Z Evidence SHA-256 `d807aee620ea47152a1fca0257a8dd12a1be4512c76cc1744c1ae9e285180918` Source locator `/tasks/scratch/deepseek~1deepseek-v4.1-flash` | | | | | | | | | [Finance Agent (v2) / overall](https://benchize.com/benchmarks/vals-ai%3Afabv2%3Aoverall.md) | 53.48% | SE ±0.39 pp | 26 / 50 | $0.21 | 364.38source duration | [Vals AI · 2](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 53.481 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.207465 \| USD \| per_test \| Lower is better \| \| `latency` \| 364.376 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.39 pp sourcePopulation 80 stderr 0.39 tokenTotals {"cache_read_tokens":182250763,"cache_write_tokens":null,"input_tokens":354713236,"output_tokens":33772547,"reasoning_tokens":28648435} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash Source version 2 Observed 2026-10-21T16:45:57.035Z Evidence SHA-256 `f731b20a26c126725d125519f2b5841aa45254656dba859a9570d7ce7e5d8191` Source locator `/tasks/overall/deepseek~1deepseek-v4.1-flash` | | | | | | | | | [IOI / ioi2025](https://benchize.com/benchmarks/vals-ai%3Aioi%3Aioi2025.md) | 35.17% | SE ±9.18 pp | 25 / 29 | $0.3 | 1,631.2source duration | [Vals AI · 2](https://www.vals.ai/_astro/benchmark_view_ioi.CuUolFNv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 35.167 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.297858 \| USD \| per_test \| Lower is better \| \| `latency` \| 1631.199 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±9.18 pp sourcePopulation 44 stderr 9.182 tokenTotals {"cache_read_tokens":51142912,"cache_write_tokens":0,"input_tokens":51333215,"output_tokens":1169612,"reasoning_tokens":1027911} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_ioi.CuUolFNv.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash Source version 2 Observed 2026-10-21T16:46:11.757Z Evidence SHA-256 `e1545b21b5b5c98d037aefa9d441b65859d07a10550f242d0e16162a49b8e8ab` Source locator `/tasks/ioi2025/deepseek~1deepseek-v4.1-flash` | | | | | | | | | [ToneBench Scriptwriting](https://benchize.com/benchmarks/tonebench%3Ascripts.md) | 1,958.6 | Range 1,920.1–1,999.6 | 40 / 126 | — | — | [tonebench](https://benchmark.towardsai.com/data/leaderboard.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `elo` \| 1958.6 \| rating \| reported \| Higher is better \| \| `overall` \| 80.35 \| percent \| reported \| Higher is better \| \| `anti_slop_numbers` \| 71.06 \| source_score \| rubric \| Higher is better \| \| `continuity_emotion` \| 80.13 \| source_score \| rubric \| Higher is better \| \| `cue_quality` \| 74.64 \| source_score \| rubric \| Higher is better \| \| `hook_strength` \| 80.04 \| source_score \| rubric \| Higher is better \| \| `length_adherence` \| 93.14 \| source_score \| rubric \| Higher is better \| \| `slop` \| 83.31 \| source_score \| rubric \| Higher is better \| \| `slop_score` \| 95.58 \| source_score \| rubric \| Higher is better \| \| `substance_accuracy` \| 80.19 \| source_score \| rubric \| Higher is better \| \| `tone_voice` \| 77.78 \| source_score \| rubric \| Higher is better \| \| `writing_quality` \| 78.22 \| source_score \| rubric \| Higher is better \| \| `youtube_best_practices` \| 79.56 \| source_score \| rubric \| Higher is better \| ### Configuration provider openrouter Reasoning effort high scoreScope 10_task_current sourceConfiguration deepseek-v4-1-flash-high-or thinkingEnabled true ### Reported statistics Uncertainty Reported interval: 1,920.1–1,999.6 interval [1920.1,1999.6] rank 40 runCount 5 ### Source [Original source](https://benchmark.towardsai.com/data/leaderboard.json) Source tonebench Original model label DeepSeek V4.1 Flash (high) Evaluated 2026-10-08 Observed 2026-10-21T16:32:48.862Z Evidence SHA-256 `4264338337f418d3ded7ace250cad64fe12cacbd4b79a312bfaf0ae9268a01c3` Source locator `/rows/39` | | | | | | | | | [Terminal-Bench-Science / physical sciences](https://benchize.com/benchmarks/vals-ai%3Aterminal-bench-science%3Aphysical_sciences.md) | 0% | SE ±0 pp | 24 / 30 | — | — | [Vals AI · 0.1](https://www.vals.ai/_astro/benchmark_view_terminal_bench_science.o-HR8Nyt.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| ### Configuration computeEffort Not reported harness Mini-SWE-agent maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 41 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_terminal_bench_science.o-HR8Nyt.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash Source version 0.1 Observed 2026-10-21T16:46:03.529Z Evidence SHA-256 `a30a8618deaecf018ea1c8c9f675576cdd9ab79742d05788067db49afcc5f91b` Source locator `/tasks/physical_sciences/deepseek~1deepseek-v4.1-flash` | | | | | | | | | [Terminal-Bench 4.0 / science](https://benchize.com/benchmarks/vals-ai%3Aterminal-bench-4%3Ascience.md) | 14.29% | SE ±4.12 pp | 24 / 36 | — | — | [Vals AI · 4.0](https://www.vals.ai/_astro/benchmark_view_terminal_bench_4.CAnGyzC_.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 14.286 \| percent \| reported \| Higher is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±4.12 pp sourcePopulation 52 stderr 4.124 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_terminal_bench_4.CAnGyzC_.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash Source version 4.0 Observed 2026-10-21T16:46:10.642Z Evidence SHA-256 `98f6964580e901f1aa143db664bd92aa558415f643569492545d67c47d7f519b` Source locator `/tasks/science/deepseek~1deepseek-v4.1-flash` | | | | | | | | | [LegalBench / rhetoric tasks](https://benchize.com/benchmarks/vals-ai%3Alegal_bench%3Arhetoric_tasks.md) | 80.13% | SE ±1.17 pp | 25 / 44 | $0.000611 | 3.18source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_bench.D6uTgA3b.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 80.127 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.000611 \| USD \| per_test \| Lower is better \| \| `latency` \| 3.183 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.17 pp sourcePopulation 151 stderr 1.169 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_bench.D6uTgA3b.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash Source version 1 Observed 2026-10-21T16:45:53.390Z Evidence SHA-256 `c8486ba756baa5cd3955d035edff67440e8db23e69d62ad1a9e18d15b8c4bc52` Source locator `/tasks/rhetoric_tasks/deepseek~1deepseek-v4.1-flash` | | | | | | | | | [SkillsBench / no skills](https://benchize.com/benchmarks/vals-ai%3Askillsbench%3Ano_skills.md) | 61.66% | SE ±3.87 pp | 1 / 20 | $0.09 | 249.38source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_skillsbench.BE2TXB-M.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 61.658 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.092261 \| USD \| per_test \| Lower is better \| \| `latency` \| 249.381 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness OpenHands maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±3.87 pp sourcePopulation 35 stderr 3.871 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_skillsbench.BE2TXB-M.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash Source version 1 Observed 2026-10-21T16:46:11.147Z Evidence SHA-256 `aaad242949d81b7c7c61c36d4895d4460e9146760c1e6b4b38d235a16043bc61` Source locator `/tasks/no_skills/deepseek~1deepseek-v4.1-flash` | | | | | | | | | [FrontierCode 1.1 extended](https://benchize.com/benchmarks/frontiercode%3Av1_1%3Aextended.md) | 56.62% | — | 53 / 125 | $0.5 | — | [frontiercode](https://cognition.com/data/frontiercode-leaderboard/data.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 0.5662 \| score_fraction \| blocking_criteria_adjusted \| Higher is better \| \| `correct` \| 0.6203 \| score_fraction \| raw_rubric \| Higher is better \| \| `cost` \| 0.4999 \| USD \| per_rollout \| Lower is better \| ### Configuration harness chisel Reasoning effort high sourceVersion v1_1 subset extended ### Reported statistics flaggedRate 0.0049 sampleSize 150 ### Source [Original source](https://cognition.com/data/frontiercode-leaderboard/data.json) Source frontiercode Original model label DeepSeek V4.1 Flash Observed 2026-10-21T16:31:00.878Z Evidence SHA-256 `ff0040778b7bdf69c705ad7474f771fb5e53cfaf5cf8573b3a4614153543d75f` Source locator `/v1_1/data/DeepSeek%20V4.1%20Flash/high/extended` | | | | | | | | | [Terminal-Bench 2.1 / medium](https://benchize.com/benchmarks/vals-ai%3Aterminal-bench-2-1%3Amedium.md) | 83.03% | SE ±1.21 pp | 12 / 43 | $0.16 | 978.2source duration | [Vals AI · 2.1](https://www.vals.ai/_astro/benchmark_view_terminal_bench_2_1.CdyoObsA.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 83.03 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.160001 \| USD \| per_test \| Lower is better \| \| `latency` \| 978.201 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.21 pp sourcePopulation 76 stderr 1.212 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_terminal_bench_2_1.CdyoObsA.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash Source version 2.1 Observed 2026-10-21T16:46:13.216Z Evidence SHA-256 `a5306805cdf86495bdcceff6b2c8534a3768b9853f5077895dab975b809b054f` Source locator `/tasks/medium/deepseek~1deepseek-v4.1-flash` | | | | | | | | | [Terminal-Bench-Science / mathematical sciences](https://benchize.com/benchmarks/vals-ai%3Aterminal-bench-science%3Amathematical_sciences.md) | 0% | SE ±0 pp | 19 / 30 | — | — | [Vals AI · 0.1](https://www.vals.ai/_astro/benchmark_view_terminal_bench_science.o-HR8Nyt.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| ### Configuration computeEffort Not reported harness Mini-SWE-agent maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 41 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_terminal_bench_science.o-HR8Nyt.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash Source version 0.1 Observed 2026-10-21T16:46:03.529Z Evidence SHA-256 `a30a8618deaecf018ea1c8c9f675576cdd9ab79742d05788067db49afcc5f91b` Source locator `/tasks/mathematical_sciences/deepseek~1deepseek-v4.1-flash` | | | | | | | | | [CyberBench v1.1 / overall](https://benchize.com/benchmarks/benchmark%3Acyber%3Aoverall.md) | 73.69% | SE ±5.48 pp | 8 / 28 | $0.07 | 830.11source duration | [Vals AI · 1.1](https://www.vals.ai/_astro/benchmark_view_cyberbench.DS6UOlu0.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 73.69 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.069681 \| USD \| per_test \| Lower is better \| \| `latency` \| 830.108 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±5.48 pp sourcePopulation 47 stderr 5.476 tokenTotals {"cache_read_tokens":390720704,"cache_write_tokens":null,"input_tokens":395046929,"output_tokens":2123954,"reasoning_tokens":1603878} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_cyberbench.DS6UOlu0.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash Source version 1.1 Observed 2026-10-21T16:46:15.238Z Evidence SHA-256 `943f593f929f8971a6b746269a2851b5febc2aad4e2f1066895f715033cdc7f1` Source locator `/tasks/overall/deepseek~1deepseek-v4.1-flash` | | | | | | | | | [MedCode / overall](https://benchize.com/benchmarks/vals-ai%3Amedcode%3Aoverall.md) | 41.17% | SE ±2.04 pp | 34 / 49 | $0.01 | 35.13source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_medcode.DehpEDgG.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 41.173 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.01245 \| USD \| per_test \| Lower is better \| \| `latency` \| 35.126 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.04 pp sourcePopulation 107 stderr 2.042 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_medcode.DehpEDgG.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash Source version 1 Observed 2026-10-21T16:45:59.516Z Evidence SHA-256 `988a6ae1e07e33ce031572ed4a804be11c36ee97a1dd75cbe4740fff3b534f8c` Source locator `/tasks/overall/deepseek~1deepseek-v4.1-flash` | | | | | | | | | [Finance Agent (v2) / comparables](https://benchize.com/benchmarks/vals-ai%3Afabv2%3Acomparables.md) | 46.05% | SE ±1.72 pp | 19 / 50 | $0.28 | 538.68source duration | [Vals AI · 2](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 46.054 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.282181 \| USD \| per_test \| Lower is better \| \| `latency` \| 538.676 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.72 pp sourcePopulation 80 stderr 1.72 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash Source version 2 Observed 2026-10-21T16:45:57.035Z Evidence SHA-256 `f731b20a26c126725d125519f2b5841aa45254656dba859a9570d7ce7e5d8191` Source locator `/tasks/comparables/deepseek~1deepseek-v4.1-flash` | | | | | | | | | [Short Story Writing — Pairwise Comparison](https://benchize.com/benchmarks/lechmazur-short-story%3Apairwise.md) | -0.49 | 95% CI -0.61–-0.35 | 34 / 50 | — | — | [lechmazur short story · 8ba548d1cffefa5c472d3832e05f64e119a88fa4](https://api.github.com/repos/lechmazur/writing/contents/data/writing_gemini_argon_20261013/ratings.csv?ref=8ba548d1cffefa5c472d3832e05f64e119a88fa4) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `thurstone_rating` \| -0.48788265529811664 \| index \| pooled_observed_panel \| Higher is better \| \| `win_probability` \| 0.4254277762172336 \| score_fraction \| comparison_pool \| Higher is better \| ### Configuration benchmarkScope data/writing_gemini_argon_20261013 Reasoning effort high thinkingEnabled true ### Reported statistics Uncertainty 95% confidence interval: -0.61–-0.35 bootstrapSamples 300 confidenceInterval95 ["-0.6105971728903827","-0.35266875687308785"] sourceRank 34 ### Source [Original source](https://api.github.com/repos/lechmazur/writing/contents/data/writing_gemini_argon_20261013/ratings.csv?ref=8ba548d1cffefa5c472d3832e05f64e119a88fa4) Source lechmazur short story Original model label DeepSeek V4.1 Flash (high) Source version 8ba548d1cffefa5c472d3832e05f64e119a88fa4 Observed 2026-10-21T16:34:21.130Z Evidence SHA-256 `c36baebed9f302ff7ad8fe97907d72e1757a015a39b6a3b23437597cf8e3bff6` Source locator `/rows/33` | | | | | | | | | [Finance Agent (v2) / precedents](https://benchize.com/benchmarks/vals-ai%3Afabv2%3Aprecedents.md) | 28.22% | SE ±3.6 pp | 20 / 50 | $0.31 | 563.78source duration | [Vals AI · 2](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 28.219 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.306545 \| USD \| per_test \| Lower is better \| \| `latency` \| 563.78 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±3.6 pp sourcePopulation 80 stderr 3.6 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash Source version 2 Observed 2026-10-21T16:45:57.035Z Evidence SHA-256 `f731b20a26c126725d125519f2b5841aa45254656dba859a9570d7ce7e5d8191` Source locator `/tasks/precedents/deepseek~1deepseek-v4.1-flash` | | | | | | | | | [Terminal-Bench-Science / life sciences](https://benchize.com/benchmarks/vals-ai%3Aterminal-bench-science%3Alife_sciences.md) | 0% | SE ±0 pp | 24 / 30 | — | — | [Vals AI · 0.1](https://www.vals.ai/_astro/benchmark_view_terminal_bench_science.o-HR8Nyt.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| ### Configuration computeEffort Not reported harness Mini-SWE-agent maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 41 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_terminal_bench_science.o-HR8Nyt.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash Source version 0.1 Observed 2026-10-21T16:46:03.529Z Evidence SHA-256 `a30a8618deaecf018ea1c8c9f675576cdd9ab79742d05788067db49afcc5f91b` Source locator `/tasks/life_sciences/deepseek~1deepseek-v4.1-flash` | | | | | | | | | [Terminal-Bench 4.0 / security](https://benchize.com/benchmarks/vals-ai%3Aterminal-bench-4%3Asecurity.md) | 6.67% | SE ±6.67 pp | 24 / 36 | — | — | [Vals AI · 4.0](https://www.vals.ai/_astro/benchmark_view_terminal_bench_4.CAnGyzC_.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 6.667 \| percent \| reported \| Higher is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±6.67 pp sourcePopulation 52 stderr 6.667 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_terminal_bench_4.CAnGyzC_.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash Source version 4.0 Observed 2026-10-21T16:46:10.642Z Evidence SHA-256 `98f6964580e901f1aa143db664bd92aa558415f643569492545d67c47d7f519b` Source locator `/tasks/security/deepseek~1deepseek-v4.1-flash` | | | | | | | | | [Terminal-Bench-Science / engineering sciences](https://benchize.com/benchmarks/vals-ai%3Aterminal-bench-science%3Aengineering_sciences.md) | 0% | SE ±0 pp | 12 / 30 | — | — | [Vals AI · 0.1](https://www.vals.ai/_astro/benchmark_view_terminal_bench_science.o-HR8Nyt.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| ### Configuration computeEffort Not reported harness Mini-SWE-agent maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 41 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_terminal_bench_science.o-HR8Nyt.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash Source version 0.1 Observed 2026-10-21T16:46:03.529Z Evidence SHA-256 `a30a8618deaecf018ea1c8c9f675576cdd9ab79742d05788067db49afcc5f91b` Source locator `/tasks/engineering_sciences/deepseek~1deepseek-v4.1-flash` | | | | | | | | | [Terminal-Bench-Science / earth sciences](https://benchize.com/benchmarks/vals-ai%3Aterminal-bench-science%3Aearth_sciences.md) | 0% | SE ±0 pp | 14 / 30 | — | — | [Vals AI · 0.1](https://www.vals.ai/_astro/benchmark_view_terminal_bench_science.o-HR8Nyt.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| ### Configuration computeEffort Not reported harness Mini-SWE-agent maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 41 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_terminal_bench_science.o-HR8Nyt.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash Source version 0.1 Observed 2026-10-21T16:46:03.529Z Evidence SHA-256 `a30a8618deaecf018ea1c8c9f675576cdd9ab79742d05788067db49afcc5f91b` Source locator `/tasks/earth_sciences/deepseek~1deepseek-v4.1-flash` | | | | | | | | | [IOI / ioi2026](https://benchize.com/benchmarks/vals-ai%3Aioi%3Aioi2026.md) | 43% | SE ±16.93 pp | 18 / 29 | $0.22 | 782.97source duration | [Vals AI · 2](https://www.vals.ai/_astro/benchmark_view_ioi.CuUolFNv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 43 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.222012 \| USD \| per_test \| Lower is better \| \| `latency` \| 782.974 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±16.93 pp sourcePopulation 44 stderr 16.931 tokenTotals {"cache_read_tokens":38204416,"cache_write_tokens":0,"input_tokens":38321706,"output_tokens":884107,"reasoning_tokens":775032} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_ioi.CuUolFNv.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash Source version 2 Observed 2026-10-21T16:46:11.757Z Evidence SHA-256 `e1545b21b5b5c98d037aefa9d441b65859d07a10550f242d0e16162a49b8e8ab` Source locator `/tasks/ioi2026/deepseek~1deepseek-v4.1-flash` | | | | | | | | | [SkillsBench / with skills](https://benchize.com/benchmarks/vals-ai%3Askillsbench%3Awith_skills.md) | 69.8% | SE ±3.88 pp | 1 / 20 | $0.09 | 223.45source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_skillsbench.BE2TXB-M.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 69.802 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.085117 \| USD \| per_test \| Lower is better \| \| `latency` \| 223.451 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness OpenHands maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±3.88 pp sourcePopulation 35 stderr 3.879 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_skillsbench.BE2TXB-M.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash Source version 1 Observed 2026-10-21T16:46:11.147Z Evidence SHA-256 `aaad242949d81b7c7c61c36d4895d4460e9146760c1e6b4b38d235a16043bc61` Source locator `/tasks/with_skills/deepseek~1deepseek-v4.1-flash` | | | | | | | | | [Vals Science Index / biomysterybench](https://benchize.com/benchmarks/vals-ai%3Ascience_index%3Abiomysterybench.md) | 67.78% | SE ±1.11 pp | 11 / 17 | $0.23 | 890.07source duration | [Vals AI · 1.0](https://www.vals.ai/_astro/benchmark_view_science_index.CN7FN-f1.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 67.778 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.233183 \| USD \| per_test \| Lower is better \| \| `latency` \| 890.067 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.11 pp sourcePopulation 21 stderr 1.111 tokenTotals {"cache_read_tokens":262450176,"cache_write_tokens":0,"input_tokens":268679692,"output_tokens":14743987,"reasoning_tokens":13033344} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_science_index.CN7FN-f1.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash Source version 1.0 Observed 2026-10-21T16:45:50.879Z Evidence SHA-256 `fa4313008e91752fcfc185aaa78665f780d0fc23d7c7198a9853dbf2eaaaef1f` Source locator `/tasks/biomysterybench/deepseek~1deepseek-v4.1-flash` | | | | | | | | | [WeirdML v3](https://benchize.com/benchmarks/weirdml%3Av3.md) | 0.06source score | Range 0.04–0.08 | 20 / 25 | — | — | [weirdml · b0120fe](https://htihle.github.io/assets/data/weirdml_v3.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 0.05946331818181818 \| source_score \| reported \| Higher is better \| \| `mean_final_best` \| 0.11577238636363636 \| source_score \| reported \| Higher is better \| ### Configuration budget {"kind":"equivalent_tokens","limit":50000000,"start":500000} harness opencode Reasoning effort high sourceConfiguration deepseek-v4.1-flash-novita-high-opencode ### Reported statistics Uncertainty Reported interval: 0.04–0.08 interval [0.04190714269423148,0.07690865385214844] intervalMethod {"assumptions":"Independent fresh runs; approximately normal scores; shared variance prior. Pool fitted within analysis mode, including incomplete models; pool estimation uncertainty is not propagated.","confidence":0.95,"draws":10000,"fixed_tasks":true,"method":"partially_pooled_variance","pool":{"degrees_of_freedom":1533,"fallback":false,"prior_df":4,"variance":0.0026383113392869583},"seed":20260915,"unit":"run"} runCount 30 taskCount 11 ### Source [Original source](https://htihle.github.io/assets/data/weirdml_v3.json) Source weirdml Original model label DeepSeek V4.1 Flash (high, Novita) Source version b0120fe Observed 2026-10-21T16:33:04.394Z Evidence SHA-256 `82c3a8de423b23ca0af9761429299a2f21edb0b819e947fdc863f8b5ea4a49b8` Source locator `/rows/19` | | | | | | | | | [Finance Agent (v2) / market analysis](https://benchize.com/benchmarks/vals-ai%3Afabv2%3Amarket_analysis.md) | 63.36% | SE ±1.61 pp | 23 / 50 | $0.1 | 273.68source duration | [Vals AI · 2](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 63.361 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.096754 \| USD \| per_test \| Lower is better \| \| `latency` \| 273.681 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.61 pp sourcePopulation 80 stderr 1.609 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash Source version 2 Observed 2026-10-21T16:45:57.035Z Evidence SHA-256 `f731b20a26c126725d125519f2b5841aa45254656dba859a9570d7ce7e5d8191` Source locator `/tasks/market_analysis/deepseek~1deepseek-v4.1-flash` | | | | | | | | | [EMB / lbo](https://benchize.com/benchmarks/vals-ai%3Aemb%3Albo.md) | 54.18% | SE ±5.29 pp | 24 / 47 | $0.34 | 1,116.96source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 54.183 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.338915 \| USD \| per_test \| Lower is better \| \| `latency` \| 1116.958 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±5.29 pp sourcePopulation 76 stderr 5.292 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash Source version 1 Observed 2026-10-21T16:45:57.495Z Evidence SHA-256 `d807aee620ea47152a1fca0257a8dd12a1be4512c76cc1744c1ae9e285180918` Source locator `/tasks/lbo/deepseek~1deepseek-v4.1-flash` | | | | | | | | | [Terminal-Bench-Science / overall](https://benchize.com/benchmarks/vals-ai%3Aterminal-bench-science%3Aoverall.md) | 0% | SE ±0 pp | 28 / 30 | $0.47 | 10,621.57source duration | [Vals AI · 0.1](https://www.vals.ai/_astro/benchmark_view_terminal_bench_science.o-HR8Nyt.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.473154 \| USD \| per_test \| Lower is better \| \| `latency` \| 10621.573 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Mini-SWE-agent maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 41 stderr 0 tokenTotals {"cache_read_tokens":1759193848,"cache_write_tokens":0,"input_tokens":3524356137,"output_tokens":30097994,"reasoning_tokens":12785452} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_terminal_bench_science.o-HR8Nyt.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash Source version 0.1 Observed 2026-10-21T16:46:03.529Z Evidence SHA-256 `a30a8618deaecf018ea1c8c9f675576cdd9ab79742d05788067db49afcc5f91b` Source locator `/tasks/overall/deepseek~1deepseek-v4.1-flash` | | | | | | | | | [SAGE / calculus](https://benchize.com/benchmarks/vals-ai%3Asage%3Acalculus.md) | 33.33% | SE ±6.23 pp | 30 / 36 | $0.01 | 20.8source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_sage.RRwIIzO5.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 33.333 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.011459 \| USD \| per_test \| Lower is better \| \| `latency` \| 20.801 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 30000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±6.23 pp sourcePopulation 91 stderr 6.225 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_sage.RRwIIzO5.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash Source version 1 Observed 2026-10-21T16:46:07.515Z Evidence SHA-256 `cc5fb3fecdb0382d492ff167a42ad84b9c5c3883373c78ed87baaa972b8b6f7c` Source locator `/tasks/calculus/deepseek~1deepseek-v4.1-flash` | | | | | | | | | [Vals Science Index / terminal bench science](https://benchize.com/benchmarks/vals-ai%3Ascience_index%3Aterminal_bench_science.md) | 0% | SE ±0 pp | 16 / 17 | $0.47 | 10,621.57source duration | [Vals AI · 1.0](https://www.vals.ai/_astro/benchmark_view_science_index.CN7FN-f1.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.473154 \| USD \| per_test \| Lower is better \| \| `latency` \| 10621.573 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 21 stderr 0 tokenTotals {"cache_read_tokens":1759193848,"cache_write_tokens":0,"input_tokens":3524356137,"output_tokens":30097994,"reasoning_tokens":12785452} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_science_index.CN7FN-f1.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash Source version 1.0 Observed 2026-10-21T16:45:50.879Z Evidence SHA-256 `fa4313008e91752fcfc185aaa78665f780d0fc23d7c7198a9853dbf2eaaaef1f` Source locator `/tasks/terminal_bench_science/deepseek~1deepseek-v4.1-flash` | | | | | | | | | [EMB / misc](https://benchize.com/benchmarks/vals-ai%3Aemb%3Amisc.md) | 53.64% | SE ±7.97 pp | 29 / 47 | $0.24 | 833.5source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 53.636 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.238434 \| USD \| per_test \| Lower is better \| \| `latency` \| 833.498 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±7.97 pp sourcePopulation 76 stderr 7.974 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash Source version 1 Observed 2026-10-21T16:45:57.495Z Evidence SHA-256 `d807aee620ea47152a1fca0257a8dd12a1be4512c76cc1744c1ae9e285180918` Source locator `/tasks/misc/deepseek~1deepseek-v4.1-flash` | | | | | | | | | [EMB / dataroom](https://benchize.com/benchmarks/vals-ai%3Aemb%3Adataroom.md) | 73.86% | SE ±8.05 pp | 33 / 47 | $0.2 | 695.25source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 73.863 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.196161 \| USD \| per_test \| Lower is better \| \| `latency` \| 695.253 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±8.05 pp sourcePopulation 76 stderr 8.047 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash Source version 1 Observed 2026-10-21T16:45:57.495Z Evidence SHA-256 `d807aee620ea47152a1fca0257a8dd12a1be4512c76cc1744c1ae9e285180918` Source locator `/tasks/dataroom/deepseek~1deepseek-v4.1-flash` | | | | | | | | | [SkillsBench / overall](https://benchize.com/benchmarks/vals-ai%3Askillsbench%3Aoverall.md) | 69.8% | SE ±3.88 pp | 1 / 20 | $0.09 | 223.45source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_skillsbench.BE2TXB-M.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 69.802 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.085117 \| USD \| per_test \| Lower is better \| \| `latency` \| 223.451 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness OpenHands maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±3.88 pp sourcePopulation 35 stderr 3.879 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_skillsbench.BE2TXB-M.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash Source version 1 Observed 2026-10-21T16:46:11.147Z Evidence SHA-256 `aaad242949d81b7c7c61c36d4895d4460e9146760c1e6b4b38d235a16043bc61` Source locator `/tasks/overall/deepseek~1deepseek-v4.1-flash` | | | | | | | | | [Terminal-Bench 4.0 / ml](https://benchize.com/benchmarks/vals-ai%3Aterminal-bench-4%3Aml.md) | 24.24% | SE ±6.06 pp | 18 / 36 | — | — | [Vals AI · 4.0](https://www.vals.ai/_astro/benchmark_view_terminal_bench_4.CAnGyzC_.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 24.242 \| percent \| reported \| Higher is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±6.06 pp sourcePopulation 52 stderr 6.061 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_terminal_bench_4.CAnGyzC_.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash Source version 4.0 Observed 2026-10-21T16:46:10.642Z Evidence SHA-256 `98f6964580e901f1aa143db664bd92aa558415f643569492545d67c47d7f519b` Source locator `/tasks/ml/deepseek~1deepseek-v4.1-flash` | | | | | | | | | [Code Migration / cli](https://benchize.com/benchmarks/vals-ai%3Acode-migration%3Acli.md) | 37.5% | SE ±2.59 pp | 14 / 47 | $1.15 | 7,282.21source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_code_migration.Dttzoqk3.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 37.497 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 1.150189 \| USD \| per_test \| Lower is better \| \| `latency` \| 7282.206 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.59 pp sourcePopulation 79 stderr 2.592 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_code_migration.Dttzoqk3.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash Source version 1 Observed 2026-10-21T16:46:12.712Z Evidence SHA-256 `99194597f2c76c402195be47839409046e18bd30454616da6dd40b736a088fa4` Source locator `/tasks/cli/deepseek~1deepseek-v4.1-flash` | | | | | | | | | [Terminal-Bench 2.1 / easy](https://benchize.com/benchmarks/vals-ai%3Aterminal-bench-2-1%3Aeasy.md) | 91.67% | SE ±8.33 pp | 15 / 43 | $2.2 | 13,450.26source duration | [Vals AI · 2.1](https://www.vals.ai/_astro/benchmark_view_terminal_bench_2_1.CdyoObsA.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 91.667 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 2.200009 \| USD \| per_test \| Lower is better \| \| `latency` \| 13450.264 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±8.33 pp sourcePopulation 76 stderr 8.333 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_terminal_bench_2_1.CdyoObsA.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash Source version 2.1 Observed 2026-10-21T16:46:13.216Z Evidence SHA-256 `a5306805cdf86495bdcceff6b2c8534a3768b9853f5077895dab975b809b054f` Source locator `/tasks/easy/deepseek~1deepseek-v4.1-flash` | | | | | | | | | [SAGE / automata](https://benchize.com/benchmarks/vals-ai%3Asage%3Aautomata.md) | 32.94% | SE ±6.1 pp | 31 / 41 | $0.00983 | 31.55source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_sage.RRwIIzO5.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 32.936 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.00983 \| USD \| per_test \| Lower is better \| \| `latency` \| 31.551 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 30000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±6.1 pp sourcePopulation 91 stderr 6.098 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_sage.RRwIIzO5.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash Source version 1 Observed 2026-10-21T16:46:07.515Z Evidence SHA-256 `cc5fb3fecdb0382d492ff167a42ad84b9c5c3883373c78ed87baaa972b8b6f7c` Source locator `/tasks/automata/deepseek~1deepseek-v4.1-flash` | | | | | | | | | [Finance Agent (v2) / general qualitative analysis](https://benchize.com/benchmarks/vals-ai%3Afabv2%3Ageneral_qualitative_analysis.md) | 67.4% | SE ±2.37 pp | 17 / 50 | $0.23 | 400.53source duration | [Vals AI · 2](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 67.397 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.228616 \| USD \| per_test \| Lower is better \| \| `latency` \| 400.534 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.37 pp sourcePopulation 80 stderr 2.372 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash Source version 2 Observed 2026-10-21T16:45:57.035Z Evidence SHA-256 `f731b20a26c126725d125519f2b5841aa45254656dba859a9570d7ce7e5d8191` Source locator `/tasks/general_qualitative_analysis/deepseek~1deepseek-v4.1-flash` | | | | | | | | | [EMB / comps](https://benchize.com/benchmarks/vals-ai%3Aemb%3Acomps.md) | 43.93% | SE ±9.09 pp | 38 / 47 | $0.22 | 630.8source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 43.925 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.218978 \| USD \| per_test \| Lower is better \| \| `latency` \| 630.8 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±9.09 pp sourcePopulation 76 stderr 9.09 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash Source version 1 Observed 2026-10-21T16:45:57.495Z Evidence SHA-256 `d807aee620ea47152a1fca0257a8dd12a1be4512c76cc1744c1ae9e285180918` Source locator `/tasks/comps/deepseek~1deepseek-v4.1-flash` | | | | | | | | | [Finance Agent (v2) / financial modeling](https://benchize.com/benchmarks/vals-ai%3Afabv2%3Afinancial_modeling.md) | 23.05% | SE ±2.84 pp | 20 / 50 | $0.19 | 412.64source duration | [Vals AI · 2](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 23.052 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.192114 \| USD \| per_test \| Lower is better \| \| `latency` \| 412.644 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.84 pp sourcePopulation 80 stderr 2.835 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash Source version 2 Observed 2026-10-21T16:45:57.035Z Evidence SHA-256 `f731b20a26c126725d125519f2b5841aa45254656dba859a9570d7ce7e5d8191` Source locator `/tasks/financial_modeling/deepseek~1deepseek-v4.1-flash` | | | | | | | | | [ARC-AGI-2](https://benchize.com/benchmarks/arc-prize%3Av2_Semi_Private.md) | 67.5% | — | 56 / 212 | 0.13usd per task | — | [arc prize · v2](https://arcprize.org/media/data/leaderboard/v2.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 0.675 \| score_fraction \| v2_Semi_Private \| Higher is better \| \| `cost_per_task` \| 0.13341649249999996 \| usd_per_task \| v2_Semi_Private \| Lower is better \| ### Configuration evaluationSplit v2_Semi_Private harness Not reported modelGroup deepseek-v4-1-flash modelType CoT Reasoning effort high sourceModelVersion deepseek-v4-1-flash-high thinkingEnabled true ### Source [Original source](https://arcprize.org/media/data/leaderboard/v2.json) Source arc prize Original model label DeepSeek V4.1 Flash (High) Source version v2 Observed 2026-10-21T16:34:20.750Z Evidence SHA-256 `bca68e12cc9a01954ae21b641844c1ecbabe146cb24b87f0f8e9b1db27433c29` Source locator `/evaluations/266` | | | | | | | | | [Terminal-Bench 4.0 / software](https://benchize.com/benchmarks/vals-ai%3Aterminal-bench-4%3Asoftware.md) | 33.33% | SE ±6.42 pp | 17 / 36 | — | — | [Vals AI · 4.0](https://www.vals.ai/_astro/benchmark_view_terminal_bench_4.CAnGyzC_.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 33.333 \| percent \| reported \| Higher is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±6.42 pp sourcePopulation 52 stderr 6.415 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_terminal_bench_4.CAnGyzC_.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash Source version 4.0 Observed 2026-10-21T16:46:10.642Z Evidence SHA-256 `98f6964580e901f1aa143db664bd92aa558415f643569492545d67c47d7f519b` Source locator `/tasks/software/deepseek~1deepseek-v4.1-flash` | | | | | | | | | [ProofBench v1.1 / overall](https://benchize.com/benchmarks/vals-ai%3Aproof_bench%3Aoverall.md) | 54% | SE ±5.01 pp | 22 / 34 | $0.13 | 546.6source duration | [Vals AI · 1.1](https://www.vals.ai/_astro/benchmark_view_proof_bench.i3Hwh8rW.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 54 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.126437 \| USD \| per_test \| Lower is better \| \| `latency` \| 546.595 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±5.01 pp sourcePopulation 52 stderr 5.009 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_proof_bench.i3Hwh8rW.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash Source version 1.1 Observed 2026-10-21T16:46:01.401Z Evidence SHA-256 `22be2aca25de5c4a9eb6448a95713e09a7c97126dfc6f4f75c4afd0e563209cb` Source locator `/tasks/overall/deepseek~1deepseek-v4.1-flash` | | | | | | | | | [FrontierCode 1.1 main](https://benchize.com/benchmarks/frontiercode%3Av1_1%3Amain.md) | 43.87% | — | 51 / 125 | $0.62 | — | [frontiercode](https://cognition.com/data/frontiercode-leaderboard/data.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 0.4387 \| score_fraction \| blocking_criteria_adjusted \| Higher is better \| \| `correct` \| 0.4863 \| score_fraction \| raw_rubric \| Higher is better \| \| `cost` \| 0.6202 \| USD \| per_rollout \| Lower is better \| ### Configuration harness chisel Reasoning effort high sourceVersion v1_1 subset main ### Reported statistics flaggedRate 0.0073 sampleSize 100 ### Source [Original source](https://cognition.com/data/frontiercode-leaderboard/data.json) Source frontiercode Original model label DeepSeek V4.1 Flash Observed 2026-10-21T16:31:00.878Z Evidence SHA-256 `ff0040778b7bdf69c705ad7474f771fb5e53cfaf5cf8573b3a4614153543d75f` Source locator `/v1_1/data/DeepSeek%20V4.1%20Flash/high/main` | | | | | | | | | [MysteryMechanism / overall](https://benchize.com/benchmarks/vals-ai%3Amysterymechanism%3Aoverall.md) | 21.17% | SE ±2.75 pp | 14 / 19 | $0.16 | 484.33source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_mysterymechanism.75uyyp4P.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 21.171 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.164089 \| USD \| per_test \| Lower is better \| \| `latency` \| 484.327 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.75 pp sourcePopulation 27 stderr 2.748 tokenTotals {"cache_read_tokens":614311296,"cache_write_tokens":0,"input_tokens":619131425,"output_tokens":26079934,"reasoning_tokens":20201286} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_mysterymechanism.75uyyp4P.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash Source version 1 Observed 2026-10-21T16:46:04.766Z Evidence SHA-256 `1ef09cd0a0ee8690b0c07b2272057461242359b1680e24975ea42971e03acea9` Source locator `/tasks/overall/deepseek~1deepseek-v4.1-flash` | | | | | | | | | [LegalBench / interpretation tasks](https://benchize.com/benchmarks/vals-ai%3Alegal_bench%3Ainterpretation_tasks.md) | 80.45% | SE ±0.2 pp | 24 / 44 | $0.00124 | 5.27source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_bench.D6uTgA3b.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 80.448 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.001236 \| USD \| per_test \| Lower is better \| \| `latency` \| 5.273 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.2 pp sourcePopulation 151 stderr 0.203 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_bench.D6uTgA3b.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash Source version 1 Observed 2026-10-21T16:45:53.390Z Evidence SHA-256 `c8486ba756baa5cd3955d035edff67440e8db23e69d62ad1a9e18d15b8c4bc52` Source locator `/tasks/interpretation_tasks/deepseek~1deepseek-v4.1-flash` | | | | | | | | | [Vals Science Index / overall](https://benchize.com/benchmarks/vals-ai%3Ascience_index%3Aoverall.md) | 29.65% | SE ±0.99 pp | 13 / 17 | $0.29 | 3,998.66source duration | [Vals AI · 1.0](https://www.vals.ai/_astro/benchmark_view_science_index.CN7FN-f1.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 29.65 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.290142 \| USD \| per_test \| Lower is better \| \| `latency` \| 3998.656 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.99 pp sourcePopulation 21 stderr 0.988 tokenTotals {"cache_read_tokens":2635955320,"cache_write_tokens":0,"input_tokens":4412167254,"output_tokens":70921915,"reasoning_tokens":46020082} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_science_index.CN7FN-f1.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash Source version 1.0 Observed 2026-10-21T16:45:50.879Z Evidence SHA-256 `fa4313008e91752fcfc185aaa78665f780d0fc23d7c7198a9853dbf2eaaaef1f` Source locator `/tasks/overall/deepseek~1deepseek-v4.1-flash` | | | | | | | | | [CyberBench v1.1 / patch](https://benchize.com/benchmarks/benchmark%3Acyber%3Apatch.md) | 85.71% | SE ±4.68 pp | 11 / 28 | $0.05 | 556.02source duration | [Vals AI · 1.1](https://www.vals.ai/_astro/benchmark_view_cyberbench.DS6UOlu0.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 85.714 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.053798 \| USD \| per_test \| Lower is better \| \| `latency` \| 556.018 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±4.68 pp sourcePopulation 47 stderr 4.676 tokenTotals {"cache_read_tokens":163675136,"cache_write_tokens":null,"input_tokens":167105583,"output_tokens":1611740,"reasoning_tokens":1272951} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_cyberbench.DS6UOlu0.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash Source version 1.1 Observed 2026-10-21T16:46:15.238Z Evidence SHA-256 `943f593f929f8971a6b746269a2851b5febc2aad4e2f1066895f715033cdc7f1` Source locator `/tasks/patch/deepseek~1deepseek-v4.1-flash` | | | | | | | | | [Finance Agent (v2) / adjustments](https://benchize.com/benchmarks/vals-ai%3Afabv2%3Aadjustments.md) | 49.24% | SE ±0.99 pp | 23 / 50 | $0.2 | 356.74source duration | [Vals AI · 2](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 49.241 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.202031 \| USD \| per_test \| Lower is better \| \| `latency` \| 356.743 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.99 pp sourcePopulation 80 stderr 0.989 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash Source version 2 Observed 2026-10-21T16:45:57.035Z Evidence SHA-256 `f731b20a26c126725d125519f2b5841aa45254656dba859a9570d7ce7e5d8191` Source locator `/tasks/adjustments/deepseek~1deepseek-v4.1-flash` | | | | | | | | | [Finance Agent (v2) / general quantitative analysis](https://benchize.com/benchmarks/vals-ai%3Afabv2%3Ageneral_quantitative_analysis.md) | 69.81% | SE ±3.02 pp | 33 / 50 | $0.21 | 392.33source duration | [Vals AI · 2](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 69.814 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.206219 \| USD \| per_test \| Lower is better \| \| `latency` \| 392.327 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±3.02 pp sourcePopulation 80 stderr 3.022 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash Source version 2 Observed 2026-10-21T16:45:57.035Z Evidence SHA-256 `f731b20a26c126725d125519f2b5841aa45254656dba859a9570d7ce7e5d8191` Source locator `/tasks/general_quantitative_analysis/deepseek~1deepseek-v4.1-flash` | | | | | | | | | [Vibe Code Bench v1.1 / overall](https://benchize.com/benchmarks/vals-ai%3Avibe-code%3Aoverall.md) | 84.74% | SE ±2.89 pp | 13 / 59 | $0.41 | 928.1source duration | [Vals AI · 1.1](https://www.vals.ai/_astro/benchmark_view_vibe_code_bench.DcEBMbCA.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 84.739 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.407445 \| USD \| per_test \| Lower is better \| \| `latency` \| 928.101 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness OpenHands maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.89 pp sourcePopulation 116 stderr 2.891 tokenTotals {"cache_read_tokens":1418642812,"cache_write_tokens":null,"input_tokens":1429431621,"output_tokens":7186472,"reasoning_tokens":1913924} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_vibe_code_bench.DcEBMbCA.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash Source version 1.1 Observed 2026-10-21T16:46:08.625Z Evidence SHA-256 `0e8c23f521a0f751c5592952620e197b376cade41c78578461b7aa5740a173e7` Source locator `/tasks/overall/deepseek~1deepseek-v4.1-flash` | | | | | | | | | [CyberBench v1.1 / poc](https://benchize.com/benchmarks/benchmark%3Acyber%3Apoc.md) | 61.67% | SE ±6.28 pp | 8 / 28 | $0.09 | 1,104.2source duration | [Vals AI · 1.1](https://www.vals.ai/_astro/benchmark_view_cyberbench.DS6UOlu0.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 61.667 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.085565 \| USD \| per_test \| Lower is better \| \| `latency` \| 1104.199 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±6.28 pp sourcePopulation 47 stderr 6.277 tokenTotals {"cache_read_tokens":617766272,"cache_write_tokens":null,"input_tokens":622988275,"output_tokens":2636167,"reasoning_tokens":1934806} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_cyberbench.DS6UOlu0.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash Source version 1.1 Observed 2026-10-21T16:46:15.238Z Evidence SHA-256 `943f593f929f8971a6b746269a2851b5febc2aad4e2f1066895f715033cdc7f1` Source locator `/tasks/poc/deepseek~1deepseek-v4.1-flash` | | | | | | | | | [ARC-AGI-3](https://benchize.com/benchmarks/arc-prize%3Av3_Semi_Private.md) | 0.48% | — | 30 / 53 | 1,005.8usd | — | [arc prize · v3](https://arcprize.org/media/data/leaderboard/v3.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 0.004846664966204641 \| score_fraction \| v3_Semi_Private \| Higher is better \| \| `total_cost` \| 1005.7964823 \| usd \| v3_Semi_Private \| Lower is better \| ### Configuration evaluationSplit v3_Semi_Private harness standard modelGroup deepseek-v4-1-flash modelType CoT Reasoning effort high sourceModelVersion deepseek-v4-1-flash-high thinkingEnabled true ### Source [Original source](https://arcprize.org/media/data/leaderboard/v3.json) Source arc prize Original model label DeepSeek V4.1 Flash (High) Source version v3 Observed 2026-10-21T16:34:21.112Z Evidence SHA-256 `1aaee2199f8c6d9b5a589aab2fd2480a729e9132f8cf8fbf7b49f81dbd5b2c49` Source locator `/evaluations/88` | | | | | | | | | [Finance Agent (v2) / disclosure analysis](https://benchize.com/benchmarks/vals-ai%3Afabv2%3Adisclosure_analysis.md) | 61.31% | SE ±1.94 pp | 29 / 50 | $0.28 | 511.69source duration | [Vals AI · 2](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 61.312 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.281162 \| USD \| per_test \| Lower is better \| \| `latency` \| 511.693 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.94 pp sourcePopulation 80 stderr 1.94 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash Source version 2 Observed 2026-10-21T16:45:57.035Z Evidence SHA-256 `f731b20a26c126725d125519f2b5841aa45254656dba859a9570d7ce7e5d8191` Source locator `/tasks/disclosure_analysis/deepseek~1deepseek-v4.1-flash` | | | | | | | | | [LegalBench / rule tasks](https://benchize.com/benchmarks/vals-ai%3Alegal_bench%3Arule_tasks.md) | 77.98% | SE ±1.69 pp | 27 / 44 | $0.00321 | 14.28source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_bench.D6uTgA3b.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 77.981 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.003209 \| USD \| per_test \| Lower is better \| \| `latency` \| 14.284 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.69 pp sourcePopulation 151 stderr 1.691 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_bench.D6uTgA3b.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash Source version 1 Observed 2026-10-21T16:45:53.390Z Evidence SHA-256 `c8486ba756baa5cd3955d035edff67440e8db23e69d62ad1a9e18d15b8c4bc52` Source locator `/tasks/rule_tasks/deepseek~1deepseek-v4.1-flash` | | | | | | | | | [WeirdML v3](https://benchize.com/benchmarks/weirdml%3Av3.md) | 0.06source score | Range 0.05–0.07 | 21 / 25 | — | — | [weirdml · b0120fe](https://htihle.github.io/assets/data/weirdml_v3.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 0.05830850151515152 \| source_score \| reported \| Higher is better \| \| `mean_final_best` \| 0.12373953181818181 \| source_score \| reported \| Higher is better \| ### Configuration budget {"kind":"equivalent_tokens","limit":50000000,"start":500000} harness codex_cli Reasoning effort high sourceConfiguration deepseek-v4.1-flash-novita-high-codex ### Reported statistics Uncertainty Reported interval: 0.05–0.07 interval [0.04578175205502483,0.07057167173922815] intervalMethod {"assumptions":"Independent fresh runs; approximately normal scores; shared variance prior. Pool fitted within analysis mode, including incomplete models; pool estimation uncertainty is not propagated.","confidence":0.95,"draws":10000,"fixed_tasks":true,"method":"partially_pooled_variance","pool":{"degrees_of_freedom":1533,"fallback":false,"prior_df":4,"variance":0.0026383113392869583},"seed":20260915,"unit":"run"} runCount 72 taskCount 11 ### Source [Original source](https://htihle.github.io/assets/data/weirdml_v3.json) Source weirdml Original model label DeepSeek V4.1 Flash (high, Novita) Source version b0120fe Observed 2026-10-21T16:33:04.394Z Evidence SHA-256 `82c3a8de423b23ca0af9761429299a2f21edb0b819e947fdc863f8b5ea4a49b8` Source locator `/rows/20` | | | | | | | | | [EMB / dcf](https://benchize.com/benchmarks/vals-ai%3Aemb%3Adcf.md) | 51.03% | SE ±8.96 pp | 31 / 47 | $0.24 | 832.09source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 51.026 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.237125 \| USD \| per_test \| Lower is better \| \| `latency` \| 832.088 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±8.96 pp sourcePopulation 76 stderr 8.961 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash Source version 1 Observed 2026-10-21T16:45:57.495Z Evidence SHA-256 `d807aee620ea47152a1fca0257a8dd12a1be4512c76cc1744c1ae9e285180918` Source locator `/tasks/dcf/deepseek~1deepseek-v4.1-flash` | | | | | | | | | [Vals Science Index / mysterymechanism](https://benchize.com/benchmarks/vals-ai%3Ascience_index%3Amysterymechanism.md) | 21.17% | SE ±2.75 pp | 13 / 17 | $0.16 | 484.33source duration | [Vals AI · 1.0](https://www.vals.ai/_astro/benchmark_view_science_index.CN7FN-f1.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 21.171 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.164089 \| USD \| per_test \| Lower is better \| \| `latency` \| 484.327 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.75 pp sourcePopulation 21 stderr 2.748 tokenTotals {"cache_read_tokens":614311296,"cache_write_tokens":0,"input_tokens":619131425,"output_tokens":26079934,"reasoning_tokens":20201286} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_science_index.CN7FN-f1.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash Source version 1.0 Observed 2026-10-21T16:45:50.879Z Evidence SHA-256 `fa4313008e91752fcfc185aaa78665f780d0fc23d7c7198a9853dbf2eaaaef1f` Source locator `/tasks/mysterymechanism/deepseek~1deepseek-v4.1-flash` | | | | | | | | | [BioMysteryBench / human difficult](https://benchize.com/benchmarks/vals-ai%3Abiomysterybench%3Ahuman_difficult.md) | 21.57% | SE ±1.96 pp | 17 / 20 | $0.43 | 1,629.1source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_biomysterybench.Bley-WAP.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 21.569 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.433129 \| USD \| per_test \| Lower is better \| \| `latency` \| 1629.099 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Terminus 2 maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.96 pp sourcePopulation 27 stderr 1.961 tokenTotals {"cache_read_tokens":104348544,"cache_write_tokens":0,"input_tokens":106436919,"output_tokens":5175983,"reasoning_tokens":4613462} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_biomysterybench.Bley-WAP.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash Source version 1 Observed 2026-10-21T16:46:04.029Z Evidence SHA-256 `59bf79d38ec415a34ffb1c2901a16ecbbe101b7fa864e86104d2571fc7cfc23d` Source locator `/tasks/human_difficult/deepseek~1deepseek-v4.1-flash` | | | | | | | | | [LegalBench / issue tasks](https://benchize.com/benchmarks/vals-ai%3Alegal_bench%3Aissue_tasks.md) | 88.75% | SE ±0.6 pp | 20 / 44 | $0.000887 | 4.23source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_bench.D6uTgA3b.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 88.753 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.000887 \| USD \| per_test \| Lower is better \| \| `latency` \| 4.229 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.6 pp sourcePopulation 151 stderr 0.597 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_bench.D6uTgA3b.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash Source version 1 Observed 2026-10-21T16:45:53.390Z Evidence SHA-256 `c8486ba756baa5cd3955d035edff67440e8db23e69d62ad1a9e18d15b8c4bc52` Source locator `/tasks/issue_tasks/deepseek~1deepseek-v4.1-flash` | | | | | | | | | [EMB / template](https://benchize.com/benchmarks/vals-ai%3Aemb%3Atemplate.md) | 48.18% | SE ±5.24 pp | 34 / 47 | $0.26 | 846source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 48.181 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.257032 \| USD \| per_test \| Lower is better \| \| `latency` \| 845.999 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±5.24 pp sourcePopulation 76 stderr 5.238 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash Source version 1 Observed 2026-10-21T16:45:57.495Z Evidence SHA-256 `d807aee620ea47152a1fca0257a8dd12a1be4512c76cc1744c1ae9e285180918` Source locator `/tasks/template/deepseek~1deepseek-v4.1-flash` | | | | | | | | | [EMB / ma](https://benchize.com/benchmarks/vals-ai%3Aemb%3Ama.md) | 57.78% | SE ±9.4 pp | 34 / 47 | $0.19 | 661.5source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 57.776 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.194981 \| USD \| per_test \| Lower is better \| \| `latency` \| 661.499 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±9.4 pp sourcePopulation 76 stderr 9.403 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash Source version 1 Observed 2026-10-21T16:45:57.495Z Evidence SHA-256 `d807aee620ea47152a1fca0257a8dd12a1be4512c76cc1744c1ae9e285180918` Source locator `/tasks/ma/deepseek~1deepseek-v4.1-flash` | | | | | | | | | [Terminal-Bench 4.0 / media](https://benchize.com/benchmarks/vals-ai%3Aterminal-bench-4%3Amedia.md) | 8.33% | SE ±8.33 pp | 22 / 36 | — | — | [Vals AI · 4.0](https://www.vals.ai/_astro/benchmark_view_terminal_bench_4.CAnGyzC_.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 8.333 \| percent \| reported \| Higher is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±8.33 pp sourcePopulation 52 stderr 8.333 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_terminal_bench_4.CAnGyzC_.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash Source version 4.0 Observed 2026-10-21T16:46:10.642Z Evidence SHA-256 `98f6964580e901f1aa143db664bd92aa558415f643569492545d67c47d7f519b` Source locator `/tasks/media/deepseek~1deepseek-v4.1-flash` | | | | | | | | | [Code Migration / overall](https://benchize.com/benchmarks/vals-ai%3Acode-migration%3Aoverall.md) | 45.62% | SE ±4.29 pp | 14 / 47 | $0.94 | 6,325.87source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_code_migration.Dttzoqk3.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 45.623 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.944694 \| USD \| per_test \| Lower is better \| \| `latency` \| 6325.865 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±4.29 pp sourcePopulation 79 stderr 4.285 tokenTotals {"cache_read_tokens":9693379447,"cache_write_tokens":null,"input_tokens":9823703780,"output_tokens":36706023,"reasoning_tokens":20727269} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_code_migration.Dttzoqk3.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash Source version 1 Observed 2026-10-21T16:46:12.712Z Evidence SHA-256 `99194597f2c76c402195be47839409046e18bd30454616da6dd40b736a088fa4` Source locator `/tasks/overall/deepseek~1deepseek-v4.1-flash` | | | | | | | | | [Terminal-Bench 2.1 / overall](https://benchize.com/benchmarks/vals-ai%3Aterminal-bench-2-1%3Aoverall.md) | 74.53% | SE ±1.63 pp | 15 / 43 | $0.1 | 604.51source duration | [Vals AI · 2.1](https://www.vals.ai/_astro/benchmark_view_terminal_bench_2_1.CdyoObsA.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 74.532 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.098877 \| USD \| per_test \| Lower is better \| \| `latency` \| 604.506 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.63 pp sourcePopulation 76 stderr 1.633 tokenTotals {"cache_read_tokens":54195840,"cache_write_tokens":0,"input_tokens":56873906,"output_tokens":6392867,"reasoning_tokens":5656496} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_terminal_bench_2_1.CdyoObsA.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash Source version 2.1 Observed 2026-10-21T16:46:13.216Z Evidence SHA-256 `a5306805cdf86495bdcceff6b2c8534a3768b9853f5077895dab975b809b054f` Source locator `/tasks/overall/deepseek~1deepseek-v4.1-flash` | | | | | | | | | [BioMysteryBench / overall](https://benchize.com/benchmarks/vals-ai%3Abiomysterybench%3Aoverall.md) | 67.78% | SE ±1.11 pp | 11 / 20 | $0.23 | 890.07source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_biomysterybench.Bley-WAP.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 67.778 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.233183 \| USD \| per_test \| Lower is better \| \| `latency` \| 890.067 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Terminus 2 maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.11 pp sourcePopulation 27 stderr 1.111 tokenTotals {"cache_read_tokens":262450176,"cache_write_tokens":0,"input_tokens":268679692,"output_tokens":14743987,"reasoning_tokens":13033344} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_biomysterybench.Bley-WAP.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash Source version 1 Observed 2026-10-21T16:46:04.029Z Evidence SHA-256 `59bf79d38ec415a34ffb1c2901a16ecbbe101b7fa864e86104d2571fc7cfc23d` Source locator `/tasks/overall/deepseek~1deepseek-v4.1-flash` | | | | | | | | | [NC Bench](https://benchize.com/benchmarks/nc-bench%3Aoverall.md) | 94.3% | — | 11 / 144 | — | — | [nc bench](https://www.nc-bench.com/models) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 94.3 \| percent \| reported \| Higher is better \| \| `tooling` \| 100 \| percent \| category \| Higher is better \| \| `creative_writing` \| 88.96 \| percent \| category \| Higher is better \| \| `language` \| 97.5 \| percent \| category \| Higher is better \| \| `utility` \| 99.43 \| percent \| category \| Higher is better \| \| `reasoning` \| 92.95 \| percent \| category \| Higher is better \| \| `text_editing` \| 98.47 \| percent \| category \| Higher is better \| \| `rule_following` \| 81.84 \| percent \| category \| Higher is better \| \| `hallucination` \| 95.27 \| percent \| category \| Higher is better \| ### Configuration Reasoning effort high Model configuration DeepSeek V4.1 Flash (Reasoning, High) thinkingEnabled true ### Source [Original source](https://www.nc-bench.com/models) Source nc bench Original model label DeepSeek V4.1 Flash (Reasoning, High) Observed 2026-10-21T16:31:48.370Z Evidence SHA-256 `ec261022458543624b08a136d01f5a2f0666cd52ffb49c64d6ca983e8543d1b5` Source locator `/rows/14` | | | | | | | | | [IOI / ioi2024](https://benchize.com/benchmarks/vals-ai%3Aioi%3Aioi2024.md) | 42.67% | SE ±14.69 pp | 23 / 29 | $0.21 | 914.56source duration | [Vals AI · 2](https://www.vals.ai/_astro/benchmark_view_ioi.CuUolFNv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 42.667 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.207781 \| USD \| per_test \| Lower is better \| \| `latency` \| 914.56 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±14.69 pp sourcePopulation 44 stderr 14.693 tokenTotals {"cache_read_tokens":27861632,"cache_write_tokens":0,"input_tokens":28008480,"output_tokens":856811,"reasoning_tokens":758852} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_ioi.CuUolFNv.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash Source version 2 Observed 2026-10-21T16:46:11.757Z Evidence SHA-256 `e1545b21b5b5c98d037aefa9d441b65859d07a10550f242d0e16162a49b8e8ab` Source locator `/tasks/ioi2024/deepseek~1deepseek-v4.1-flash` | | | | | | | | | [BioMysteryBench / human solvable](https://benchize.com/benchmarks/vals-ai%3Abiomysterybench%3Ahuman_solvable.md) | 78.54% | SE ±1.65 pp | 10 / 20 | $0.19 | 717.96source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_biomysterybench.Bley-WAP.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 78.539 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.186621 \| USD \| per_test \| Lower is better \| \| `latency` \| 717.964 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Terminus 2 maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.65 pp sourcePopulation 27 stderr 1.646 tokenTotals {"cache_read_tokens":158101632,"cache_write_tokens":0,"input_tokens":162242773,"output_tokens":9568004,"reasoning_tokens":8419882} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_biomysterybench.Bley-WAP.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash Source version 1 Observed 2026-10-21T16:46:04.029Z Evidence SHA-256 `59bf79d38ec415a34ffb1c2901a16ecbbe101b7fa864e86104d2571fc7cfc23d` Source locator `/tasks/human_solvable/deepseek~1deepseek-v4.1-flash` | | | | | | | | | [Terminal-Bench 4.0 / overall](https://benchize.com/benchmarks/vals-ai%3Aterminal-bench-4%3Aoverall.md) | 19.7% | SE ±1.75 pp | 20 / 36 | $0.5 | 2,902.32source duration | [Vals AI · 4.0](https://www.vals.ai/_astro/benchmark_view_terminal_bench_4.CAnGyzC_.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 19.697 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.499473 \| USD \| per_test \| Lower is better \| \| `latency` \| 2902.323 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.75 pp sourcePopulation 52 stderr 1.75 tokenTotals {"cache_read_tokens":2058436005,"cache_write_tokens":0,"input_tokens":2065521020,"output_tokens":15407576,"reasoning_tokens":12058588} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_terminal_bench_4.CAnGyzC_.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash Source version 4.0 Observed 2026-10-21T16:46:10.642Z Evidence SHA-256 `98f6964580e901f1aa143db664bd92aa558415f643569492545d67c47d7f519b` Source locator `/tasks/overall/deepseek~1deepseek-v4.1-flash` | | | | | | | | | [EMB / operating](https://benchize.com/benchmarks/vals-ai%3Aemb%3Aoperating.md) | 66.05% | SE ±6.29 pp | 17 / 47 | $0.21 | 687.31source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 66.048 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.206267 \| USD \| per_test \| Lower is better \| \| `latency` \| 687.305 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±6.29 pp sourcePopulation 76 stderr 6.291 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash Source version 1 Observed 2026-10-21T16:45:57.495Z Evidence SHA-256 `d807aee620ea47152a1fca0257a8dd12a1be4512c76cc1744c1ae9e285180918` Source locator `/tasks/operating/deepseek~1deepseek-v4.1-flash` | | | | | | | | | [Terminal-Bench 2.1 / hard](https://benchize.com/benchmarks/vals-ai%3Aterminal-bench-2-1%3Ahard.md) | 56.67% | SE ±1.93 pp | 23 / 43 | $0.29 | 1,793.37source duration | [Vals AI · 2.1](https://www.vals.ai/_astro/benchmark_view_terminal_bench_2_1.CdyoObsA.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 56.667 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.293335 \| USD \| per_test \| Lower is better \| \| `latency` \| 1793.369 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.93 pp sourcePopulation 76 stderr 1.925 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_terminal_bench_2_1.CdyoObsA.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash Source version 2.1 Observed 2026-10-21T16:46:13.216Z Evidence SHA-256 `a5306805cdf86495bdcceff6b2c8534a3768b9853f5077895dab975b809b054f` Source locator `/tasks/hard/deepseek~1deepseek-v4.1-flash` | | | | | | | | --- Cite as: DeepSeek V4.1 FlashHigh — Benchize, https://benchize.com/models/deepseek-v4-1-flash-high (retrieved 2026-10-21)