# EMB / dcf

> Source: Benchize (https://benchize.com) — https://benchize.com/benchmarks/vals-ai%3Aemb%3Adcf. When citing or reusing this content, credit "Benchize" and link to https://benchize.com/benchmarks/vals-ai%3Aemb%3Adcf.

Evaluating agents on Excel-based financial modeling tasks

Canonical URL: https://benchize.com/benchmarks/vals-ai%3Aemb%3Adcf
Last updated: Wed, 21 Oct 2026 18:52:20 GMT

Reproduction and quotation must name Benchize, link to the canonical HTML URL, and retain individual source citations.

Evaluating agents on Excel-based financial modeling tasks

Professional work · Models tested: 47 · Score publishers: Vals AI

## Model results

| # | Model | Score | ± | Cost | Latency |  |
| --- | --- | --- | --- | --- | --- | --- |
| 1 | [Claude Opus 5.5](https://benchize.com/models/claude-opus-5-5-max.md)Max | 70.14% | SE ±7.67 pp | $9.33 | 1,663.97source duration |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 70.139 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 9.328331 \| USD \| per_test \| Lower is better \| \| `latency` \| 1663.966 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±7.67 pp sourcePopulation 76 stderr 7.674 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) Source Vals AI Original model label anthropic/claude-opus-5-5 Source version 1 Observed 2026-10-21T16:45:57.495Z Evidence SHA-256 `d807aee620ea47152a1fca0257a8dd12a1be4512c76cc1744c1ae9e285180918` Source locator `/tasks/dcf/anthropic~1claude-opus-5-5` |  |  |  |  |  |  |
| 2 | [Claude Sonnet 5.5](https://benchize.com/models/claude-sonnet-5-5-max.md)Max | 70.07% | SE ±6.7 pp | $9.08 | 2,289.8source duration |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 70.065 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 9.078529 \| USD \| per_test \| Lower is better \| \| `latency` \| 2289.802 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±6.7 pp sourcePopulation 76 stderr 6.7 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) Source Vals AI Original model label anthropic/claude-sonnet-5-5 Source version 1 Observed 2026-10-21T16:45:57.495Z Evidence SHA-256 `d807aee620ea47152a1fca0257a8dd12a1be4512c76cc1744c1ae9e285180918` Source locator `/tasks/dcf/anthropic~1claude-sonnet-5-5` |  |  |  |  |  |  |
| 3 | [Claude Opus 5](https://benchize.com/models/claude-opus-5-max.md)Max | 70.02% | SE ±7.65 pp | $5.79 | 1,580.51source duration |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 70.018 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 5.792757 \| USD \| per_test \| Lower is better \| \| `latency` \| 1580.513 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±7.65 pp sourcePopulation 76 stderr 7.654 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) Source Vals AI Original model label anthropic/claude-opus-5 Source version 1 Observed 2026-10-21T16:45:57.495Z Evidence SHA-256 `d807aee620ea47152a1fca0257a8dd12a1be4512c76cc1744c1ae9e285180918` Source locator `/tasks/dcf/anthropic~1claude-opus-5` |  |  |  |  |  |  |
| 4 | [Claude Fable 5.1](https://benchize.com/models/claude-fable-5-1-max.md)Max | 69.88% | SE ±6.69 pp | $16.13 | 1,488.13source duration |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 69.88 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 16.129755 \| USD \| per_test \| Lower is better \| \| `latency` \| 1488.127 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±6.69 pp sourcePopulation 76 stderr 6.693 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) Source Vals AI Original model label anthropic/claude-fable-5-1 Source version 1 Observed 2026-10-21T16:45:57.495Z Evidence SHA-256 `d807aee620ea47152a1fca0257a8dd12a1be4512c76cc1744c1ae9e285180918` Source locator `/tasks/dcf/anthropic~1claude-fable-5-1` |  |  |  |  |  |  |
| 5 | [Claude Fable 5](https://benchize.com/models/claude-fable-5-max.md)Max | 68.87% | SE ±7.54 pp | $11.41 | 1,571.04source duration |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 68.865 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 11.409991 \| USD \| per_test \| Lower is better \| \| `latency` \| 1571.036 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±7.54 pp sourcePopulation 76 stderr 7.542 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) Source Vals AI Original model label anthropic/claude-fable-5 Source version 1 Observed 2026-10-21T16:45:57.495Z Evidence SHA-256 `d807aee620ea47152a1fca0257a8dd12a1be4512c76cc1744c1ae9e285180918` Source locator `/tasks/dcf/anthropic~1claude-fable-5` |  |  |  |  |  |  |
| 6 | [Claude Opus 4.8](https://benchize.com/models/claude-opus-4-8-max.md)Max | 66.33% | SE ±7.34 pp | $12.06 | 2,796.51source duration |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 66.334 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 12.058156 \| USD \| per_test \| Lower is better \| \| `latency` \| 2796.508 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±7.34 pp sourcePopulation 76 stderr 7.339 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) Source Vals AI Original model label anthropic/claude-opus-4-8 Source version 1 Observed 2026-10-21T16:45:57.495Z Evidence SHA-256 `d807aee620ea47152a1fca0257a8dd12a1be4512c76cc1744c1ae9e285180918` Source locator `/tasks/dcf/anthropic~1claude-opus-4-8` |  |  |  |  |  |  |
| 7 | [Gemini 3.7 Flash](https://benchize.com/models/gemini-3-7-flash-high.md)High | 66.06% | SE ±6.73 pp | $5.59 | 611.84source duration |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 66.059 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 5.594566 \| USD \| per_test \| Lower is better \| \| `latency` \| 611.84 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±6.73 pp sourcePopulation 76 stderr 6.729 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) Source Vals AI Original model label google/gemini-3.7-flash Source version 1 Observed 2026-10-21T16:45:57.495Z Evidence SHA-256 `d807aee620ea47152a1fca0257a8dd12a1be4512c76cc1744c1ae9e285180918` Source locator `/tasks/dcf/google~1gemini-3.7-flash` |  |  |  |  |  |  |
| 8 | [Gemini 4 Argon](https://benchize.com/models/gemini-4-argon-high.md)High | 65.72% | SE ±7.02 pp | $12.25 | 2,275.27source duration |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 65.719 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 12.254499 \| USD \| per_test \| Lower is better \| \| `latency` \| 2275.267 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 262144 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±7.02 pp sourcePopulation 76 stderr 7.019 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) Source Vals AI Original model label google/gemini-4-argon Source version 1 Observed 2026-10-21T16:45:57.495Z Evidence SHA-256 `d807aee620ea47152a1fca0257a8dd12a1be4512c76cc1744c1ae9e285180918` Source locator `/tasks/dcf/google~1gemini-4-argon` |  |  |  |  |  |  |
| 9 | [GPT-6.1 Sol](https://benchize.com/models/gpt-6-1-sol-max.md)Max | 65.31% | SE ±7.53 pp | $0.76 | 784.13source duration |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 65.313 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.759726 \| USD \| per_test \| Lower is better \| \| `latency` \| 784.133 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±7.53 pp sourcePopulation 76 stderr 7.529 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) Source Vals AI Original model label openai/gpt-6.1-sol Source version 1 Observed 2026-10-21T16:45:57.495Z Evidence SHA-256 `d807aee620ea47152a1fca0257a8dd12a1be4512c76cc1744c1ae9e285180918` Source locator `/tasks/dcf/openai~1gpt-6.1-sol` |  |  |  |  |  |  |
| 10 | [GPT-6 Astra](https://benchize.com/models/gpt-6-astra-max.md)Max | 64.48% | SE ±7.44 pp | $4.11 | 649.61source duration |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 64.483 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 4.111119 \| USD \| per_test \| Lower is better \| \| `latency` \| 649.606 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±7.44 pp sourcePopulation 76 stderr 7.442 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) Source Vals AI Original model label openai/gpt-6-astra Source version 1 Observed 2026-10-21T16:45:57.495Z Evidence SHA-256 `d807aee620ea47152a1fca0257a8dd12a1be4512c76cc1744c1ae9e285180918` Source locator `/tasks/dcf/openai~1gpt-6-astra` |  |  |  |  |  |  |
| 11 | [Gemini 3.8 Flash](https://benchize.com/models/gemini-3-8-flash-high.md)High | 63.66% | SE ±6.76 pp | $6.77 | 673.25source duration |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 63.655 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 6.77049 \| USD \| per_test \| Lower is better \| \| `latency` \| 673.252 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±6.76 pp sourcePopulation 76 stderr 6.76 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) Source Vals AI Original model label google/gemini-3.8-flash Source version 1 Observed 2026-10-21T16:45:57.495Z Evidence SHA-256 `d807aee620ea47152a1fca0257a8dd12a1be4512c76cc1744c1ae9e285180918` Source locator `/tasks/dcf/google~1gemini-3.8-flash` |  |  |  |  |  |  |
| 12 | [GPT-6 Sol](https://benchize.com/models/gpt-6-sol-max.md)Max | 63.14% | SE ±7.39 pp | $0.94 | 548.15source duration |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 63.142 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.938299 \| USD \| per_test \| Lower is better \| \| `latency` \| 548.146 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±7.39 pp sourcePopulation 76 stderr 7.389 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) Source Vals AI Original model label openai/gpt-6-sol Source version 1 Observed 2026-10-21T16:45:57.495Z Evidence SHA-256 `d807aee620ea47152a1fca0257a8dd12a1be4512c76cc1744c1ae9e285180918` Source locator `/tasks/dcf/openai~1gpt-6-sol` |  |  |  |  |  |  |
| 13 | [GPT-6 Luna](https://benchize.com/models/gpt-6-luna-max.md)Max | 62.77% | SE ±7.53 pp | $0.11 | 1,209.68source duration |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 62.771 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.105228 \| USD \| per_test \| Lower is better \| \| `latency` \| 1209.678 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±7.53 pp sourcePopulation 76 stderr 7.527 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) Source Vals AI Original model label openai/gpt-6-luna Source version 1 Observed 2026-10-21T16:45:57.495Z Evidence SHA-256 `d807aee620ea47152a1fca0257a8dd12a1be4512c76cc1744c1ae9e285180918` Source locator `/tasks/dcf/openai~1gpt-6-luna` |  |  |  |  |  |  |
| 14 | [GPT-5.6 Sol](https://benchize.com/models/gpt-5-6-sol-max.md)Max | 62.76% | SE ±7.23 pp | $3.85 | 510.32source duration |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 62.764 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 3.850982 \| USD \| per_test \| Lower is better \| \| `latency` \| 510.322 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±7.23 pp sourcePopulation 76 stderr 7.225 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) Source Vals AI Original model label openai/gpt-5.6-sol Source version 1 Observed 2026-10-21T16:45:57.495Z Evidence SHA-256 `d807aee620ea47152a1fca0257a8dd12a1be4512c76cc1744c1ae9e285180918` Source locator `/tasks/dcf/openai~1gpt-5.6-sol` |  |  |  |  |  |  |
| 15 | [GPT-5.6 Luna](https://benchize.com/models/gpt-5-6-luna-max.md)Max | 60.48% | SE ±7.08 pp | $0.22 | 635.86source duration |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 60.482 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.221953 \| USD \| per_test \| Lower is better \| \| `latency` \| 635.862 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±7.08 pp sourcePopulation 76 stderr 7.08 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) Source Vals AI Original model label openai/gpt-5.6-luna Source version 1 Observed 2026-10-21T16:45:57.495Z Evidence SHA-256 `d807aee620ea47152a1fca0257a8dd12a1be4512c76cc1744c1ae9e285180918` Source locator `/tasks/dcf/openai~1gpt-5.6-luna` |  |  |  |  |  |  |
| 16 | [Gemini 3.6 Flash](https://benchize.com/models/gemini-3-6-flash-high.md)High | 59.41% | SE ±6.35 pp | $3.07 | 686.71source duration |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 59.408 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 3.066222 \| USD \| per_test \| Lower is better \| \| `latency` \| 686.706 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±6.35 pp sourcePopulation 76 stderr 6.348 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) Source Vals AI Original model label google/gemini-3.6-flash Source version 1 Observed 2026-10-21T16:45:57.495Z Evidence SHA-256 `d807aee620ea47152a1fca0257a8dd12a1be4512c76cc1744c1ae9e285180918` Source locator `/tasks/dcf/google~1gemini-3.6-flash` |  |  |  |  |  |  |
| 17 | [MiMo-V2.6-Flash](https://benchize.com/models/mimo-v2-6-flash.md) | 58.95% | SE ±7.29 pp | $0.11 | 1,395.59source duration |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 58.952 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.114439 \| USD \| per_test \| Lower is better \| \| `latency` \| 1395.594 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider Xiaomi reasoning Not reported Reasoning effort unknown Temperature 1 topP 0.95 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±7.29 pp sourcePopulation 76 stderr 7.294 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) Source Vals AI Original model label xiaomi/mimo-v2.6-flash Source version 1 Observed 2026-10-21T16:45:57.495Z Evidence SHA-256 `d807aee620ea47152a1fca0257a8dd12a1be4512c76cc1744c1ae9e285180918` Source locator `/tasks/dcf/xiaomi~1mimo-v2.6-flash` |  |  |  |  |  |  |
| 18 | [Gemini 3.5 Flash](https://benchize.com/models/gemini-3-5-flash-high.md)High | 58.36% | SE ±6.89 pp | $4.06 | 975.44source duration |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 58.36 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 4.056284 \| USD \| per_test \| Lower is better \| \| `latency` \| 975.444 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±6.89 pp sourcePopulation 76 stderr 6.889 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) Source Vals AI Original model label google/gemini-3.5-flash Source version 1 Observed 2026-10-21T16:45:57.495Z Evidence SHA-256 `d807aee620ea47152a1fca0257a8dd12a1be4512c76cc1744c1ae9e285180918` Source locator `/tasks/dcf/google~1gemini-3.5-flash` |  |  |  |  |  |  |
| 19 | [MiMo-V2.6-Pro](https://benchize.com/models/mimo-v2-6-pro.md) | 58.12% | SE ±7.97 pp | $0.31 | 2,025.65source duration |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 58.121 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.306338 \| USD \| per_test \| Lower is better \| \| `latency` \| 2025.653 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider Xiaomi reasoning Not reported Reasoning effort unknown Temperature 1 topP 0.95 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±7.97 pp sourcePopulation 76 stderr 7.97 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) Source Vals AI Original model label xiaomi/mimo-v2.6-pro Source version 1 Observed 2026-10-21T16:45:57.495Z Evidence SHA-256 `d807aee620ea47152a1fca0257a8dd12a1be4512c76cc1744c1ae9e285180918` Source locator `/tasks/dcf/xiaomi~1mimo-v2.6-pro` |  |  |  |  |  |  |
| 20 | [Claude Opus 4.7](https://benchize.com/models/claude-opus-4-7-max.md)Max | 57.07% | SE ±8.27 pp | $14.01 | 2,515.23source duration |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 57.069 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 14.008004 \| USD \| per_test \| Lower is better \| \| `latency` \| 2515.234 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±8.27 pp sourcePopulation 76 stderr 8.273 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) Source Vals AI Original model label anthropic/claude-opus-4-7 Source version 1 Observed 2026-10-21T16:45:57.495Z Evidence SHA-256 `d807aee620ea47152a1fca0257a8dd12a1be4512c76cc1744c1ae9e285180918` Source locator `/tasks/dcf/anthropic~1claude-opus-4-7` |  |  |  |  |  |  |
| 21 | [GPT-5.5](https://benchize.com/models/gpt-5-5-xhigh.md)Extra high | 56.74% | SE ±7.84 pp | $2.84 | 883.69source duration |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 56.736 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 2.839702 \| USD \| per_test \| Lower is better \| \| `latency` \| 883.688 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort xhigh Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±7.84 pp sourcePopulation 76 stderr 7.836 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) Source Vals AI Original model label openai/gpt-5.5 Source version 1 Observed 2026-10-21T16:45:57.495Z Evidence SHA-256 `d807aee620ea47152a1fca0257a8dd12a1be4512c76cc1744c1ae9e285180918` Source locator `/tasks/dcf/openai~1gpt-5.5` |  |  |  |  |  |  |
| 22 | [Claude Sonnet 5](https://benchize.com/models/claude-sonnet-5-max.md)Max | 56.4% | SE ±8.13 pp | $9.69 | 2,886.85source duration |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 56.404 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 9.689342 \| USD \| per_test \| Lower is better \| \| `latency` \| 2886.854 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±8.13 pp sourcePopulation 76 stderr 8.129 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) Source Vals AI Original model label anthropic/claude-sonnet-5 Source version 1 Observed 2026-10-21T16:45:57.495Z Evidence SHA-256 `d807aee620ea47152a1fca0257a8dd12a1be4512c76cc1744c1ae9e285180918` Source locator `/tasks/dcf/anthropic~1claude-sonnet-5` |  |  |  |  |  |  |
| 23 | [Muse Spark 1.3](https://benchize.com/models/muse-spark-1-3-xhigh.md)Extra high | 55.7% | SE ±8.02 pp | $2.44 | 1,217.18source duration |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 55.697 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 2.437825 \| USD \| per_test \| Lower is better \| \| `latency` \| 1217.179 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 131072 provider Meta reasoning Not reported Reasoning effort xhigh Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±8.02 pp sourcePopulation 76 stderr 8.018 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) Source Vals AI Original model label meta/muse_spark_1_3 Source version 1 Observed 2026-10-21T16:45:57.495Z Evidence SHA-256 `d807aee620ea47152a1fca0257a8dd12a1be4512c76cc1744c1ae9e285180918` Source locator `/tasks/dcf/meta~1muse_spark_1_3` |  |  |  |  |  |  |
| 24 | [Muse Spark 1.2](https://benchize.com/models/muse-spark-1-2-xhigh.md)Extra high | 55.56% | SE ±7.27 pp | $1.96 | 877.04source duration |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 55.562 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 1.955636 \| USD \| per_test \| Lower is better \| \| `latency` \| 877.044 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 131072 provider Meta reasoning Not reported Reasoning effort xhigh Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±7.27 pp sourcePopulation 76 stderr 7.274 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) Source Vals AI Original model label meta/muse_spark_1_2 Source version 1 Observed 2026-10-21T16:45:57.495Z Evidence SHA-256 `d807aee620ea47152a1fca0257a8dd12a1be4512c76cc1744c1ae9e285180918` Source locator `/tasks/dcf/meta~1muse_spark_1_2` |  |  |  |  |  |  |
| 25 | [Claude Haiku 5.5](https://benchize.com/models/claude-haiku-5-5-max.md)Max | 55.11% | SE ±10.41 pp | $0.92 | 950.59source duration |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 55.108 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.919685 \| USD \| per_test \| Lower is better \| \| `latency` \| 950.59 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±10.41 pp sourcePopulation 76 stderr 10.412 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) Source Vals AI Original model label anthropic/claude-haiku-5-5 Source version 1 Observed 2026-10-21T16:45:57.495Z Evidence SHA-256 `d807aee620ea47152a1fca0257a8dd12a1be4512c76cc1744c1ae9e285180918` Source locator `/tasks/dcf/anthropic~1claude-haiku-5-5` |  |  |  |  |  |  |
| 26 | [GPT-5.6 Terra](https://benchize.com/models/gpt-5-6-terra-max.md)Max | 53.4% | SE ±7.75 pp | $1.44 | 672.59source duration |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 53.395 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 1.438464 \| USD \| per_test \| Lower is better \| \| `latency` \| 672.589 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±7.75 pp sourcePopulation 76 stderr 7.751 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) Source Vals AI Original model label openai/gpt-5.6-terra Source version 1 Observed 2026-10-21T16:45:57.495Z Evidence SHA-256 `d807aee620ea47152a1fca0257a8dd12a1be4512c76cc1744c1ae9e285180918` Source locator `/tasks/dcf/openai~1gpt-5.6-terra` |  |  |  |  |  |  |
| 27 | [DeepSeek V4.1 Flash 1017](https://benchize.com/models/deepseek-v4-1-flash-1017-max.md)Max | 53% | SE ±9.43 pp | $0.35 | 1,032.07source duration |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 53 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.350055 \| USD \| per_test \| Lower is better \| \| `latency` \| 1032.065 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±9.43 pp sourcePopulation 76 stderr 9.434 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash-1017 Source version 1 Observed 2026-10-21T16:45:57.495Z Evidence SHA-256 `d807aee620ea47152a1fca0257a8dd12a1be4512c76cc1744c1ae9e285180918` Source locator `/tasks/dcf/deepseek~1deepseek-v4.1-flash-1017` |  |  |  |  |  |  |
| 28 | [MiMo-V2.5](https://benchize.com/models/mimo-v2-5.md) | 52.54% | SE ±7.81 pp | $0.08 | 2,105.83source duration |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 52.539 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.078854 \| USD \| per_test \| Lower is better \| \| `latency` \| 2105.827 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider Xiaomi reasoning Not reported Reasoning effort unknown Temperature 1 topP 0.95 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±7.81 pp sourcePopulation 76 stderr 7.805 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) Source Vals AI Original model label xiaomi/mimo-v2.5 Source version 1 Observed 2026-10-21T16:45:57.495Z Evidence SHA-256 `d807aee620ea47152a1fca0257a8dd12a1be4512c76cc1744c1ae9e285180918` Source locator `/tasks/dcf/xiaomi~1mimo-v2.5` |  |  |  |  |  |  |
| 29 | [DeepSeek V4 Flash 0731](https://benchize.com/models/deepseek-v4-flash-0731-high.md)High | 51.29% | SE ±7.34 pp | $0.38 | 1,432.3source duration |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 51.288 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.38317 \| USD \| per_test \| Lower is better \| \| `latency` \| 1432.295 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±7.34 pp sourcePopulation 76 stderr 7.335 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) Source Vals AI Original model label deepseek/deepseek-v4-flash-0731 Source version 1 Observed 2026-10-21T16:45:57.495Z Evidence SHA-256 `d807aee620ea47152a1fca0257a8dd12a1be4512c76cc1744c1ae9e285180918` Source locator `/tasks/dcf/deepseek~1deepseek-v4-flash-0731` |  |  |  |  |  |  |
| 30 | [Muse Wonder](https://benchize.com/models/muse-wonder-max.md)Max | 51.04% | SE ±8.97 pp | $5.12 | 968.13source duration |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 51.035 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 5.11822 \| USD \| per_test \| Lower is better \| \| `latency` \| 968.128 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 131072 provider Meta reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±8.97 pp sourcePopulation 76 stderr 8.965 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) Source Vals AI Original model label meta/muse-wonder Source version 1 Observed 2026-10-21T16:45:57.495Z Evidence SHA-256 `d807aee620ea47152a1fca0257a8dd12a1be4512c76cc1744c1ae9e285180918` Source locator `/tasks/dcf/meta~1muse-wonder` |  |  |  |  |  |  |
| 31 | [DeepSeek V4.1 Flash](https://benchize.com/models/deepseek-v4-1-flash-high.md)High | 51.03% | SE ±8.96 pp | $0.24 | 832.09source duration |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 51.026 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.237125 \| USD \| per_test \| Lower is better \| \| `latency` \| 832.088 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±8.96 pp sourcePopulation 76 stderr 8.961 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash Source version 1 Observed 2026-10-21T16:45:57.495Z Evidence SHA-256 `d807aee620ea47152a1fca0257a8dd12a1be4512c76cc1744c1ae9e285180918` Source locator `/tasks/dcf/deepseek~1deepseek-v4.1-flash` |  |  |  |  |  |  |
| 32 | [Muse Spark Open](https://benchize.com/models/muse-spark-open-xhigh.md)Extra high | 47.54% | SE ±8.25 pp | $4.65 | 5,120.53source duration |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 47.537 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 4.653132 \| USD \| per_test \| Lower is better \| \| `latency` \| 5120.532 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 131072 provider Together AI reasoning Not reported Reasoning effort xhigh Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±8.25 pp sourcePopulation 76 stderr 8.245 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) Source Vals AI Original model label meta/muse-spark-open Source version 1 Observed 2026-10-21T16:45:57.495Z Evidence SHA-256 `d807aee620ea47152a1fca0257a8dd12a1be4512c76cc1744c1ae9e285180918` Source locator `/tasks/dcf/meta~1muse-spark-open` |  |  |  |  |  |  |
| 33 | [Step 5 Preview](https://benchize.com/models/step-5-preview.md) | 47.39% | SE ±9.65 pp | $1.21 | 2,762.16source duration |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 47.387 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 1.211646 \| USD \| per_test \| Lower is better \| \| `latency` \| 2762.159 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 1024000 provider Stepfun reasoning Not reported Reasoning effort unknown Temperature 1 topP 0.95 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±9.65 pp sourcePopulation 76 stderr 9.65 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) Source Vals AI Original model label stepfun/step-5-preview Source version 1 Observed 2026-10-21T16:45:57.495Z Evidence SHA-256 `d807aee620ea47152a1fca0257a8dd12a1be4512c76cc1744c1ae9e285180918` Source locator `/tasks/dcf/stepfun~1step-5-preview` |  |  |  |  |  |  |
| 34 | [Gemini 3.1 Pro Preview](https://benchize.com/models/gemini-3-1-pro-preview-high.md)High | 46.39% | SE ±7.72 pp | $2.5 | 584.59source duration |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 46.388 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 2.495552 \| USD \| per_test \| Lower is better \| \| `latency` \| 584.588 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±7.72 pp sourcePopulation 76 stderr 7.715 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) Source Vals AI Original model label google/gemini-3.1-pro-preview Source version 1 Observed 2026-10-21T16:45:57.495Z Evidence SHA-256 `d807aee620ea47152a1fca0257a8dd12a1be4512c76cc1744c1ae9e285180918` Source locator `/tasks/dcf/google~1gemini-3.1-pro-preview` |  |  |  |  |  |  |
| 35 | [Muse Spark 1.1](https://benchize.com/models/muse-spark-1-1-xhigh.md)Extra high | 46.33% | SE ±8.07 pp | $1 | 834.42source duration |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 46.328 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.996697 \| USD \| per_test \| Lower is better \| \| `latency` \| 834.416 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider Meta reasoning Not reported Reasoning effort xhigh Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±8.07 pp sourcePopulation 76 stderr 8.065 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) Source Vals AI Original model label meta/muse_spark_1_1 Source version 1 Observed 2026-10-21T16:45:57.495Z Evidence SHA-256 `d807aee620ea47152a1fca0257a8dd12a1be4512c76cc1744c1ae9e285180918` Source locator `/tasks/dcf/meta~1muse_spark_1_1` |  |  |  |  |  |  |
| 36 | [DeepSeek V4 Pro](https://benchize.com/models/deepseek-v4-pro-max.md)Max | 44.44% | SE ±6.24 pp | $0.65 | 816.69source duration |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 44.435 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.646466 \| USD \| per_test \| Lower is better \| \| `latency` \| 816.693 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±6.24 pp sourcePopulation 76 stderr 6.242 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) Source Vals AI Original model label deepseek/deepseek-v4-pro Source version 1 Observed 2026-10-21T16:45:57.495Z Evidence SHA-256 `d807aee620ea47152a1fca0257a8dd12a1be4512c76cc1744c1ae9e285180918` Source locator `/tasks/dcf/deepseek~1deepseek-v4-pro` |  |  |  |  |  |  |
| 37 | [DeepSeek V4 Pro 0813](https://benchize.com/models/deepseek-v4-pro-0813-max.md)Max | 43.65% | SE ±9.07 pp | $0.99 | 2,340.58source duration |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 43.653 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.986264 \| USD \| per_test \| Lower is better \| \| `latency` \| 2340.579 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±9.07 pp sourcePopulation 76 stderr 9.068 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) Source Vals AI Original model label deepseek/deepseek-v4-pro-0813 Source version 1 Observed 2026-10-21T16:45:57.495Z Evidence SHA-256 `d807aee620ea47152a1fca0257a8dd12a1be4512c76cc1744c1ae9e285180918` Source locator `/tasks/dcf/deepseek~1deepseek-v4-pro-0813` |  |  |  |  |  |  |
| 38 | [MiMo-V2.5-Pro](https://benchize.com/models/mimo-v2-5-pro.md) | 43.63% | SE ±7.73 pp | $0.22 | 1,961.14source duration |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 43.63 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.219228 \| USD \| per_test \| Lower is better \| \| `latency` \| 1961.142 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider Xiaomi reasoning Not reported Reasoning effort unknown Temperature 1 topP 0.95 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±7.73 pp sourcePopulation 76 stderr 7.728 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) Source Vals AI Original model label xiaomi/mimo-v2.5-pro Source version 1 Observed 2026-10-21T16:45:57.495Z Evidence SHA-256 `d807aee620ea47152a1fca0257a8dd12a1be4512c76cc1744c1ae9e285180918` Source locator `/tasks/dcf/xiaomi~1mimo-v2.5-pro` |  |  |  |  |  |  |
| 39 | [Claude Sonnet 4.6](https://benchize.com/models/claude-sonnet-4-6-max.md)Max | 38.28% | SE ±9.7 pp | $7.36 | 3,293source duration |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 38.28 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 7.355698 \| USD \| per_test \| Lower is better \| \| `latency` \| 3293.001 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning true Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±9.7 pp sourcePopulation 76 stderr 9.702 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) Source Vals AI Original model label anthropic/claude-sonnet-4-6 Source version 1 Observed 2026-10-21T16:45:57.495Z Evidence SHA-256 `d807aee620ea47152a1fca0257a8dd12a1be4512c76cc1744c1ae9e285180918` Source locator `/tasks/dcf/anthropic~1claude-sonnet-4-6` |  |  |  |  |  |  |
| 40 | [MiniMax M3](https://benchize.com/models/minimax-m3.md) | 36.63% | SE ±8.66 pp | $2.44 | 1,752.15source duration |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 36.629 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 2.435709 \| USD \| per_test \| Lower is better \| \| `latency` \| 1752.148 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 512000 provider MiniMax reasoning Not reported Reasoning effort unknown Temperature 1 topP 0.95 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±8.66 pp sourcePopulation 76 stderr 8.66 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) Source Vals AI Original model label minimax/MiniMax-M3 Source version 1 Observed 2026-10-21T16:45:57.495Z Evidence SHA-256 `d807aee620ea47152a1fca0257a8dd12a1be4512c76cc1744c1ae9e285180918` Source locator `/tasks/dcf/minimax~1MiniMax-M3` |  |  |  |  |  |  |
| 41 | [Gemini 3.5 Flash-Lite](https://benchize.com/models/gemini-3-5-flash-lite-high.md)High | 33.61% | SE ±7.56 pp | $1.27 | 529.93source duration |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 33.614 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 1.27137 \| USD \| per_test \| Lower is better \| \| `latency` \| 529.933 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±7.56 pp sourcePopulation 76 stderr 7.561 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) Source Vals AI Original model label google/gemini-3.5-flash-lite Source version 1 Observed 2026-10-21T16:45:57.495Z Evidence SHA-256 `d807aee620ea47152a1fca0257a8dd12a1be4512c76cc1744c1ae9e285180918` Source locator `/tasks/dcf/google~1gemini-3.5-flash-lite` |  |  |  |  |  |  |
| 42 | [Nemotron 3 Ultra](https://benchize.com/models/nemotron-3-ultra-550b-a55b.md) | 29.55% | SE ±6.77 pp | — | 2,934.04source duration |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 29.545 \| percent \| reported \| Higher is better \| \| `latency` \| 2934.041 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider Nvidia reasoning Not reported Reasoning effort unknown Temperature Not reported topP 0.95 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±6.77 pp sourcePopulation 76 stderr 6.769 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) Source Vals AI Original model label nvidia/nemotron-3-ultra-550b-a55b Source version 1 Observed 2026-10-21T16:45:57.495Z Evidence SHA-256 `d807aee620ea47152a1fca0257a8dd12a1be4512c76cc1744c1ae9e285180918` Source locator `/tasks/dcf/nvidia~1nemotron-3-ultra-550b-a55b` |  |  |  |  |  |  |
| 43 | [MiniMax M2.7](https://benchize.com/models/minimax-m2-7.md) | 21.88% | SE ±5.96 pp | $1.66 | 1,115.22source duration |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 21.881 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 1.656978 \| USD \| per_test \| Lower is better \| \| `latency` \| 1115.218 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 80000 provider MiniMax reasoning Not reported Reasoning effort unknown Temperature 1 topP 0.95 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±5.96 pp sourcePopulation 76 stderr 5.961 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) Source Vals AI Original model label minimax/MiniMax-M2.7 Source version 1 Observed 2026-10-21T16:45:57.495Z Evidence SHA-256 `d807aee620ea47152a1fca0257a8dd12a1be4512c76cc1744c1ae9e285180918` Source locator `/tasks/dcf/minimax~1MiniMax-M2.7` |  |  |  |  |  |  |
| 44 | [Gemini 3 Flash Preview](https://benchize.com/models/gemini-3-flash-preview-high.md)High | 21.39% | SE ±6.31 pp | $0.32 | 294.13source duration |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 21.392 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.323658 \| USD \| per_test \| Lower is better \| \| `latency` \| 294.126 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±6.31 pp sourcePopulation 76 stderr 6.312 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) Source Vals AI Original model label google/gemini-3-flash-preview Source version 1 Observed 2026-10-21T16:45:57.495Z Evidence SHA-256 `d807aee620ea47152a1fca0257a8dd12a1be4512c76cc1744c1ae9e285180918` Source locator `/tasks/dcf/google~1gemini-3-flash-preview` |  |  |  |  |  |  |
| 45 | [Gemini 3.1 Flash Lite Preview](https://benchize.com/models/gemini-3-1-flash-lite-preview-high.md)High | 5.1% | SE ±1.01 pp | $0.05 | 64.7source duration |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 5.098 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.046823 \| USD \| per_test \| Lower is better \| \| `latency` \| 64.7 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.01 pp sourcePopulation 76 stderr 1.006 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) Source Vals AI Original model label google/gemini-3.1-flash-lite-preview Source version 1 Observed 2026-10-21T16:45:57.495Z Evidence SHA-256 `d807aee620ea47152a1fca0257a8dd12a1be4512c76cc1744c1ae9e285180918` Source locator `/tasks/dcf/google~1gemini-3.1-flash-lite-preview` |  |  |  |  |  |  |
| 46 | [Mercury 2.5](https://benchize.com/models/mercury-2-5-high.md)High | 4.72% | SE ±1.92 pp | $0.41 | 280.22source duration |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 4.721 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.411431 \| USD \| per_test \| Lower is better \| \| `latency` \| 280.224 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Inception reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.92 pp sourcePopulation 76 stderr 1.918 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) Source Vals AI Original model label inception/mercury-2.5 Source version 1 Observed 2026-10-21T16:45:57.495Z Evidence SHA-256 `d807aee620ea47152a1fca0257a8dd12a1be4512c76cc1744c1ae9e285180918` Source locator `/tasks/dcf/inception~1mercury-2.5` |  |  |  |  |  |  |
| 47 | [Mistral Medium 3.5](https://benchize.com/models/mistral-medium-3-5-high.md)High | 2.22% | SE ±1.85 pp | $10.04 | 2,845.78source duration |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 2.218 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 10.043204 \| USD \| per_test \| Lower is better \| \| `latency` \| 2845.783 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 80000 provider Mistral AI reasoning Not reported Reasoning effort high Temperature 0.7 topP 0.95 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.85 pp sourcePopulation 76 stderr 1.847 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) Source Vals AI Original model label mistralai/mistral-medium-3.5 Source version 1 Observed 2026-10-21T16:45:57.495Z Evidence SHA-256 `d807aee620ea47152a1fca0257a8dd12a1be4512c76cc1744c1ae9e285180918` Source locator `/tasks/dcf/mistralai~1mistral-medium-3.5` |  |  |  |  |  |  |

---
Cite as: EMB / dcf — Benchize, https://benchize.com/benchmarks/vals-ai%3Aemb%3Adcf (retrieved 2026-10-21)