# EMB v1.1 / overall > Source: Benchize (https://benchize.com) — https://benchize.com/benchmarks/vals-ai%3Aemb-v1-1%3Aoverall. When citing or reusing this content, credit "Benchize" and link to https://benchize.com/benchmarks/vals-ai%3Aemb-v1-1%3Aoverall. Evaluating agents on Excel-based financial modeling tasks, graded by the GPT-6 Luna rubric judge with template scoring fixes Canonical URL: https://benchize.com/benchmarks/vals-ai%3Aemb-v1-1%3Aoverall Last updated: Wed, 21 Oct 2026 18:52:20 GMT Reproduction and quotation must name Benchize, link to the canonical HTML URL, and retain individual source citations. Evaluating agents on Excel-based financial modeling tasks, graded by the GPT-6 Luna rubric judge with template scoring fixes Professional work · Models tested: 19 · Score publishers: Vals AI ## Model results | # | Model | Score | ± | Cost | Latency | | | --- | --- | --- | --- | --- | --- | --- | | 1 | [Claude Fable 5.1](https://benchize.com/models/claude-fable-5-1-max.md)Max | 76.73% | SE ±2.13 pp | $15.93 | 1,771.56source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 76.728 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 15.93249 \| USD \| per_test \| Lower is better \| \| `latency` \| 1771.558 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.13 pp sourcePopulation 30 stderr 2.129 tokenTotals {"cache_read_tokens":378586242,"cache_write_tokens":76164267,"input_tokens":454757375,"output_tokens":12499169,"reasoning_tokens":8150917} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb_v1_1.11L06J7L.json) Source Vals AI Original model label anthropic/claude-fable-5-1 Source version 1.1 Observed 2026-10-21T16:45:55.456Z Evidence SHA-256 `eea2ef2c83a6dc7d553c7c6531c287710ea7ffbc56ee1ccec06c85b104b2408e` Source locator `/tasks/overall/anthropic~1claude-fable-5-1` | | | | | | | | 2 | [Claude Sonnet 5.5](https://benchize.com/models/claude-sonnet-5-5-max.md)Max | 75.9% | SE ±2.47 pp | $8.79 | 2,204.99source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 75.899 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 8.792863 \| USD \| per_test \| Lower is better \| \| `latency` \| 2204.991 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.47 pp sourcePopulation 30 stderr 2.468 tokenTotals {"cache_read_tokens":2311427347,"cache_write_tokens":56507215,"input_tokens":2367951142,"output_tokens":29328538,"reasoning_tokens":22320203} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb_v1_1.11L06J7L.json) Source Vals AI Original model label anthropic/claude-sonnet-5-5 Source version 1.1 Observed 2026-10-21T16:45:55.456Z Evidence SHA-256 `eea2ef2c83a6dc7d553c7c6531c287710ea7ffbc56ee1ccec06c85b104b2408e` Source locator `/tasks/overall/anthropic~1claude-sonnet-5-5` | | | | | | | | 3 | [Claude Opus 5.5](https://benchize.com/models/claude-opus-5-5-max.md)Max | 75.84% | SE ±2.41 pp | $11.02 | 2,106.43source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 75.836 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 11.018974 \| USD \| per_test \| Lower is better \| \| `latency` \| 2106.434 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.41 pp sourcePopulation 30 stderr 2.41 tokenTotals {"cache_read_tokens":861165257,"cache_write_tokens":61067293,"input_tokens":922243024,"output_tokens":23704546,"reasoning_tokens":18569025} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb_v1_1.11L06J7L.json) Source Vals AI Original model label anthropic/claude-opus-5-5 Source version 1.1 Observed 2026-10-21T16:45:55.456Z Evidence SHA-256 `eea2ef2c83a6dc7d553c7c6531c287710ea7ffbc56ee1ccec06c85b104b2408e` Source locator `/tasks/overall/anthropic~1claude-opus-5-5` | | | | | | | | 4 | [Gemini 4 Argon](https://benchize.com/models/gemini-4-argon-high.md)High | 74.83% | SE ±2.49 pp | $11.8 | 2,305.82source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 74.828 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 11.8029 \| USD \| per_test \| Lower is better \| \| `latency` \| 2305.821 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 262144 provider Google reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.49 pp sourcePopulation 30 stderr 2.494 tokenTotals {"cache_read_tokens":945025185,"cache_write_tokens":null,"input_tokens":1127460088,"output_tokens":14257556,"reasoning_tokens":9497334} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb_v1_1.11L06J7L.json) Source Vals AI Original model label google/gemini-4-argon Source version 1.1 Observed 2026-10-21T16:45:55.456Z Evidence SHA-256 `eea2ef2c83a6dc7d553c7c6531c287710ea7ffbc56ee1ccec06c85b104b2408e` Source locator `/tasks/overall/google~1gemini-4-argon` | | | | | | | | 5 | [Gemini 3.8 Flash](https://benchize.com/models/gemini-3-8-flash-high.md)High | 72.87% | SE ±2.42 pp | $8.24 | 769.2source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 72.871 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 8.243862 \| USD \| per_test \| Lower is better \| \| `latency` \| 769.197 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.42 pp sourcePopulation 30 stderr 2.424 tokenTotals {"cache_read_tokens":2787511478,"cache_write_tokens":null,"input_tokens":2960728904,"output_tokens":21722815,"reasoning_tokens":13162823} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb_v1_1.11L06J7L.json) Source Vals AI Original model label google/gemini-3.8-flash Source version 1.1 Observed 2026-10-21T16:45:55.456Z Evidence SHA-256 `eea2ef2c83a6dc7d553c7c6531c287710ea7ffbc56ee1ccec06c85b104b2408e` Source locator `/tasks/overall/google~1gemini-3.8-flash` | | | | | | | | 6 | [GPT-6 Astra](https://benchize.com/models/gpt-6-astra-max.md)Max | 72.68% | SE ±2.3 pp | $5.83 | 804.28source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 72.683 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 5.827063 \| USD \| per_test \| Lower is better \| \| `latency` \| 804.28 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.3 pp sourcePopulation 30 stderr 2.297 tokenTotals {"cache_read_tokens":104449111,"cache_write_tokens":null,"input_tokens":123989946,"output_tokens":5542631,"reasoning_tokens":3051513} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb_v1_1.11L06J7L.json) Source Vals AI Original model label openai/gpt-6-astra Source version 1.1 Observed 2026-10-21T16:45:55.456Z Evidence SHA-256 `eea2ef2c83a6dc7d553c7c6531c287710ea7ffbc56ee1ccec06c85b104b2408e` Source locator `/tasks/overall/openai~1gpt-6-astra` | | | | | | | | 7 | [Muse Wonder](https://benchize.com/models/muse-wonder-max.md)Max | 72.5% | SE ±2.41 pp | $5.6 | 1,034.44source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 72.503 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 5.599337 \| USD \| per_test \| Lower is better \| \| `latency` \| 1034.437 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 131072 provider Meta reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.41 pp sourcePopulation 30 stderr 2.414 tokenTotals {"cache_read_tokens":1021915980,"cache_write_tokens":0,"input_tokens":1072958990,"output_tokens":8128857,"reasoning_tokens":5046443} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb_v1_1.11L06J7L.json) Source Vals AI Original model label meta/muse-wonder Source version 1.1 Observed 2026-10-21T16:45:55.456Z Evidence SHA-256 `eea2ef2c83a6dc7d553c7c6531c287710ea7ffbc56ee1ccec06c85b104b2408e` Source locator `/tasks/overall/meta~1muse-wonder` | | | | | | | | 8 | [GPT-6.1 Sol](https://benchize.com/models/gpt-6-1-sol-max.md)Max | 71.09% | SE ±2.75 pp | $1.14 | 1,107.81source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 71.088 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 1.143199 \| USD \| per_test \| Lower is better \| \| `latency` \| 1107.811 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.75 pp sourcePopulation 30 stderr 2.748 tokenTotals {"cache_read_tokens":112922114,"cache_write_tokens":16703171,"input_tokens":129631093,"output_tokens":6056928,"reasoning_tokens":3495567} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb_v1_1.11L06J7L.json) Source Vals AI Original model label openai/gpt-6.1-sol Source version 1.1 Observed 2026-10-21T16:45:55.456Z Evidence SHA-256 `eea2ef2c83a6dc7d553c7c6531c287710ea7ffbc56ee1ccec06c85b104b2408e` Source locator `/tasks/overall/openai~1gpt-6.1-sol` | | | | | | | | 9 | [GPT-6 Luna](https://benchize.com/models/gpt-6-luna-max.md)Max | 69.95% | SE ±2.54 pp | $0.14 | 1,041.05source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 69.954 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.141708 \| USD \| per_test \| Lower is better \| \| `latency` \| 1041.052 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.54 pp sourcePopulation 30 stderr 2.542 tokenTotals {"cache_read_tokens":332678280,"cache_write_tokens":29598418,"input_tokens":362285754,"output_tokens":11007243,"reasoning_tokens":8590626} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb_v1_1.11L06J7L.json) Source Vals AI Original model label openai/gpt-6-luna Source version 1.1 Observed 2026-10-21T16:45:55.456Z Evidence SHA-256 `eea2ef2c83a6dc7d553c7c6531c287710ea7ffbc56ee1ccec06c85b104b2408e` Source locator `/tasks/overall/openai~1gpt-6-luna` | | | | | | | | 10 | [Step 5 Preview](https://benchize.com/models/step-5-preview.md) | 69.55% | SE ±2.49 pp | $1.33 | 2,615.1source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 69.552 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 1.329318 \| USD \| per_test \| Lower is better \| \| `latency` \| 2615.096 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 1024000 provider Stepfun reasoning Not reported Reasoning effort unknown Temperature 1 topP 0.95 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.49 pp sourcePopulation 30 stderr 2.486 tokenTotals {"cache_read_tokens":1227785728,"cache_write_tokens":null,"input_tokens":1262287639,"output_tokens":14703425,"reasoning_tokens":0} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb_v1_1.11L06J7L.json) Source Vals AI Original model label stepfun/step-5-preview Source version 1.1 Observed 2026-10-21T16:45:55.456Z Evidence SHA-256 `eea2ef2c83a6dc7d553c7c6531c287710ea7ffbc56ee1ccec06c85b104b2408e` Source locator `/tasks/overall/stepfun~1step-5-preview` | | | | | | | | 11 | [DeepSeek V4.1 Flash 1017](https://benchize.com/models/deepseek-v4-1-flash-1017-max.md)Max | 69.17% | SE ±2.4 pp | $0.33 | 1,971.69source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 69.173 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.330922 \| USD \| per_test \| Lower is better \| \| `latency` \| 1971.687 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.4 pp sourcePopulation 30 stderr 2.395 tokenTotals {"cache_read_tokens":1797102837,"cache_write_tokens":null,"input_tokens":1809205546,"output_tokens":16117195,"reasoning_tokens":11510477} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb_v1_1.11L06J7L.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash-1017 Source version 1.1 Observed 2026-10-21T16:45:55.456Z Evidence SHA-256 `eea2ef2c83a6dc7d553c7c6531c287710ea7ffbc56ee1ccec06c85b104b2408e` Source locator `/tasks/overall/deepseek~1deepseek-v4.1-flash-1017` | | | | | | | | 12 | [MiMo-V2.6-Flash](https://benchize.com/models/mimo-v2-6-flash.md) | 68.88% | SE ±2.27 pp | $0.11 | 1,521.91source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 68.882 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.108634 \| USD \| per_test \| Lower is better \| \| `latency` \| 1521.906 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider Xiaomi reasoning Not reported Reasoning effort unknown Temperature 1 topP 0.95 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.27 pp sourcePopulation 30 stderr 2.267 tokenTotals {"cache_read_tokens":861570496,"cache_write_tokens":null,"input_tokens":891374421,"output_tokens":16056047,"reasoning_tokens":12655956} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb_v1_1.11L06J7L.json) Source Vals AI Original model label xiaomi/mimo-v2.6-flash Source version 1.1 Observed 2026-10-21T16:45:55.456Z Evidence SHA-256 `eea2ef2c83a6dc7d553c7c6531c287710ea7ffbc56ee1ccec06c85b104b2408e` Source locator `/tasks/overall/xiaomi~1mimo-v2.6-flash` | | | | | | | | 13 | [MiniMax M3.1 Flash](https://benchize.com/models/minimax-m3-1-flash-max.md)Max | 67.84% | SE ±2.29 pp | $0.27 | 1,847.78source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 67.843 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.268958 \| USD \| per_test \| Lower is better \| \| `latency` \| 1847.78 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 262144 provider MiniMax reasoning Not reported Reasoning effort max Temperature 1 topP 0.95 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.29 pp sourcePopulation 30 stderr 2.29 tokenTotals {"cache_read_tokens":1112061687,"cache_write_tokens":0,"input_tokens":1127000678,"output_tokens":14186544,"reasoning_tokens":10606514} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb_v1_1.11L06J7L.json) Source Vals AI Original model label minimax/MiniMax-M3.1-flash Source version 1.1 Observed 2026-10-21T16:45:55.456Z Evidence SHA-256 `eea2ef2c83a6dc7d553c7c6531c287710ea7ffbc56ee1ccec06c85b104b2408e` Source locator `/tasks/overall/minimax~1MiniMax-M3.1-flash` | | | | | | | | 14 | [DeepSeek V4.1 Flash](https://benchize.com/models/deepseek-v4-1-flash-max.md)Max | 67.76% | SE ±2.57 pp | $0.23 | 780.46source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 67.758 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.234152 \| USD \| per_test \| Lower is better \| \| `latency` \| 780.458 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.57 pp sourcePopulation 30 stderr 2.565 tokenTotals {"cache_read_tokens":884618493,"cache_write_tokens":null,"input_tokens":893420650,"output_tokens":13279281,"reasoning_tokens":10184478} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb_v1_1.11L06J7L.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash Source version 1.1 Observed 2026-10-21T16:45:55.456Z Evidence SHA-256 `eea2ef2c83a6dc7d553c7c6531c287710ea7ffbc56ee1ccec06c85b104b2408e` Source locator `/tasks/overall/deepseek~1deepseek-v4.1-flash` | | | | | | | | 15 | [MiMo-V2.6-Pro](https://benchize.com/models/mimo-v2-6-pro.md) | 67.35% | SE ±2.9 pp | $0.33 | 3,313.92source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 67.347 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.328999 \| USD \| per_test \| Lower is better \| \| `latency` \| 3313.924 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider Xiaomi reasoning Not reported Reasoning effort unknown Temperature 1 topP 0.95 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.9 pp sourcePopulation 30 stderr 2.903 tokenTotals {"cache_read_tokens":813521152,"cache_write_tokens":null,"input_tokens":847310790,"output_tokens":17554862,"reasoning_tokens":14594001} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb_v1_1.11L06J7L.json) Source Vals AI Original model label xiaomi/mimo-v2.6-pro Source version 1.1 Observed 2026-10-21T16:45:55.456Z Evidence SHA-256 `eea2ef2c83a6dc7d553c7c6531c287710ea7ffbc56ee1ccec06c85b104b2408e` Source locator `/tasks/overall/xiaomi~1mimo-v2.6-pro` | | | | | | | | 16 | [Claude Haiku 5.5](https://benchize.com/models/claude-haiku-5-5-max.md)Max | 65.48% | SE ±3.69 pp | $1.36 | 1,195.53source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 65.477 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 1.364559 \| USD \| per_test \| Lower is better \| \| `latency` \| 1195.527 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±3.69 pp sourcePopulation 30 stderr 3.691 tokenTotals {"cache_read_tokens":1444841999,"cache_write_tokens":37877007,"input_tokens":1482730908,"output_tokens":25203657,"reasoning_tokens":19717258} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb_v1_1.11L06J7L.json) Source Vals AI Original model label anthropic/claude-haiku-5-5 Source version 1.1 Observed 2026-10-21T16:45:55.456Z Evidence SHA-256 `eea2ef2c83a6dc7d553c7c6531c287710ea7ffbc56ee1ccec06c85b104b2408e` Source locator `/tasks/overall/anthropic~1claude-haiku-5-5` | | | | | | | | 17 | [DeepSeek V4 Pro 0813](https://benchize.com/models/deepseek-v4-pro-0813-max.md)Max | 65.13% | SE ±2.35 pp | $1.08 | 2,247.24source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 65.128 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 1.08141 \| USD \| per_test \| Lower is better \| \| `latency` \| 2247.236 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.35 pp sourcePopulation 30 stderr 2.35 tokenTotals {"cache_read_tokens":855589504,"cache_write_tokens":null,"input_tokens":864870937,"output_tokens":15254147,"reasoning_tokens":11469965} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb_v1_1.11L06J7L.json) Source Vals AI Original model label deepseek/deepseek-v4-pro-0813 Source version 1.1 Observed 2026-10-21T16:45:55.456Z Evidence SHA-256 `eea2ef2c83a6dc7d553c7c6531c287710ea7ffbc56ee1ccec06c85b104b2408e` Source locator `/tasks/overall/deepseek~1deepseek-v4-pro-0813` | | | | | | | | 18 | [Muse Spark Open](https://benchize.com/models/muse-spark-open-xhigh.md)Extra high | 61.98% | SE ±2.7 pp | $5.35 | 5,078.61source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 61.983 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 5.348821 \| USD \| per_test \| Lower is better \| \| `latency` \| 5078.614 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 131072 provider Meta reasoning Not reported Reasoning effort xhigh Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.7 pp sourcePopulation 30 stderr 2.704 tokenTotals {"cache_read_tokens":642573184,"cache_write_tokens":null,"input_tokens":973042320,"output_tokens":8495845,"reasoning_tokens":4888586} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb_v1_1.11L06J7L.json) Source Vals AI Original model label meta/muse-spark-open Source version 1.1 Observed 2026-10-21T16:45:55.456Z Evidence SHA-256 `eea2ef2c83a6dc7d553c7c6531c287710ea7ffbc56ee1ccec06c85b104b2408e` Source locator `/tasks/overall/meta~1muse-spark-open` | | | | | | | | 19 | [MiniMax M3](https://benchize.com/models/minimax-m3.md) | 59.75% | SE ±2.54 pp | $3.18 | 1,298.81source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 59.754 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 3.18384 \| USD \| per_test \| Lower is better \| \| `latency` \| 1298.806 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 512000 provider MiniMax reasoning Not reported Reasoning effort unknown Temperature 1 topP 0.95 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.54 pp sourcePopulation 30 stderr 2.537 tokenTotals {"cache_read_tokens":2146906887,"cache_write_tokens":0,"input_tokens":2167916385,"output_tokens":14444692,"reasoning_tokens":4485341} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb_v1_1.11L06J7L.json) Source Vals AI Original model label minimax/MiniMax-M3 Source version 1.1 Observed 2026-10-21T16:45:55.456Z Evidence SHA-256 `eea2ef2c83a6dc7d553c7c6531c287710ea7ffbc56ee1ccec06c85b104b2408e` Source locator `/tasks/overall/minimax~1MiniMax-M3` | | | | | | | --- Cite as: EMB v1.1 / overall — Benchize, https://benchize.com/benchmarks/vals-ai%3Aemb-v1-1%3Aoverall (retrieved 2026-10-21)