# EMB v1.1 / comps > Source: Benchize (https://benchize.com) — https://benchize.com/benchmarks/vals-ai%3Aemb-v1-1%3Acomps. When citing or reusing this content, credit "Benchize" and link to https://benchize.com/benchmarks/vals-ai%3Aemb-v1-1%3Acomps. Evaluating agents on Excel-based financial modeling tasks, graded by the GPT-6 Luna rubric judge with template scoring fixes Canonical URL: https://benchize.com/benchmarks/vals-ai%3Aemb-v1-1%3Acomps Last updated: Wed, 21 Oct 2026 18:52:20 GMT Reproduction and quotation must name Benchize, link to the canonical HTML URL, and retain individual source citations. Evaluating agents on Excel-based financial modeling tasks, graded by the GPT-6 Luna rubric judge with template scoring fixes Professional work · Models tested: 19 · Score publishers: Vals AI ## Model results | # | Model | Score | ± | Cost | Latency | | | --- | --- | --- | --- | --- | --- | --- | | 1 | [Gemini 4 Argon](https://benchize.com/models/gemini-4-argon-high.md)High | 82.94% | SE ±3.44 pp | $11.77 | 2,301.06source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 82.94 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 11.770794 \| USD \| per_test \| Lower is better \| \| `latency` \| 2301.06 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 262144 provider Google reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±3.44 pp sourcePopulation 30 stderr 3.441 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb_v1_1.11L06J7L.json) Source Vals AI Original model label google/gemini-4-argon Source version 1.1 Observed 2026-10-21T16:45:55.456Z Evidence SHA-256 `eea2ef2c83a6dc7d553c7c6531c287710ea7ffbc56ee1ccec06c85b104b2408e` Source locator `/tasks/comps/google~1gemini-4-argon` | | | | | | | | 2 | [Claude Fable 5.1](https://benchize.com/models/claude-fable-5-1-max.md)Max | 82% | SE ±4.08 pp | $24.6 | 2,943source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 81.998 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 24.604393 \| USD \| per_test \| Lower is better \| \| `latency` \| 2942.999 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±4.08 pp sourcePopulation 30 stderr 4.075 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb_v1_1.11L06J7L.json) Source Vals AI Original model label anthropic/claude-fable-5-1 Source version 1.1 Observed 2026-10-21T16:45:55.456Z Evidence SHA-256 `eea2ef2c83a6dc7d553c7c6531c287710ea7ffbc56ee1ccec06c85b104b2408e` Source locator `/tasks/comps/anthropic~1claude-fable-5-1` | | | | | | | | 3 | [Claude Sonnet 5.5](https://benchize.com/models/claude-sonnet-5-5-max.md)Max | 79.41% | SE ±4.22 pp | $10.25 | 2,463.28source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 79.406 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 10.248719 \| USD \| per_test \| Lower is better \| \| `latency` \| 2463.278 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±4.22 pp sourcePopulation 30 stderr 4.218 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb_v1_1.11L06J7L.json) Source Vals AI Original model label anthropic/claude-sonnet-5-5 Source version 1.1 Observed 2026-10-21T16:45:55.456Z Evidence SHA-256 `eea2ef2c83a6dc7d553c7c6531c287710ea7ffbc56ee1ccec06c85b104b2408e` Source locator `/tasks/comps/anthropic~1claude-sonnet-5-5` | | | | | | | | 4 | [Muse Wonder](https://benchize.com/models/muse-wonder-max.md)Max | 77.9% | SE ±4.38 pp | $6.19 | 996.37source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 77.902 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 6.189429 \| USD \| per_test \| Lower is better \| \| `latency` \| 996.365 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 131072 provider Meta reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±4.38 pp sourcePopulation 30 stderr 4.38 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb_v1_1.11L06J7L.json) Source Vals AI Original model label meta/muse-wonder Source version 1.1 Observed 2026-10-21T16:45:55.456Z Evidence SHA-256 `eea2ef2c83a6dc7d553c7c6531c287710ea7ffbc56ee1ccec06c85b104b2408e` Source locator `/tasks/comps/meta~1muse-wonder` | | | | | | | | 5 | [Claude Haiku 5.5](https://benchize.com/models/claude-haiku-5-5-max.md)Max | 77.7% | SE ±4.24 pp | $1.77 | 1,326.93source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 77.701 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 1.766809 \| USD \| per_test \| Lower is better \| \| `latency` \| 1326.925 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±4.24 pp sourcePopulation 30 stderr 4.238 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb_v1_1.11L06J7L.json) Source Vals AI Original model label anthropic/claude-haiku-5-5 Source version 1.1 Observed 2026-10-21T16:45:55.456Z Evidence SHA-256 `eea2ef2c83a6dc7d553c7c6531c287710ea7ffbc56ee1ccec06c85b104b2408e` Source locator `/tasks/comps/anthropic~1claude-haiku-5-5` | | | | | | | | 6 | [GPT-6 Astra](https://benchize.com/models/gpt-6-astra-max.md)Max | 76.52% | SE ±3.87 pp | $12.62 | 1,532source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 76.524 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 12.624116 \| USD \| per_test \| Lower is better \| \| `latency` \| 1532.002 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±3.87 pp sourcePopulation 30 stderr 3.865 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb_v1_1.11L06J7L.json) Source Vals AI Original model label openai/gpt-6-astra Source version 1.1 Observed 2026-10-21T16:45:55.456Z Evidence SHA-256 `eea2ef2c83a6dc7d553c7c6531c287710ea7ffbc56ee1ccec06c85b104b2408e` Source locator `/tasks/comps/openai~1gpt-6-astra` | | | | | | | | 7 | [Gemini 3.8 Flash](https://benchize.com/models/gemini-3-8-flash-high.md)High | 76.03% | SE ±4.26 pp | $9.1 | 667.12source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 76.029 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 9.096536 \| USD \| per_test \| Lower is better \| \| `latency` \| 667.116 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±4.26 pp sourcePopulation 30 stderr 4.257 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb_v1_1.11L06J7L.json) Source Vals AI Original model label google/gemini-3.8-flash Source version 1.1 Observed 2026-10-21T16:45:55.456Z Evidence SHA-256 `eea2ef2c83a6dc7d553c7c6531c287710ea7ffbc56ee1ccec06c85b104b2408e` Source locator `/tasks/comps/google~1gemini-3.8-flash` | | | | | | | | 8 | [Claude Opus 5.5](https://benchize.com/models/claude-opus-5-5-max.md)Max | 75.66% | SE ±6.46 pp | $20.51 | 4,242.93source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 75.658 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 20.510975 \| USD \| per_test \| Lower is better \| \| `latency` \| 4242.927 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±6.46 pp sourcePopulation 30 stderr 6.46 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb_v1_1.11L06J7L.json) Source Vals AI Original model label anthropic/claude-opus-5-5 Source version 1.1 Observed 2026-10-21T16:45:55.456Z Evidence SHA-256 `eea2ef2c83a6dc7d553c7c6531c287710ea7ffbc56ee1ccec06c85b104b2408e` Source locator `/tasks/comps/anthropic~1claude-opus-5-5` | | | | | | | | 9 | [MiniMax M3.1 Flash](https://benchize.com/models/minimax-m3-1-flash-max.md)Max | 71.58% | SE ±3.54 pp | $0.26 | 770.52source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 71.581 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.255073 \| USD \| per_test \| Lower is better \| \| `latency` \| 770.52 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 262144 provider MiniMax reasoning Not reported Reasoning effort max Temperature 1 topP 0.95 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±3.54 pp sourcePopulation 30 stderr 3.537 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb_v1_1.11L06J7L.json) Source Vals AI Original model label minimax/MiniMax-M3.1-flash Source version 1.1 Observed 2026-10-21T16:45:55.456Z Evidence SHA-256 `eea2ef2c83a6dc7d553c7c6531c287710ea7ffbc56ee1ccec06c85b104b2408e` Source locator `/tasks/comps/minimax~1MiniMax-M3.1-flash` | | | | | | | | 10 | [DeepSeek V4.1 Flash 1017](https://benchize.com/models/deepseek-v4-1-flash-1017-max.md)Max | 70.77% | SE ±4.34 pp | $0.33 | 790.73source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 70.771 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.331686 \| USD \| per_test \| Lower is better \| \| `latency` \| 790.726 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±4.34 pp sourcePopulation 30 stderr 4.335 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb_v1_1.11L06J7L.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash-1017 Source version 1.1 Observed 2026-10-21T16:45:55.456Z Evidence SHA-256 `eea2ef2c83a6dc7d553c7c6531c287710ea7ffbc56ee1ccec06c85b104b2408e` Source locator `/tasks/comps/deepseek~1deepseek-v4.1-flash-1017` | | | | | | | | 11 | [GPT-6 Luna](https://benchize.com/models/gpt-6-luna-max.md)Max | 69.67% | SE ±6.12 pp | $0.37 | 1,820.73source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 69.668 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.373356 \| USD \| per_test \| Lower is better \| \| `latency` \| 1820.73 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±6.12 pp sourcePopulation 30 stderr 6.122 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb_v1_1.11L06J7L.json) Source Vals AI Original model label openai/gpt-6-luna Source version 1.1 Observed 2026-10-21T16:45:55.456Z Evidence SHA-256 `eea2ef2c83a6dc7d553c7c6531c287710ea7ffbc56ee1ccec06c85b104b2408e` Source locator `/tasks/comps/openai~1gpt-6-luna` | | | | | | | | 12 | [MiMo-V2.6-Flash](https://benchize.com/models/mimo-v2-6-flash.md) | 69.27% | SE ±5.98 pp | $0.11 | 1,230.32source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 69.274 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.110268 \| USD \| per_test \| Lower is better \| \| `latency` \| 1230.32 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider Xiaomi reasoning Not reported Reasoning effort unknown Temperature 1 topP 0.95 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±5.98 pp sourcePopulation 30 stderr 5.975 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb_v1_1.11L06J7L.json) Source Vals AI Original model label xiaomi/mimo-v2.6-flash Source version 1.1 Observed 2026-10-21T16:45:55.456Z Evidence SHA-256 `eea2ef2c83a6dc7d553c7c6531c287710ea7ffbc56ee1ccec06c85b104b2408e` Source locator `/tasks/comps/xiaomi~1mimo-v2.6-flash` | | | | | | | | 13 | [DeepSeek V4.1 Flash](https://benchize.com/models/deepseek-v4-1-flash-max.md)Max | 67.99% | SE ±5.6 pp | $0.22 | 630.8source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 67.988 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.218978 \| USD \| per_test \| Lower is better \| \| `latency` \| 630.8 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±5.6 pp sourcePopulation 30 stderr 5.6 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb_v1_1.11L06J7L.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash Source version 1.1 Observed 2026-10-21T16:45:55.456Z Evidence SHA-256 `eea2ef2c83a6dc7d553c7c6531c287710ea7ffbc56ee1ccec06c85b104b2408e` Source locator `/tasks/comps/deepseek~1deepseek-v4.1-flash` | | | | | | | | 14 | [MiMo-V2.6-Pro](https://benchize.com/models/mimo-v2-6-pro.md) | 67.98% | SE ±6.39 pp | $0.37 | 2,820.67source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 67.982 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.373493 \| USD \| per_test \| Lower is better \| \| `latency` \| 2820.674 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider Xiaomi reasoning Not reported Reasoning effort unknown Temperature 1 topP 0.95 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±6.39 pp sourcePopulation 30 stderr 6.386 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb_v1_1.11L06J7L.json) Source Vals AI Original model label xiaomi/mimo-v2.6-pro Source version 1.1 Observed 2026-10-21T16:45:55.456Z Evidence SHA-256 `eea2ef2c83a6dc7d553c7c6531c287710ea7ffbc56ee1ccec06c85b104b2408e` Source locator `/tasks/comps/xiaomi~1mimo-v2.6-pro` | | | | | | | | 15 | [Step 5 Preview](https://benchize.com/models/step-5-preview.md) | 66.84% | SE ±4.3 pp | $1.34 | 2,163.54source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 66.843 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 1.341743 \| USD \| per_test \| Lower is better \| \| `latency` \| 2163.542 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 1024000 provider Stepfun reasoning Not reported Reasoning effort unknown Temperature 1 topP 0.95 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±4.3 pp sourcePopulation 30 stderr 4.301 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb_v1_1.11L06J7L.json) Source Vals AI Original model label stepfun/step-5-preview Source version 1.1 Observed 2026-10-21T16:45:55.456Z Evidence SHA-256 `eea2ef2c83a6dc7d553c7c6531c287710ea7ffbc56ee1ccec06c85b104b2408e` Source locator `/tasks/comps/stepfun~1step-5-preview` | | | | | | | | 16 | [GPT-6.1 Sol](https://benchize.com/models/gpt-6-1-sol-max.md)Max | 61.23% | SE ±8.55 pp | $2.74 | 1,969.17source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 61.233 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 2.739501 \| USD \| per_test \| Lower is better \| \| `latency` \| 1969.169 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±8.55 pp sourcePopulation 30 stderr 8.554 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb_v1_1.11L06J7L.json) Source Vals AI Original model label openai/gpt-6.1-sol Source version 1.1 Observed 2026-10-21T16:45:55.456Z Evidence SHA-256 `eea2ef2c83a6dc7d553c7c6531c287710ea7ffbc56ee1ccec06c85b104b2408e` Source locator `/tasks/comps/openai~1gpt-6.1-sol` | | | | | | | | 17 | [MiniMax M3](https://benchize.com/models/minimax-m3.md) | 60.11% | SE ±4.39 pp | $4.77 | 946.91source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 60.114 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 4.769582 \| USD \| per_test \| Lower is better \| \| `latency` \| 946.911 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 512000 provider MiniMax reasoning Not reported Reasoning effort unknown Temperature 1 topP 0.95 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±4.39 pp sourcePopulation 30 stderr 4.388 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb_v1_1.11L06J7L.json) Source Vals AI Original model label minimax/MiniMax-M3 Source version 1.1 Observed 2026-10-21T16:45:55.456Z Evidence SHA-256 `eea2ef2c83a6dc7d553c7c6531c287710ea7ffbc56ee1ccec06c85b104b2408e` Source locator `/tasks/comps/minimax~1MiniMax-M3` | | | | | | | | 18 | [DeepSeek V4 Pro 0813](https://benchize.com/models/deepseek-v4-pro-0813-max.md)Max | 59.31% | SE ±5.25 pp | $1.07 | 1,746.6source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 59.312 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 1.073792 \| USD \| per_test \| Lower is better \| \| `latency` \| 1746.604 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±5.25 pp sourcePopulation 30 stderr 5.247 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb_v1_1.11L06J7L.json) Source Vals AI Original model label deepseek/deepseek-v4-pro-0813 Source version 1.1 Observed 2026-10-21T16:45:55.456Z Evidence SHA-256 `eea2ef2c83a6dc7d553c7c6531c287710ea7ffbc56ee1ccec06c85b104b2408e` Source locator `/tasks/comps/deepseek~1deepseek-v4-pro-0813` | | | | | | | | 19 | [Muse Spark Open](https://benchize.com/models/muse-spark-open-xhigh.md)Extra high | 56.04% | SE ±6.92 pp | $6.33 | 4,497.27source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 56.044 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 6.33156 \| USD \| per_test \| Lower is better \| \| `latency` \| 4497.271 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 131072 provider Meta reasoning Not reported Reasoning effort xhigh Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±6.92 pp sourcePopulation 30 stderr 6.916 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb_v1_1.11L06J7L.json) Source Vals AI Original model label meta/muse-spark-open Source version 1.1 Observed 2026-10-21T16:45:55.456Z Evidence SHA-256 `eea2ef2c83a6dc7d553c7c6531c287710ea7ffbc56ee1ccec06c85b104b2408e` Source locator `/tasks/comps/meta~1muse-spark-open` | | | | | | | --- Cite as: EMB v1.1 / comps — Benchize, https://benchize.com/benchmarks/vals-ai%3Aemb-v1-1%3Acomps (retrieved 2026-10-21)