# Muse Spark 1.1Extra high > Source: Benchize (https://benchize.com) — https://benchize.com/models/muse-spark-1-1-xhigh. When citing or reusing this content, credit "Benchize" and link to https://benchize.com/models/muse-spark-1-1-xhigh. Meta Canonical URL: https://benchize.com/models/muse-spark-1-1-xhigh Last updated: Wed, 21 Oct 2026 18:52:20 GMT Reproduction and quotation must name Benchize, link to the canonical HTML URL, and retain individual source citations. Meta ## Benchmarks | Benchmark | Score | ± | Rank | Cost | Latency | Source | | | --- | --- | --- | --- | --- | --- | --- | --- | | [SWE-bench / <15 min fix](https://benchize.com/benchmarks/vals-ai%3Aswebench%3A%3C15%20min%20fix.md) | 88.14% | SE ±2.33 pp | 15 / 39 | $0.24 | 151.06source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_swebench.Ae8ni7h0.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 88.144 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.235316 \| USD \| per_test \| Lower is better \| \| `latency` \| 151.063 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider Meta reasoning Not reported Reasoning effort xhigh Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.33 pp sourcePopulation 88 stderr 2.327 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_swebench.Ae8ni7h0.json) Source Vals AI Original model label meta/muse_spark_1_1 Source version 1 Observed 2026-10-21T16:46:13.711Z Evidence SHA-256 `7a4e2e8485283caac500127419f68ec4bd5414c837ade114fc6afa5b2d34ccaa` Source locator `/tasks/<15 min fix/meta~1muse_spark_1_1` | | | | | | | | | [ProgramBench / partial](https://benchize.com/benchmarks/vals-ai%3Aprogrambench%3Apartial.md) | 27.12% | SE ±2.37 pp | 35 / 40 | $13.06 | 20,592.05source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_programbench.C_J7ZU4X.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 27.122 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 13.063498 \| USD \| per_test \| Lower is better \| \| `latency` \| 20592.047 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider Meta reasoning Not reported Reasoning effort xhigh Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.37 pp sourcePopulation 64 stderr 2.37 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_programbench.C_J7ZU4X.json) Source Vals AI Original model label meta/muse_spark_1_1 Source version 1 Observed 2026-10-21T16:46:10.137Z Evidence SHA-256 `e9bbb4378c7a058441a21d456baf9316b96c2cf5b58d371a3c7c803f5000bd8c` Source locator `/tasks/partial/meta~1muse_spark_1_1` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / bankruptcy restructuring](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Abankruptcy_restructuring.md) | 20% | SE ±20 pp | 2 / 49 | $0.88 | 838.64source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 20 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.878241 \| USD \| per_test \| Lower is better \| \| `latency` \| 838.64 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider Meta reasoning Not reported Reasoning effort xhigh Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±20 pp sourcePopulation 81 stderr 20 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label meta/muse_spark_1_1 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/bankruptcy_restructuring/meta~1muse_spark_1_1` | | | | | | | | | [SWE-bench / >4 hours](https://benchize.com/benchmarks/vals-ai%3Aswebench%3A%3E4%20hours.md) | 66.67% | SE ±33.33 pp | 6 / 39 | $0.68 | 518.43source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_swebench.Ae8ni7h0.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 66.667 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.676282 \| USD \| per_test \| Lower is better \| \| `latency` \| 518.434 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider Meta reasoning Not reported Reasoning effort xhigh Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±33.33 pp sourcePopulation 88 stderr 33.333 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_swebench.Ae8ni7h0.json) Source Vals AI Original model label meta/muse_spark_1_1 Source version 1 Observed 2026-10-21T16:46:13.711Z Evidence SHA-256 `7a4e2e8485283caac500127419f68ec4bd5414c837ade114fc6afa5b2d34ccaa` Source locator `/tasks/>4 hours/meta~1muse_spark_1_1` | | | | | | | | | [Vals Multimodal Index / mortgage tax](https://benchize.com/benchmarks/vals-ai%3Avals_multimodal_index%3Amortgage_tax.md) | 66.14% | SE ±0.93 pp | 17 / 20 | $0.02 | 24.82source duration | [Vals AI · 1.2](https://www.vals.ai/_astro/benchmark_view_vals_multimodal_index.virVpWMs.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 66.136 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.022312 \| USD \| per_test \| Lower is better \| \| `latency` \| 24.818 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider Meta reasoning Not reported Reasoning effort xhigh Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.93 pp sourcePopulation 33 stderr 0.928 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_vals_multimodal_index.virVpWMs.json) Source Vals AI Original model label meta/muse_spark_1_1 Source version 1.2 Observed 2026-10-21T16:45:51.881Z Evidence SHA-256 `077b22575f6e075b7b78157d000ec8f94d7ead409c7dcbcf04887630ed572a2d` Source locator `/tasks/mortgage_tax/meta~1muse_spark_1_1` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / criteria pass rate tax](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__tax.md) | 91.3% | SE ±1.63 pp | 3 / 49 | $0.71 | 698.21source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 91.304 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.714588 \| USD \| per_test \| Lower is better \| \| `latency` \| 698.208 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider Meta reasoning Not reported Reasoning effort xhigh Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.63 pp sourcePopulation 81 stderr 1.632 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label meta/muse_spark_1_1 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__tax/meta~1muse_spark_1_1` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / employment labor](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Aemployment_labor.md) | 0% | SE ±0 pp | 1 / 49 | $0.59 | 452.27source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.589655 \| USD \| per_test \| Lower is better \| \| `latency` \| 452.27 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider Meta reasoning Not reported Reasoning effort xhigh Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 81 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label meta/muse_spark_1_1 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/employment_labor/meta~1muse_spark_1_1` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / criteria pass rate arbitration international dispute resolution](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__arbitration_international_dispute_resolution.md) | 94.27% | SE ±1.39 pp | 2 / 49 | $0.97 | 774.5source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 94.265 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.973952 \| USD \| per_test \| Lower is better \| \| `latency` \| 774.504 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider Meta reasoning Not reported Reasoning effort xhigh Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.39 pp sourcePopulation 81 stderr 1.394 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label meta/muse_spark_1_1 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__arbitration_international_dispute_resolution/meta~1muse_spark_1_1` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / criteria pass rate emerging companies venture capital](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__emerging_companies_venture_capital.md) | 91.81% | SE ±1.61 pp | 10 / 49 | $0.75 | 798.34source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 91.809 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.754855 \| USD \| per_test \| Lower is better \| \| `latency` \| 798.338 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider Meta reasoning Not reported Reasoning effort xhigh Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.61 pp sourcePopulation 81 stderr 1.605 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label meta/muse_spark_1_1 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__emerging_companies_venture_capital/meta~1muse_spark_1_1` | | | | | | | | | [Legal Research Bench / health](https://benchize.com/benchmarks/vals-ai%3Alegal_research%3Ahealth.md) | 40% | SE ±9.1 pp | 35 / 50 | $0.51 | 298.42source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 40 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.506452 \| USD \| per_test \| Lower is better \| \| `latency` \| 298.421 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider Meta reasoning Not reported Reasoning effort xhigh Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±9.1 pp sourcePopulation 82 stderr 9.097 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) Source Vals AI Original model label meta/muse_spark_1_1 Source version 1 Observed 2026-10-21T16:45:52.389Z Evidence SHA-256 `26c58dfd47d0d2dc86f33c6d056f30309e0ee63570e7cd752c593b385f9d862e` Source locator `/tasks/health/meta~1muse_spark_1_1` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / criteria pass rate banking finance](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__banking_finance.md) | 94.8% | SE ±1.36 pp | 3 / 49 | $0.66 | 553.18source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 94.796 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.664525 \| USD \| per_test \| Lower is better \| \| `latency` \| 553.184 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider Meta reasoning Not reported Reasoning effort xhigh Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.36 pp sourcePopulation 81 stderr 1.357 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label meta/muse_spark_1_1 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__banking_finance/meta~1muse_spark_1_1` | | | | | | | | | [Public Benefits Bench v1.1 / overall](https://benchize.com/benchmarks/vals-ai%3Apublic-benefits-bench%3Aoverall.md) | 61.3% | SE ±1.27 pp | 21 / 34 | $0.28 | 395.05source duration | [Vals AI · 1.1](https://www.vals.ai/_astro/benchmark_view_snap.CUcEPb0v.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 61.299 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.278428 \| USD \| per_test \| Lower is better \| \| `latency` \| 395.054 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider Meta reasoning Not reported Reasoning effort xhigh Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.27 pp sourcePopulation 50 stderr 1.267 tokenTotals {"cache_read_tokens":199788960,"cache_write_tokens":null,"input_tokens":213360122,"output_tokens":4024960,"reasoning_tokens":929960} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_snap.CUcEPb0v.json) Source Vals AI Original model label meta/muse_spark_1_1 Source version 1.1 Observed 2026-10-21T16:46:17.865Z Evidence SHA-256 `6f091831ab607a26d65d2ade18cba7ce345ea3a67297b8c378e8a06ce398360d` Source locator `/tasks/overall/meta~1muse_spark_1_1` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / criteria pass rate bankruptcy restructuring](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__bankruptcy_restructuring.md) | 92.04% | SE ±1.6 pp | 3 / 49 | $0.88 | 838.64source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 92.042 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.878241 \| USD \| per_test \| Lower is better \| \| `latency` \| 838.64 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider Meta reasoning Not reported Reasoning effort xhigh Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.6 pp sourcePopulation 81 stderr 1.595 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label meta/muse_spark_1_1 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__bankruptcy_restructuring/meta~1muse_spark_1_1` | | | | | | | | | [Vals Multimodal Index / terminal bench 2 1](https://benchize.com/benchmarks/vals-ai%3Avals_multimodal_index%3Aterminal_bench_2_1.md) | 69.29% | SE ±0.99 pp | 13 / 20 | $0.42 | 624.77source duration | [Vals AI · 1.2](https://www.vals.ai/_astro/benchmark_view_vals_multimodal_index.virVpWMs.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 69.288 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.417396 \| USD \| per_test \| Lower is better \| \| `latency` \| 624.772 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider Meta reasoning Not reported Reasoning effort xhigh Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.99 pp sourcePopulation 33 stderr 0.991 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_vals_multimodal_index.virVpWMs.json) Source Vals AI Original model label meta/muse_spark_1_1 Source version 1.2 Observed 2026-10-21T16:45:51.881Z Evidence SHA-256 `077b22575f6e075b7b78157d000ec8f94d7ead409c7dcbcf04887630ed572a2d` Source locator `/tasks/terminal_bench_2_1/meta~1muse_spark_1_1` | | | | | | | | | [Terminal-Bench 2.1 / overall](https://benchize.com/benchmarks/vals-ai%3Aterminal-bench-2-1%3Aoverall.md) | 69.29% | SE ±0.99 pp | 24 / 43 | $0.42 | 624.77source duration | [Vals AI · 2.1](https://www.vals.ai/_astro/benchmark_view_terminal_bench_2_1.CdyoObsA.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 69.288 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.417396 \| USD \| per_test \| Lower is better \| \| `latency` \| 624.772 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider Meta reasoning Not reported Reasoning effort xhigh Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.99 pp sourcePopulation 76 stderr 0.991 tokenTotals {"cache_read_tokens":136207264,"cache_write_tokens":0,"input_tokens":138955979,"output_tokens":3124998,"reasoning_tokens":1933296} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_terminal_bench_2_1.CdyoObsA.json) Source Vals AI Original model label meta/muse_spark_1_1 Source version 2.1 Observed 2026-10-21T16:46:13.216Z Evidence SHA-256 `a5306805cdf86495bdcceff6b2c8534a3768b9853f5077895dab975b809b054f` Source locator `/tasks/overall/meta~1muse_spark_1_1` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / corporate ma](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acorporate_ma.md) | 20% | SE ±20 pp | 1 / 49 | $0.85 | 778.88source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 20 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.847903 \| USD \| per_test \| Lower is better \| \| `latency` \| 778.878 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider Meta reasoning Not reported Reasoning effort xhigh Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±20 pp sourcePopulation 81 stderr 20 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label meta/muse_spark_1_1 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/corporate_ma/meta~1muse_spark_1_1` | | | | | | | | | [SWE-bench / 15 min - 1 hour](https://benchize.com/benchmarks/vals-ai%3Aswebench%3A15%20min%20-%201%20hour.md) | 82.38% | SE ±2.36 pp | 10 / 39 | $0.38 | 227.78source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_swebench.Ae8ni7h0.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 82.375 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.381795 \| USD \| per_test \| Lower is better \| \| `latency` \| 227.782 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider Meta reasoning Not reported Reasoning effort xhigh Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.36 pp sourcePopulation 88 stderr 2.363 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_swebench.Ae8ni7h0.json) Source Vals AI Original model label meta/muse_spark_1_1 Source version 1 Observed 2026-10-21T16:46:13.711Z Evidence SHA-256 `7a4e2e8485283caac500127419f68ec4bd5414c837ade114fc6afa5b2d34ccaa` Source locator `/tasks/15 min - 1 hour/meta~1muse_spark_1_1` | | | | | | | | | [MMLU Pro / history](https://benchize.com/benchmarks/vals-ai%3Ammlu_pro%3Ahistory.md) | 79.79% | SE ±2.06 pp | 17 / 40 | $0.00874 | 16.33source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_mmlu_pro.y416pXDi.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 79.79 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.00874 \| USD \| per_test \| Lower is better \| \| `latency` \| 16.33 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider Meta reasoning Not reported Reasoning effort xhigh Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.06 pp sourcePopulation 138 stderr 2.057 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_mmlu_pro.y416pXDi.json) Source Vals AI Original model label meta/muse_spark_1_1 Source version 1 Observed 2026-10-21T16:46:06.503Z Evidence SHA-256 `dab025507f46c716dd8d145e90ec0a737d9c56219853f41e4b1cc3c5e8c222b4` Source locator `/tasks/history/meta~1muse_spark_1_1` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / criteria pass rate intellectual property](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__intellectual_property.md) | 95.41% | SE ±1.25 pp | 3 / 49 | $0.7 | 561.32source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 95.406 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.701418 \| USD \| per_test \| Lower is better \| \| `latency` \| 561.322 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider Meta reasoning Not reported Reasoning effort xhigh Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.25 pp sourcePopulation 81 stderr 1.247 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label meta/muse_spark_1_1 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__intellectual_property/meta~1muse_spark_1_1` | | | | | | | | | [SkillsBench / overall](https://benchize.com/benchmarks/vals-ai%3Askillsbench%3Aoverall.md) | 59.19% | SE ±4.44 pp | 8 / 20 | $0.45 | 376.27source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_skillsbench.BE2TXB-M.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 59.189 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.449432 \| USD \| per_test \| Lower is better \| \| `latency` \| 376.268 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness OpenHands maxOutputTokens 256000 provider Meta reasoning Not reported Reasoning effort xhigh Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±4.44 pp sourcePopulation 35 stderr 4.444 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_skillsbench.BE2TXB-M.json) Source Vals AI Original model label meta/muse_spark_1_1 Source version 1 Observed 2026-10-21T16:46:11.147Z Evidence SHA-256 `aaad242949d81b7c7c61c36d4895d4460e9146760c1e6b4b38d235a16043bc61` Source locator `/tasks/overall/meta~1muse_spark_1_1` | | | | | | | | | [MMLU Pro / psychology](https://benchize.com/benchmarks/vals-ai%3Ammlu_pro%3Apsychology.md) | 87.22% | SE ±1.19 pp | 18 / 40 | $0.00774 | 15.16source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_mmlu_pro.y416pXDi.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 87.218 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.007738 \| USD \| per_test \| Lower is better \| \| `latency` \| 15.16 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider Meta reasoning Not reported Reasoning effort xhigh Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.19 pp sourcePopulation 138 stderr 1.187 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_mmlu_pro.y416pXDi.json) Source Vals AI Original model label meta/muse_spark_1_1 Source version 1 Observed 2026-10-21T16:46:06.503Z Evidence SHA-256 `dab025507f46c716dd8d145e90ec0a737d9c56219853f41e4b1cc3c5e8c222b4` Source locator `/tasks/psychology/meta~1muse_spark_1_1` | | | | | | | | | [Time Horizon Index: KSP / overall](https://benchize.com/benchmarks/vals-ai%3Atime_horizon_index%3Aoverall.md) | 1.67% | SE ±0 pp | 10 / 10 | $144.92 | 0source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_ksp_bench.CDDTSam7.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 1.667 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 144.921195 \| USD \| per_test \| Lower is better \| \| `latency` \| 0 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider Meta reasoning Not reported Reasoning effort xhigh Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 13 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_ksp_bench.CDDTSam7.json) Source Vals AI Original model label meta/muse_spark_1_1 Source version 1 Observed 2026-10-21T16:46:16.730Z Evidence SHA-256 `be251b72ecceae7912b9577f9b3a3512740c308a34f81ac823952e919adae50f` Source locator `/tasks/overall/meta~1muse_spark_1_1` | | | | | | | | | [MMLU Pro / physics](https://benchize.com/benchmarks/vals-ai%3Ammlu_pro%3Aphysics.md) | 93.84% | SE ±0.69 pp | 8 / 40 | $0.01 | 21.65source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_mmlu_pro.y416pXDi.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 93.841 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.011016 \| USD \| per_test \| Lower is better \| \| `latency` \| 21.651 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider Meta reasoning Not reported Reasoning effort xhigh Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.69 pp sourcePopulation 138 stderr 0.69 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_mmlu_pro.y416pXDi.json) Source Vals AI Original model label meta/muse_spark_1_1 Source version 1 Observed 2026-10-21T16:46:06.503Z Evidence SHA-256 `dab025507f46c716dd8d145e90ec0a737d9c56219853f41e4b1cc3c5e8c222b4` Source locator `/tasks/physics/meta~1muse_spark_1_1` | | | | | | | | | [τ-bench banking knowledge](https://benchize.com/benchmarks/sierra%3Abanking_knowledge.md) | 40.46% | — | 4 / 19 | — | — | [sierra](https://sierra-tau-bench-public.s3.us-west-2.amazonaws.com/submissions/muse-spark-1-1_sierra_2026-08-04/submission.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `pass_1` \| 40.4639175257732 \| percent \| consistent_success_across_k_trials \| Higher is better \| \| `pass_2` \| 29.55326460481099 \| percent \| consistent_success_across_k_trials \| Higher is better \| \| `pass_3` \| 23.969072164948454 \| percent \| consistent_success_across_k_trials \| Higher is better \| \| `pass_4` \| 20.618556701030926 \| percent \| consistent_success_across_k_trials \| Higher is better \| ### Configuration benchmarkVersion 1.0.1 harness tau-bench methodology {"evaluation_date":"2026-07-23","notes":"Banking_knowledge evaluation with AllTools retrieval, GPT-5.2 low-reasoning user simulation, four trials, and seed 300.","tau2_bench_version":"1.0.1","user_simulator":"gpt-5.2","verification":{"modified_prompts":false,"omitted_questions":false}} modality text Reasoning effort xhigh retrieval alltools sourceSubmission muse-spark-1-1_sierra_2026-08-04 ### Source [Original source](https://sierra-tau-bench-public.s3.us-west-2.amazonaws.com/submissions/muse-spark-1-1_sierra_2026-08-04/submission.json) Source sierra Original model label Muse Spark 1.1 Evaluated 2026-08-04 Observed 2026-10-21T16:54:52.457Z Evidence SHA-256 `5ec4d2b119542bb40358374e3408515dccf5f0b3dace60de7936a20cb9bf13da` Source locator `/results/banking_knowledge` | | | | | | | | | [Legal Research Bench / overall](https://benchize.com/benchmarks/vals-ai%3Alegal_research%3Aoverall.md) | 37.98% | SE ±3.37 pp | 27 / 50 | $0.38 | 312.99source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 37.981 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.377379 \| USD \| per_test \| Lower is better \| \| `latency` \| 312.994 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider Meta reasoning Not reported Reasoning effort xhigh Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±3.37 pp sourcePopulation 82 stderr 3.373 tokenTotals {"cache_read_tokens":94898720,"cache_write_tokens":0,"input_tokens":126240889,"output_tokens":5901711,"reasoning_tokens":3106623} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) Source Vals AI Original model label meta/muse_spark_1_1 Source version 1 Observed 2026-10-21T16:45:52.389Z Evidence SHA-256 `26c58dfd47d0d2dc86f33c6d056f30309e0ee63570e7cd752c593b385f9d862e` Source locator `/tasks/overall/meta~1muse_spark_1_1` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / white collar defense investigations](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Awhite_collar_defense_investigations.md) | 0% | SE ±0 pp | 1 / 49 | $0.74 | 532.96source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.739764 \| USD \| per_test \| Lower is better \| \| `latency` \| 532.962 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider Meta reasoning Not reported Reasoning effort xhigh Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 81 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label meta/muse_spark_1_1 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/white_collar_defense_investigations/meta~1muse_spark_1_1` | | | | | | | | | [MMLU Pro / biology](https://benchize.com/benchmarks/vals-ai%3Ammlu_pro%3Abiology.md) | 94.98% | SE ±0.9 pp | 4 / 40 | $0.00822 | 15.73source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_mmlu_pro.y416pXDi.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 94.979 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.008217 \| USD \| per_test \| Lower is better \| \| `latency` \| 15.731 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider Meta reasoning Not reported Reasoning effort xhigh Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.9 pp sourcePopulation 138 stderr 0.896 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_mmlu_pro.y416pXDi.json) Source Vals AI Original model label meta/muse_spark_1_1 Source version 1 Observed 2026-10-21T16:46:06.503Z Evidence SHA-256 `dab025507f46c716dd8d145e90ec0a737d9c56219853f41e4b1cc3c5e8c222b4` Source locator `/tasks/biology/meta~1muse_spark_1_1` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / immigration](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Aimmigration.md) | 20% | SE ±20 pp | 2 / 49 | $0.48 | 529.21source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 20 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.483627 \| USD \| per_test \| Lower is better \| \| `latency` \| 529.208 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider Meta reasoning Not reported Reasoning effort xhigh Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±20 pp sourcePopulation 81 stderr 20 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label meta/muse_spark_1_1 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/immigration/meta~1muse_spark_1_1` | | | | | | | | | [LegalBench / overall](https://benchize.com/benchmarks/vals-ai%3Alegal_bench%3Aoverall.md) | 84.98% | SE ±0.43 pp | 16 / 44 | $0.00854 | 17.15source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_bench.D6uTgA3b.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 84.981 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.008535 \| USD \| per_test \| Lower is better \| \| `latency` \| 17.148 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider Meta reasoning Not reported Reasoning effort xhigh Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.43 pp sourcePopulation 151 stderr 0.425 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_bench.D6uTgA3b.json) Source Vals AI Original model label meta/muse_spark_1_1 Source version 1 Observed 2026-10-21T16:45:53.390Z Evidence SHA-256 `c8486ba756baa5cd3955d035edff67440e8db23e69d62ad1a9e18d15b8c4bc52` Source locator `/tasks/overall/meta~1muse_spark_1_1` | | | | | | | | | [GPQA Diamond / diamond few shot cot](https://benchize.com/benchmarks/vals-ai%3Agpqa%3Adiamond_few_shot_cot.md) | 90.4% | SE ±2.62 pp | 16 / 40 | $0.03 | 67.93source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_gpqa.ghZZBpyU.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 90.404 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.030784 \| USD \| per_test \| Lower is better \| \| `latency` \| 67.934 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider Meta reasoning Not reported Reasoning effort xhigh Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.62 pp sourcePopulation 138 stderr 2.616 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_gpqa.ghZZBpyU.json) Source Vals AI Original model label meta/muse_spark_1_1 Source version 1 Observed 2026-10-21T16:46:06.040Z Evidence SHA-256 `951eb5224097733a9d09d06c14908ae2980e2e640e539f2c284877c5092d7077` Source locator `/tasks/diamond_few_shot_cot/meta~1muse_spark_1_1` | | | | | | | | | [SAGE / automata](https://benchize.com/benchmarks/vals-ai%3Asage%3Aautomata.md) | 28.17% | SE ±5.65 pp | 39 / 41 | $0.03 | 58.28source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_sage.RRwIIzO5.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 28.174 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.026942 \| USD \| per_test \| Lower is better \| \| `latency` \| 58.277 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 30000 provider Meta reasoning Not reported Reasoning effort xhigh Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±5.65 pp sourcePopulation 91 stderr 5.654 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_sage.RRwIIzO5.json) Source Vals AI Original model label meta/muse_spark_1_1 Source version 1 Observed 2026-10-21T16:46:07.515Z Evidence SHA-256 `cc5fb3fecdb0382d492ff167a42ad84b9c5c3883373c78ed87baaa972b8b6f7c` Source locator `/tasks/automata/meta~1muse_spark_1_1` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / criteria pass rate energy natural resources](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__energy_natural_resources.md) | 96.37% | SE ±1.08 pp | 5 / 49 | $1.22 | 1,031.51source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 96.37 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 1.224794 \| USD \| per_test \| Lower is better \| \| `latency` \| 1031.512 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider Meta reasoning Not reported Reasoning effort xhigh Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.08 pp sourcePopulation 81 stderr 1.076 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label meta/muse_spark_1_1 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__energy_natural_resources/meta~1muse_spark_1_1` | | | | | | | | | [LegalBench / conclusion tasks](https://benchize.com/benchmarks/vals-ai%3Alegal_bench%3Aconclusion_tasks.md) | 87.33% | SE ±0.74 pp | 30 / 44 | $0.00837 | 18.31source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_bench.D6uTgA3b.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 87.328 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.008371 \| USD \| per_test \| Lower is better \| \| `latency` \| 18.313 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider Meta reasoning Not reported Reasoning effort xhigh Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.74 pp sourcePopulation 151 stderr 0.742 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_bench.D6uTgA3b.json) Source Vals AI Original model label meta/muse_spark_1_1 Source version 1 Observed 2026-10-21T16:45:53.390Z Evidence SHA-256 `c8486ba756baa5cd3955d035edff67440e8db23e69d62ad1a9e18d15b8c4bc52` Source locator `/tasks/conclusion_tasks/meta~1muse_spark_1_1` | | | | | | | | | [APEX-Agents](https://benchize.com/benchmarks/mercor%3Aapex-agents%3A1.1.md) | 31.8% | — | 34 / 38 | — | — | [mercor · 1.1](https://www.mercor.com/apex/apex-agents-leaderboard/) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `pass_1` \| 31.8 \| percent \| task_completion_pass_at_1 \| Higher is better \| \| `mean_score` \| 46.2 \| percent \| rubric_mean \| Higher is better \| ### Configuration datasetUrl https://huggingface.co/datasets/mercor/apex-agents-v1.1 datasetVersion 1.1 harness loop_truncated_tools_agent Reasoning effort xhigh ### Reported statistics reportedErrors {"mean_score":5,"pass_1":5.1} sampleSize 819 ### Source [Original source](https://www.mercor.com/apex/apex-agents-leaderboard/) Source mercor Original model label Muse Spark 1.1 Source version 1.1 Observed 2026-10-21T16:33:27.509Z Evidence SHA-256 `359c569c3b0b62ab375895eabf3c45846fa45551d419d5c9d6a4c9117db78184` Source locator `/__NEXT_DATA__/props/pageProps/leaderboardData/47` | | | | | | | | | [Legal Research Bench / weighted pass administrative regulatory](https://benchize.com/benchmarks/vals-ai%3Alegal_research%3Aweighted_pass__administrative_regulatory.md) | 83.62% | SE ±2.85 pp | 33 / 50 | $0.38 | 282.39source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 83.623 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.382532 \| USD \| per_test \| Lower is better \| \| `latency` \| 282.392 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider Meta reasoning Not reported Reasoning effort xhigh Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.85 pp sourcePopulation 82 stderr 2.846 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) Source Vals AI Original model label meta/muse_spark_1_1 Source version 1 Observed 2026-10-21T16:45:52.389Z Evidence SHA-256 `26c58dfd47d0d2dc86f33c6d056f30309e0ee63570e7cd752c593b385f9d862e` Source locator `/tasks/weighted_pass__administrative_regulatory/meta~1muse_spark_1_1` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / criteria pass rate real estate](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__real_estate.md) | 95.63% | SE ±1.11 pp | 4 / 49 | $0.96 | 935source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 95.627 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.95827 \| USD \| per_test \| Lower is better \| \| `latency` \| 934.998 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider Meta reasoning Not reported Reasoning effort xhigh Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.11 pp sourcePopulation 81 stderr 1.106 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label meta/muse_spark_1_1 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__real_estate/meta~1muse_spark_1_1` | | | | | | | | | [Terminal-Bench 2.1 / medium](https://benchize.com/benchmarks/vals-ai%3Aterminal-bench-2-1%3Amedium.md) | 78.18% | SE ±1.05 pp | 19 / 43 | $0.31 | 1,010.99source duration | [Vals AI · 2.1](https://www.vals.ai/_astro/benchmark_view_terminal_bench_2_1.CdyoObsA.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 78.182 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.312504 \| USD \| per_test \| Lower is better \| \| `latency` \| 1010.994 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider Meta reasoning Not reported Reasoning effort xhigh Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.05 pp sourcePopulation 76 stderr 1.05 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_terminal_bench_2_1.CdyoObsA.json) Source Vals AI Original model label meta/muse_spark_1_1 Source version 2.1 Observed 2026-10-21T16:46:13.216Z Evidence SHA-256 `a5306805cdf86495bdcceff6b2c8534a3768b9853f5077895dab975b809b054f` Source locator `/tasks/medium/meta~1muse_spark_1_1` | | | | | | | | | [SWE-bench / 1-4 hours](https://benchize.com/benchmarks/vals-ai%3Aswebench%3A1-4%20hours.md) | 52.38% | SE ±7.8 pp | 14 / 39 | $0.68 | 339.17source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_swebench.Ae8ni7h0.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 52.381 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.682916 \| USD \| per_test \| Lower is better \| \| `latency` \| 339.169 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider Meta reasoning Not reported Reasoning effort xhigh Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±7.8 pp sourcePopulation 88 stderr 7.8 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_swebench.Ae8ni7h0.json) Source Vals AI Original model label meta/muse_spark_1_1 Source version 1 Observed 2026-10-21T16:46:13.711Z Evidence SHA-256 `7a4e2e8485283caac500127419f68ec4bd5414c837ade114fc6afa5b2d34ccaa` Source locator `/tasks/1-4 hours/meta~1muse_spark_1_1` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / criteria pass rate corporate governance](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__corporate_governance.md) | 96.92% | SE ±1.01 pp | 2 / 49 | $1.03 | 1,321.06source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 96.918 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 1.030941 \| USD \| per_test \| Lower is better \| \| `latency` \| 1321.064 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider Meta reasoning Not reported Reasoning effort xhigh Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.01 pp sourcePopulation 81 stderr 1.013 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label meta/muse_spark_1_1 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__corporate_governance/meta~1muse_spark_1_1` | | | | | | | | | [TaxEval v2 / stepwise reasoning](https://benchize.com/benchmarks/vals-ai%3Atax_eval_v2%3Astepwise_reasoning.md) | 92.56% | SE ±0.75 pp | 2 / 40 | $0.01 | 32.82source duration | [Vals AI · 2](https://www.vals.ai/_astro/benchmark_view_tax_eval_v2.B3yoffr8.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 92.559 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.014921 \| USD \| per_test \| Lower is better \| \| `latency` \| 32.816 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider Meta reasoning Not reported Reasoning effort xhigh Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.75 pp sourcePopulation 145 stderr 0.75 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_tax_eval_v2.B3yoffr8.json) Source Vals AI Original model label meta/muse_spark_1_1 Source version 2 Observed 2026-10-21T16:45:59.000Z Evidence SHA-256 `8b354a6ae848f5bb25a2e8b061b5f576d5b9e0c11b941dc1e6bed078e6529cd6` Source locator `/tasks/stepwise_reasoning/meta~1muse_spark_1_1` | | | | | | | | | [Vals Multimodal Index / overall](https://benchize.com/benchmarks/vals-ai%3Avals_multimodal_index%3Aoverall.md) | 66.74% | SE ±1.15 pp | 10 / 20 | $0.36 | 288.19source duration | [Vals AI · 1.2](https://www.vals.ai/_astro/benchmark_view_vals_multimodal_index.virVpWMs.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 66.742 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.363107 \| USD \| per_test \| Lower is better \| \| `latency` \| 288.194 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider Meta reasoning Not reported Reasoning effort xhigh Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.15 pp sourcePopulation 33 stderr 1.151 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_vals_multimodal_index.virVpWMs.json) Source Vals AI Original model label meta/muse_spark_1_1 Source version 1.2 Observed 2026-10-21T16:45:51.881Z Evidence SHA-256 `077b22575f6e075b7b78157d000ec8f94d7ead409c7dcbcf04887630ed572a2d` Source locator `/tasks/overall/meta~1muse_spark_1_1` | | | | | | | | | [MMLU-Pro](https://benchize.com/benchmarks/mmlu-pro.md) | 88.73% | SE ±0.32 pp | 12 / 40 | $0.01 | 21.03source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_mmlu_pro.y416pXDi.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 88.726 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.010704 \| USD \| per_test \| Lower is better \| \| `latency` \| 21.03 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider Meta reasoning Not reported Reasoning effort xhigh Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.32 pp sourcePopulation 138 stderr 0.318 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_mmlu_pro.y416pXDi.json) Source Vals AI Original model label meta/muse_spark_1_1 Source version 1 Observed 2026-10-21T16:46:06.503Z Evidence SHA-256 `dab025507f46c716dd8d145e90ec0a737d9c56219853f41e4b1cc3c5e8c222b4` Source locator `/tasks/overall/meta~1muse_spark_1_1` | | | | | | | | | [Legal Research Bench / weighted pass constitutional civil rights](https://benchize.com/benchmarks/vals-ai%3Alegal_research%3Aweighted_pass__constitutional_civil_rights.md) | 85.62% | SE ±2.66 pp | 18 / 50 | $0.3 | 310.44source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 85.619 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.303515 \| USD \| per_test \| Lower is better \| \| `latency` \| 310.439 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider Meta reasoning Not reported Reasoning effort xhigh Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.66 pp sourcePopulation 82 stderr 2.661 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) Source Vals AI Original model label meta/muse_spark_1_1 Source version 1 Observed 2026-10-21T16:45:52.389Z Evidence SHA-256 `26c58dfd47d0d2dc86f33c6d056f30309e0ee63570e7cd752c593b385f9d862e` Source locator `/tasks/weighted_pass__constitutional_civil_rights/meta~1muse_spark_1_1` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / antitrust competition](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Aantitrust_competition.md) | 40% | SE ±24.5 pp | 1 / 49 | $0.78 | 708.48source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 40 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.782306 \| USD \| per_test \| Lower is better \| \| `latency` \| 708.48 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider Meta reasoning Not reported Reasoning effort xhigh Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±24.5 pp sourcePopulation 81 stderr 24.495 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label meta/muse_spark_1_1 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/antitrust_competition/meta~1muse_spark_1_1` | | | | | | | | | [Legal Research Bench / weighted pass health](https://benchize.com/benchmarks/vals-ai%3Alegal_research%3Aweighted_pass__health.md) | 87.94% | SE ±2.58 pp | 34 / 50 | $0.51 | 298.42source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 87.936 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.506452 \| USD \| per_test \| Lower is better \| \| `latency` \| 298.421 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider Meta reasoning Not reported Reasoning effort xhigh Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.58 pp sourcePopulation 82 stderr 2.581 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) Source Vals AI Original model label meta/muse_spark_1_1 Source version 1 Observed 2026-10-21T16:45:52.389Z Evidence SHA-256 `26c58dfd47d0d2dc86f33c6d056f30309e0ee63570e7cd752c593b385f9d862e` Source locator `/tasks/weighted_pass__health/meta~1muse_spark_1_1` | | | | | | | | | [LiveCodeBench / overall](https://benchize.com/benchmarks/vals-ai%3Alcb%3Aoverall.md) | 85.91% | SE ±0.99 pp | 17 / 43 | $0.05 | 113.25source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_lcb.miyRUvVa.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 85.906 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.046497 \| USD \| per_test \| Lower is better \| \| `latency` \| 113.253 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider Meta reasoning Not reported Reasoning effort xhigh Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.99 pp sourcePopulation 143 stderr 0.992 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_lcb.miyRUvVa.json) Source Vals AI Original model label meta/muse_spark_1_1 Source version 1 Observed 2026-10-21T16:46:14.220Z Evidence SHA-256 `1a7c7dabc9d95a0182886b1edc42623987cac2095a0d407a5e334f37acdfa2f5` Source locator `/tasks/overall/meta~1muse_spark_1_1` | | | | | | | | | [MedScribe / overall](https://benchize.com/benchmarks/vals-ai%3Amedscribe%3Aoverall.md) | 88.89% | SE ±1.95 pp | 7 / 49 | $0.03 | 63.34source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_medscribe.BYf5a3C-.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 88.888 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.034628 \| USD \| per_test \| Lower is better \| \| `latency` \| 63.335 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 30000 provider Meta reasoning Not reported Reasoning effort xhigh Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.95 pp sourcePopulation 109 stderr 1.95 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_medscribe.BYf5a3C-.json) Source Vals AI Original model label meta/muse_spark_1_1 Source version 1 Observed 2026-10-21T16:46:00.004Z Evidence SHA-256 `271fc122ddc2b0602bd6c7947f65a9a7f0d8cab94f6dc27d62a9380a9317f9c0` Source locator `/tasks/overall/meta~1muse_spark_1_1` | | | | | | | | | [MMLU Pro / business](https://benchize.com/benchmarks/vals-ai%3Ammlu_pro%3Abusiness.md) | 94.55% | SE ±0.82 pp | 2 / 40 | $0.00989 | 20.04source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_mmlu_pro.y416pXDi.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 94.55 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.009893 \| USD \| per_test \| Lower is better \| \| `latency` \| 20.041 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider Meta reasoning Not reported Reasoning effort xhigh Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.82 pp sourcePopulation 138 stderr 0.817 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_mmlu_pro.y416pXDi.json) Source Vals AI Original model label meta/muse_spark_1_1 Source version 1 Observed 2026-10-21T16:46:06.503Z Evidence SHA-256 `dab025507f46c716dd8d145e90ec0a737d9c56219853f41e4b1cc3c5e8c222b4` Source locator `/tasks/business/meta~1muse_spark_1_1` | | | | | | | | | [EMB / dataroom](https://benchize.com/benchmarks/vals-ai%3Aemb%3Adataroom.md) | 78.62% | SE ±5.76 pp | 26 / 47 | $0.75 | 706.31source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 78.618 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.749924 \| USD \| per_test \| Lower is better \| \| `latency` \| 706.309 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider Meta reasoning Not reported Reasoning effort xhigh Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±5.76 pp sourcePopulation 76 stderr 5.76 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) Source Vals AI Original model label meta/muse_spark_1_1 Source version 1 Observed 2026-10-21T16:45:57.495Z Evidence SHA-256 `d807aee620ea47152a1fca0257a8dd12a1be4512c76cc1744c1ae9e285180918` Source locator `/tasks/dataroom/meta~1muse_spark_1_1` | | | | | | | | | [MMLU Pro / health](https://benchize.com/benchmarks/vals-ai%3Ammlu_pro%3Ahealth.md) | 83.37% | SE ±1.31 pp | 13 / 40 | $0.0099 | 19.77source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_mmlu_pro.y416pXDi.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 83.374 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.0099 \| USD \| per_test \| Lower is better \| \| `latency` \| 19.769 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider Meta reasoning Not reported Reasoning effort xhigh Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.31 pp sourcePopulation 138 stderr 1.309 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_mmlu_pro.y416pXDi.json) Source Vals AI Original model label meta/muse_spark_1_1 Source version 1 Observed 2026-10-21T16:46:06.503Z Evidence SHA-256 `dab025507f46c716dd8d145e90ec0a737d9c56219853f41e4b1cc3c5e8c222b4` Source locator `/tasks/health/meta~1muse_spark_1_1` | | | | | | | | | [CorpFin v2 / shared max context](https://benchize.com/benchmarks/vals-ai%3Acorp_fin_v2%3Ashared_max_context.md) | 73.08% | SE ±1.51 pp | 2 / 39 | $0.09 | 25.41source duration | [Vals AI · 2](https://www.vals.ai/_astro/benchmark_view_corp_fin_v2.CectKwl1.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 73.077 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.087456 \| USD \| per_test \| Lower is better \| \| `latency` \| 25.409 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider Meta reasoning Not reported Reasoning effort xhigh Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.51 pp sourcePopulation 134 stderr 1.514 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_corp_fin_v2.CectKwl1.json) Source Vals AI Original model label meta/muse_spark_1_1 Source version 2 Observed 2026-10-21T16:45:57.998Z Evidence SHA-256 `7cf2217ab3728d5a84bffce82aab9fc52d7f810b0cbe8c65b387df47f3d31c9b` Source locator `/tasks/shared_max_context/meta~1muse_spark_1_1` | | | | | | | | | [Finance Agent (v2) / overall](https://benchize.com/benchmarks/vals-ai%3Afabv2%3Aoverall.md) | 57.21% | SE ±0.79 pp | 13 / 50 | $0.72 | 414.77source duration | [Vals AI · 2](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 57.207 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.723391 \| USD \| per_test \| Lower is better \| \| `latency` \| 414.77 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider Meta reasoning Not reported Reasoning effort xhigh Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.79 pp sourcePopulation 80 stderr 0.789 tokenTotals {"cache_read_tokens":177594427,"cache_write_tokens":null,"input_tokens":363982665,"output_tokens":15279252,"reasoning_tokens":10574096} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) Source Vals AI Original model label meta/muse_spark_1_1 Source version 2 Observed 2026-10-21T16:45:57.035Z Evidence SHA-256 `f731b20a26c126725d125519f2b5841aa45254656dba859a9570d7ce7e5d8191` Source locator `/tasks/overall/meta~1muse_spark_1_1` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / criteria pass rate funds asset management](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__funds_asset_management.md) | 92.13% | SE ±1.46 pp | 2 / 49 | $1 | 1,163.57source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 92.128 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.99674 \| USD \| per_test \| Lower is better \| \| `latency` \| 1163.572 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider Meta reasoning Not reported Reasoning effort xhigh Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.46 pp sourcePopulation 81 stderr 1.456 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label meta/muse_spark_1_1 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__funds_asset_management/meta~1muse_spark_1_1` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / healthcare life sciences](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Ahealthcare_life_sciences.md) | 20% | SE ±20 pp | 5 / 49 | $0.99 | 1,001.37source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 20 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.991405 \| USD \| per_test \| Lower is better \| \| `latency` \| 1001.374 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider Meta reasoning Not reported Reasoning effort xhigh Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±20 pp sourcePopulation 81 stderr 20 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label meta/muse_spark_1_1 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/healthcare_life_sciences/meta~1muse_spark_1_1` | | | | | | | | | [EMB / dcf](https://benchize.com/benchmarks/vals-ai%3Aemb%3Adcf.md) | 46.33% | SE ±8.07 pp | 35 / 47 | $1 | 834.42source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 46.328 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.996697 \| USD \| per_test \| Lower is better \| \| `latency` \| 834.416 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider Meta reasoning Not reported Reasoning effort xhigh Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±8.07 pp sourcePopulation 76 stderr 8.065 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) Source Vals AI Original model label meta/muse_spark_1_1 Source version 1 Observed 2026-10-21T16:45:57.495Z Evidence SHA-256 `d807aee620ea47152a1fca0257a8dd12a1be4512c76cc1744c1ae9e285180918` Source locator `/tasks/dcf/meta~1muse_spark_1_1` | | | | | | | | | [MortgageTax / numerical extraction](https://benchize.com/benchmarks/vals-ai%3Amortgage_tax%3Anumerical_extraction.md) | 74.48% | SE ±1.23 pp | 20 / 31 | $0.03 | 30.7source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_mortgage_tax.DCd-oO5Y.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 74.483 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.025379 \| USD \| per_test \| Lower is better \| \| `latency` \| 30.7 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider Meta reasoning Not reported Reasoning effort xhigh Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.23 pp sourcePopulation 98 stderr 1.229 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_mortgage_tax.DCd-oO5Y.json) Source Vals AI Original model label meta/muse_spark_1_1 Source version 1 Observed 2026-10-21T16:45:58.500Z Evidence SHA-256 `e5c844561f97d119d66d70269e726ecfb68be79eae39acf14ead3e95422e648c` Source locator `/tasks/numerical_extraction/meta~1muse_spark_1_1` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / criteria pass rate white collar defense investigations](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__white_collar_defense_investigations.md) | 89.56% | SE ±1.78 pp | 5 / 49 | $0.74 | 532.96source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 89.562 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.739764 \| USD \| per_test \| Lower is better \| \| `latency` \| 532.962 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider Meta reasoning Not reported Reasoning effort xhigh Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.78 pp sourcePopulation 81 stderr 1.777 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label meta/muse_spark_1_1 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__white_collar_defense_investigations/meta~1muse_spark_1_1` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / criteria pass rate corporate ma](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__corporate_ma.md) | 96.15% | SE ±1.09 pp | 1 / 49 | $0.85 | 778.88source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 96.154 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.847903 \| USD \| per_test \| Lower is better \| \| `latency` \| 778.878 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider Meta reasoning Not reported Reasoning effort xhigh Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.09 pp sourcePopulation 81 stderr 1.09 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label meta/muse_spark_1_1 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__corporate_ma/meta~1muse_spark_1_1` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / overall](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Aoverall.md) | 20% | SE ±2.94 pp | 3 / 49 | $0.8 | 740.83source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 20 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.800257 \| USD \| per_test \| Lower is better \| \| `latency` \| 740.833 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider Meta reasoning Not reported Reasoning effort xhigh Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.94 pp sourcePopulation 81 stderr 2.943 tokenTotals {"cache_read_tokens":263652672,"cache_write_tokens":null,"input_tokens":283802188,"output_tokens":5292532,"reasoning_tokens":497732} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label meta/muse_spark_1_1 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/overall/meta~1muse_spark_1_1` | | | | | | | | | [Finance Agent (v2) / earnings analysis](https://benchize.com/benchmarks/vals-ai%3Afabv2%3Aearnings_analysis.md) | 76.2% | SE ±0.67 pp | 14 / 50 | $0.28 | 240.69source duration | [Vals AI · 2](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 76.197 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.282783 \| USD \| per_test \| Lower is better \| \| `latency` \| 240.689 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider Meta reasoning Not reported Reasoning effort xhigh Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.67 pp sourcePopulation 80 stderr 0.67 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) Source Vals AI Original model label meta/muse_spark_1_1 Source version 2 Observed 2026-10-21T16:45:57.035Z Evidence SHA-256 `f731b20a26c126725d125519f2b5841aa45254656dba859a9570d7ce7e5d8191` Source locator `/tasks/earnings_analysis/meta~1muse_spark_1_1` | | | | | | | | | [Vals Multimodal Index / sage](https://benchize.com/benchmarks/vals-ai%3Avals_multimodal_index%3Asage.md) | 45.58% | SE ±3.28 pp | 17 / 20 | $0.03 | 49.95source duration | [Vals AI · 1.2](https://www.vals.ai/_astro/benchmark_view_vals_multimodal_index.virVpWMs.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 45.58 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.025502 \| USD \| per_test \| Lower is better \| \| `latency` \| 49.948 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider Meta reasoning Not reported Reasoning effort xhigh Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±3.28 pp sourcePopulation 33 stderr 3.276 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_vals_multimodal_index.virVpWMs.json) Source Vals AI Original model label meta/muse_spark_1_1 Source version 1.2 Observed 2026-10-21T16:45:51.881Z Evidence SHA-256 `077b22575f6e075b7b78157d000ec8f94d7ead409c7dcbcf04887630ed572a2d` Source locator `/tasks/sage/meta~1muse_spark_1_1` | | | | | | | | | [SkillsBench / with skills](https://benchize.com/benchmarks/vals-ai%3Askillsbench%3Awith_skills.md) | 59.19% | SE ±4.44 pp | 8 / 20 | $0.45 | 376.27source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_skillsbench.BE2TXB-M.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 59.189 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.449432 \| USD \| per_test \| Lower is better \| \| `latency` \| 376.268 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness OpenHands maxOutputTokens 256000 provider Meta reasoning Not reported Reasoning effort xhigh Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±4.44 pp sourcePopulation 35 stderr 4.444 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_skillsbench.BE2TXB-M.json) Source Vals AI Original model label meta/muse_spark_1_1 Source version 1 Observed 2026-10-21T16:46:11.147Z Evidence SHA-256 `aaad242949d81b7c7c61c36d4895d4460e9146760c1e6b4b38d235a16043bc61` Source locator `/tasks/with_skills/meta~1muse_spark_1_1` | | | | | | | | | [Vibe Code Bench v1.1 / overall](https://benchize.com/benchmarks/vals-ai%3Avibe-code%3Aoverall.md) | 72.16% | SE ±3.33 pp | 29 / 59 | $0.92 | 615.43source duration | [Vals AI · 1.1](https://www.vals.ai/_astro/benchmark_view_vibe_code_bench.DcEBMbCA.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 72.158 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.923283 \| USD \| per_test \| Lower is better \| \| `latency` \| 615.428 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness OpenHands maxOutputTokens 256000 provider Meta reasoning Not reported Reasoning effort xhigh Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±3.33 pp sourcePopulation 116 stderr 3.327 tokenTotals {"cache_read_tokens":182918822,"cache_write_tokens":0,"input_tokens":213516750,"output_tokens":2218049,"reasoning_tokens":189712} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_vibe_code_bench.DcEBMbCA.json) Source Vals AI Original model label meta/muse_spark_1_1 Source version 1.1 Observed 2026-10-21T16:46:08.625Z Evidence SHA-256 `0e8c23f521a0f751c5592952620e197b376cade41c78578461b7aa5740a173e7` Source locator `/tasks/overall/meta~1muse_spark_1_1` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / criteria pass rate international trade sanctions](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__international_trade_sanctions.md) | 90% | SE ±4.29 pp | 10 / 49 | $0.36 | 372.4source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 90 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.355364 \| USD \| per_test \| Lower is better \| \| `latency` \| 372.4 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider Meta reasoning Not reported Reasoning effort xhigh Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±4.29 pp sourcePopulation 81 stderr 4.286 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label meta/muse_spark_1_1 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__international_trade_sanctions/meta~1muse_spark_1_1` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / corporate governance](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acorporate_governance.md) | 20% | SE ±20 pp | 3 / 49 | $1.03 | 1,321.06source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 20 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 1.030941 \| USD \| per_test \| Lower is better \| \| `latency` \| 1321.064 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider Meta reasoning Not reported Reasoning effort xhigh Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±20 pp sourcePopulation 81 stderr 20 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label meta/muse_spark_1_1 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/corporate_governance/meta~1muse_spark_1_1` | | | | | | | | | [MMLU Pro / philosophy](https://benchize.com/benchmarks/vals-ai%3Ammlu_pro%3Aphilosophy.md) | 87.17% | SE ±1.51 pp | 13 / 40 | $0.00922 | 18.41source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_mmlu_pro.y416pXDi.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 87.174 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.009215 \| USD \| per_test \| Lower is better \| \| `latency` \| 18.405 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider Meta reasoning Not reported Reasoning effort xhigh Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.51 pp sourcePopulation 138 stderr 1.507 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_mmlu_pro.y416pXDi.json) Source Vals AI Original model label meta/muse_spark_1_1 Source version 1 Observed 2026-10-21T16:46:06.503Z Evidence SHA-256 `dab025507f46c716dd8d145e90ec0a737d9c56219853f41e4b1cc3c5e8c222b4` Source locator `/tasks/philosophy/meta~1muse_spark_1_1` | | | | | | | | | [Code Migration / code quality](https://benchize.com/benchmarks/vals-ai%3Acode-migration%3Acode_quality.md) | 2.81% | SE ±0.09 pp | 39 / 47 | $4.56 | 2,351.31source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_code_migration.Dttzoqk3.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 2.805 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 4.560329 \| USD \| per_test \| Lower is better \| \| `latency` \| 2351.314 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider Meta reasoning Not reported Reasoning effort xhigh Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.09 pp sourcePopulation 79 stderr 0.085 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_code_migration.Dttzoqk3.json) Source Vals AI Original model label meta/muse_spark_1_1 Source version 1 Observed 2026-10-21T16:46:12.712Z Evidence SHA-256 `99194597f2c76c402195be47839409046e18bd30454616da6dd40b736a088fa4` Source locator `/tasks/code_quality/meta~1muse_spark_1_1` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / criteria pass rate immigration](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__immigration.md) | 93.21% | SE ±1.7 pp | 3 / 49 | $0.48 | 529.21source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 93.213 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.483627 \| USD \| per_test \| Lower is better \| \| `latency` \| 529.208 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider Meta reasoning Not reported Reasoning effort xhigh Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.7 pp sourcePopulation 81 stderr 1.696 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label meta/muse_spark_1_1 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__immigration/meta~1muse_spark_1_1` | | | | | | | | | [EMB / comps](https://benchize.com/benchmarks/vals-ai%3Aemb%3Acomps.md) | 52.72% | SE ±7.88 pp | 32 / 47 | $1.86 | 948.28source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 52.72 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 1.857162 \| USD \| per_test \| Lower is better \| \| `latency` \| 948.276 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider Meta reasoning Not reported Reasoning effort xhigh Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±7.88 pp sourcePopulation 76 stderr 7.879 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) Source Vals AI Original model label meta/muse_spark_1_1 Source version 1 Observed 2026-10-21T16:45:57.495Z Evidence SHA-256 `d807aee620ea47152a1fca0257a8dd12a1be4512c76cc1744c1ae9e285180918` Source locator `/tasks/comps/meta~1muse_spark_1_1` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / criteria pass rate litigation dispute resolution](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__litigation_dispute_resolution.md) | 93.63% | SE ±1.5 pp | 3 / 49 | $0.88 | 748.11source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 93.633 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.879933 \| USD \| per_test \| Lower is better \| \| `latency` \| 748.106 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider Meta reasoning Not reported Reasoning effort xhigh Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.5 pp sourcePopulation 81 stderr 1.497 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label meta/muse_spark_1_1 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__litigation_dispute_resolution/meta~1muse_spark_1_1` | | | | | | | | | [Legal Research Bench / weighted pass criminal](https://benchize.com/benchmarks/vals-ai%3Alegal_research%3Aweighted_pass__criminal.md) | 85.45% | SE ±2.33 pp | 12 / 50 | $0.35 | 320.96source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 85.445 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.345173 \| USD \| per_test \| Lower is better \| \| `latency` \| 320.962 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider Meta reasoning Not reported Reasoning effort xhigh Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.33 pp sourcePopulation 82 stderr 2.33 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) Source Vals AI Original model label meta/muse_spark_1_1 Source version 1 Observed 2026-10-21T16:45:52.389Z Evidence SHA-256 `26c58dfd47d0d2dc86f33c6d056f30309e0ee63570e7cd752c593b385f9d862e` Source locator `/tasks/weighted_pass__criminal/meta~1muse_spark_1_1` | | | | | | | | | [Finance Agent (v2) / general qualitative analysis](https://benchize.com/benchmarks/vals-ai%3Afabv2%3Ageneral_qualitative_analysis.md) | 66.37% | SE ±1.94 pp | 22 / 50 | $0.75 | 409.17source duration | [Vals AI · 2](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 66.365 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.746224 \| USD \| per_test \| Lower is better \| \| `latency` \| 409.17 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider Meta reasoning Not reported Reasoning effort xhigh Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.94 pp sourcePopulation 80 stderr 1.941 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) Source Vals AI Original model label meta/muse_spark_1_1 Source version 2 Observed 2026-10-21T16:45:57.035Z Evidence SHA-256 `f731b20a26c126725d125519f2b5841aa45254656dba859a9570d7ce7e5d8191` Source locator `/tasks/general_qualitative_analysis/meta~1muse_spark_1_1` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / arbitration international dispute resolution](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Aarbitration_international_dispute_resolution.md) | 0% | SE ±0 pp | 4 / 49 | $0.97 | 774.5source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.973952 \| USD \| per_test \| Lower is better \| \| `latency` \| 774.504 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider Meta reasoning Not reported Reasoning effort xhigh Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 81 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label meta/muse_spark_1_1 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/arbitration_international_dispute_resolution/meta~1muse_spark_1_1` | | | | | | | | | [LegalBench / issue tasks](https://benchize.com/benchmarks/vals-ai%3Alegal_bench%3Aissue_tasks.md) | 88.99% | SE ±0.57 pp | 18 / 44 | $0.00992 | 16.81source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_bench.D6uTgA3b.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 88.993 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.009923 \| USD \| per_test \| Lower is better \| \| `latency` \| 16.813 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider Meta reasoning Not reported Reasoning effort xhigh Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.57 pp sourcePopulation 151 stderr 0.571 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_bench.D6uTgA3b.json) Source Vals AI Original model label meta/muse_spark_1_1 Source version 1 Observed 2026-10-21T16:45:53.390Z Evidence SHA-256 `c8486ba756baa5cd3955d035edff67440e8db23e69d62ad1a9e18d15b8c4bc52` Source locator `/tasks/issue_tasks/meta~1muse_spark_1_1` | | | | | | | | | [Terminal-Bench 2.1 / easy](https://benchize.com/benchmarks/vals-ai%3Aterminal-bench-2-1%3Aeasy.md) | 75% | SE ±0 pp | 29 / 43 | $0.17 | 13,901.17source duration | [Vals AI · 2.1](https://www.vals.ai/_astro/benchmark_view_terminal_bench_2_1.CdyoObsA.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 75 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.168085 \| USD \| per_test \| Lower is better \| \| `latency` \| 13901.169 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider Meta reasoning Not reported Reasoning effort xhigh Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 76 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_terminal_bench_2_1.CdyoObsA.json) Source Vals AI Original model label meta/muse_spark_1_1 Source version 2.1 Observed 2026-10-21T16:46:13.216Z Evidence SHA-256 `a5306805cdf86495bdcceff6b2c8534a3768b9853f5077895dab975b809b054f` Source locator `/tasks/easy/meta~1muse_spark_1_1` | | | | | | | | | [Vals Multimodal Index / corp fin v2](https://benchize.com/benchmarks/vals-ai%3Avals_multimodal_index%3Acorp_fin_v2.md) | 73.08% | SE ±1.51 pp | 2 / 20 | $0.09 | 25.41source duration | [Vals AI · 1.2](https://www.vals.ai/_astro/benchmark_view_vals_multimodal_index.virVpWMs.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 73.077 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.087456 \| USD \| per_test \| Lower is better \| \| `latency` \| 25.409 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider Meta reasoning Not reported Reasoning effort xhigh Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.51 pp sourcePopulation 33 stderr 1.514 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_vals_multimodal_index.virVpWMs.json) Source Vals AI Original model label meta/muse_spark_1_1 Source version 1.2 Observed 2026-10-21T16:45:51.881Z Evidence SHA-256 `077b22575f6e075b7b78157d000ec8f94d7ead409c7dcbcf04887630ed572a2d` Source locator `/tasks/corp_fin_v2/meta~1muse_spark_1_1` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / funds asset management](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Afunds_asset_management.md) | 40% | SE ±24.5 pp | 1 / 49 | $1 | 1,163.57source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 40 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.99674 \| USD \| per_test \| Lower is better \| \| `latency` \| 1163.572 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider Meta reasoning Not reported Reasoning effort xhigh Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±24.5 pp sourcePopulation 81 stderr 24.495 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label meta/muse_spark_1_1 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/funds_asset_management/meta~1muse_spark_1_1` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / criteria pass rate capital markets](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__capital_markets.md) | 92.55% | SE ±1.65 pp | 8 / 49 | $0.65 | 557.86source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 92.549 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.645246 \| USD \| per_test \| Lower is better \| \| `latency` \| 557.858 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider Meta reasoning Not reported Reasoning effort xhigh Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.65 pp sourcePopulation 81 stderr 1.648 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label meta/muse_spark_1_1 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__capital_markets/meta~1muse_spark_1_1` | | | | | | | | | [Legal Research Bench / business commercial](https://benchize.com/benchmarks/vals-ai%3Alegal_research%3Abusiness_commercial.md) | 51.16% | SE ±7.71 pp | 4 / 50 | $0.36 | 319.15source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 51.163 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.363621 \| USD \| per_test \| Lower is better \| \| `latency` \| 319.151 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider Meta reasoning Not reported Reasoning effort xhigh Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±7.71 pp sourcePopulation 82 stderr 7.713 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) Source Vals AI Original model label meta/muse_spark_1_1 Source version 1 Observed 2026-10-21T16:45:52.389Z Evidence SHA-256 `26c58dfd47d0d2dc86f33c6d056f30309e0ee63570e7cd752c593b385f9d862e` Source locator `/tasks/business_commercial/meta~1muse_spark_1_1` | | | | | | | | | [LiveCodeBench / easy](https://benchize.com/benchmarks/vals-ai%3Alcb%3Aeasy.md) | 98.76% | SE ±0.62 pp | 14 / 43 | $0.00869 | 18.99source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_lcb.miyRUvVa.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 98.758 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.00869 \| USD \| per_test \| Lower is better \| \| `latency` \| 18.987 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider Meta reasoning Not reported Reasoning effort xhigh Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.62 pp sourcePopulation 143 stderr 0.617 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_lcb.miyRUvVa.json) Source Vals AI Original model label meta/muse_spark_1_1 Source version 1 Observed 2026-10-21T16:46:14.220Z Evidence SHA-256 `1a7c7dabc9d95a0182886b1edc42623987cac2095a0d407a5e334f37acdfa2f5` Source locator `/tasks/easy/meta~1muse_spark_1_1` | | | | | | | | | [Legal Research Bench / weighted pass civil litigation](https://benchize.com/benchmarks/vals-ai%3Alegal_research%3Aweighted_pass__civil_litigation.md) | 88.95% | SE ±3.25 pp | 12 / 50 | $0.37 | 308.85source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 88.953 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.367043 \| USD \| per_test \| Lower is better \| \| `latency` \| 308.847 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider Meta reasoning Not reported Reasoning effort xhigh Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±3.25 pp sourcePopulation 82 stderr 3.252 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) Source Vals AI Original model label meta/muse_spark_1_1 Source version 1 Observed 2026-10-21T16:45:52.389Z Evidence SHA-256 `26c58dfd47d0d2dc86f33c6d056f30309e0ee63570e7cd752c593b385f9d862e` Source locator `/tasks/weighted_pass__civil_litigation/meta~1muse_spark_1_1` | | | | | | | | | [EMB / overall](https://benchize.com/benchmarks/vals-ai%3Aemb%3Aoverall.md) | 56.36% | SE ±3.15 pp | 33 / 47 | $1.11 | 891.51source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 56.361 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 1.112498 \| USD \| per_test \| Lower is better \| \| `latency` \| 891.512 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider Meta reasoning Not reported Reasoning effort xhigh Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±3.15 pp sourcePopulation 76 stderr 3.153 tokenTotals {"cache_read_tokens":350123312,"cache_write_tokens":null,"input_tokens":368051977,"output_tokens":9069516,"reasoning_tokens":4627614} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) Source Vals AI Original model label meta/muse_spark_1_1 Source version 1 Observed 2026-10-21T16:45:57.495Z Evidence SHA-256 `d807aee620ea47152a1fca0257a8dd12a1be4512c76cc1744c1ae9e285180918` Source locator `/tasks/overall/meta~1muse_spark_1_1` | | | | | | | | | [EMB / scratch](https://benchize.com/benchmarks/vals-ai%3Aemb%3Ascratch.md) | 63.29% | SE ±2.57 pp | 30 / 47 | $1.02 | 868.09source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 63.285 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 1.015739 \| USD \| per_test \| Lower is better \| \| `latency` \| 868.085 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider Meta reasoning Not reported Reasoning effort xhigh Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.57 pp sourcePopulation 76 stderr 2.57 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) Source Vals AI Original model label meta/muse_spark_1_1 Source version 1 Observed 2026-10-21T16:45:57.495Z Evidence SHA-256 `d807aee620ea47152a1fca0257a8dd12a1be4512c76cc1744c1ae9e285180918` Source locator `/tasks/scratch/meta~1muse_spark_1_1` | | | | | | | | | [ProgramBench / overall](https://benchize.com/benchmarks/vals-ai%3Aprogrambench%3Aoverall.md) | 0% | SE ±0 pp | 23 / 40 | $13.06 | 20,592.05source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_programbench.C_J7ZU4X.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 13.063498 \| USD \| per_test \| Lower is better \| \| `latency` \| 20592.047 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider Meta reasoning Not reported Reasoning effort xhigh Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 64 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_programbench.C_J7ZU4X.json) Source Vals AI Original model label meta/muse_spark_1_1 Source version 1 Observed 2026-10-21T16:46:10.137Z Evidence SHA-256 `e9bbb4378c7a058441a21d456baf9316b96c2cf5b58d371a3c7c803f5000bd8c` Source locator `/tasks/overall/meta~1muse_spark_1_1` | | | | | | | | | [EMB / template](https://benchize.com/benchmarks/vals-ai%3Aemb%3Atemplate.md) | 49.41% | SE ±5.19 pp | 33 / 47 | $1.21 | 914.94source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 49.41 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 1.209257 \| USD \| per_test \| Lower is better \| \| `latency` \| 914.94 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider Meta reasoning Not reported Reasoning effort xhigh Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±5.19 pp sourcePopulation 76 stderr 5.188 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) Source Vals AI Original model label meta/muse_spark_1_1 Source version 1 Observed 2026-10-21T16:45:57.495Z Evidence SHA-256 `d807aee620ea47152a1fca0257a8dd12a1be4512c76cc1744c1ae9e285180918` Source locator `/tasks/template/meta~1muse_spark_1_1` | | | | | | | | | [CorpFin v2 / max fitting context](https://benchize.com/benchmarks/vals-ai%3Acorp_fin_v2%3Amax_fitting_context.md) | 74.36% | SE ±1.49 pp | 3 / 39 | $0.23 | 39.38source duration | [Vals AI · 2](https://www.vals.ai/_astro/benchmark_view_corp_fin_v2.CectKwl1.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 74.359 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.230768 \| USD \| per_test \| Lower is better \| \| `latency` \| 39.377 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider Meta reasoning Not reported Reasoning effort xhigh Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.49 pp sourcePopulation 134 stderr 1.491 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_corp_fin_v2.CectKwl1.json) Source Vals AI Original model label meta/muse_spark_1_1 Source version 2 Observed 2026-10-21T16:45:57.998Z Evidence SHA-256 `7cf2217ab3728d5a84bffce82aab9fc52d7f810b0cbe8c65b387df47f3d31c9b` Source locator `/tasks/max_fitting_context/meta~1muse_spark_1_1` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / trusts estates private client](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Atrusts_estates_private_client.md) | 20% | SE ±20 pp | 2 / 49 | $0.72 | 613.88source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 20 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.719941 \| USD \| per_test \| Lower is better \| \| `latency` \| 613.88 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider Meta reasoning Not reported Reasoning effort xhigh Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±20 pp sourcePopulation 81 stderr 20 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label meta/muse_spark_1_1 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/trusts_estates_private_client/meta~1muse_spark_1_1` | | | | | | | | | [SkillsBench / no skills](https://benchize.com/benchmarks/vals-ai%3Askillsbench%3Ano_skills.md) | 46.27% | SE ±4.59 pp | 4 / 20 | $0.46 | 364.45source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_skillsbench.BE2TXB-M.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 46.266 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.462228 \| USD \| per_test \| Lower is better \| \| `latency` \| 364.452 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness OpenHands maxOutputTokens 256000 provider Meta reasoning Not reported Reasoning effort xhigh Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±4.59 pp sourcePopulation 35 stderr 4.589 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_skillsbench.BE2TXB-M.json) Source Vals AI Original model label meta/muse_spark_1_1 Source version 1 Observed 2026-10-21T16:46:11.147Z Evidence SHA-256 `aaad242949d81b7c7c61c36d4895d4460e9146760c1e6b4b38d235a16043bc61` Source locator `/tasks/no_skills/meta~1muse_spark_1_1` | | | | | | | | | [Legal Research Bench / administrative regulatory](https://benchize.com/benchmarks/vals-ai%3Alegal_research%3Aadministrative_regulatory.md) | 37.88% | SE ±6.02 pp | 33 / 50 | $0.38 | 282.39source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 37.879 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.382532 \| USD \| per_test \| Lower is better \| \| `latency` \| 282.392 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider Meta reasoning Not reported Reasoning effort xhigh Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±6.02 pp sourcePopulation 82 stderr 6.017 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) Source Vals AI Original model label meta/muse_spark_1_1 Source version 1 Observed 2026-10-21T16:45:52.389Z Evidence SHA-256 `26c58dfd47d0d2dc86f33c6d056f30309e0ee63570e7cd752c593b385f9d862e` Source locator `/tasks/administrative_regulatory/meta~1muse_spark_1_1` | | | | | | | | | [ProgramBench / almost](https://benchize.com/benchmarks/vals-ai%3Aprogrambench%3Aalmost.md) | 2.5% | SE ±1.11 pp | 27 / 40 | $13.06 | 20,592.05source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_programbench.C_J7ZU4X.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 2.5 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 13.063498 \| USD \| per_test \| Lower is better \| \| `latency` \| 20592.047 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider Meta reasoning Not reported Reasoning effort xhigh Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.11 pp sourcePopulation 64 stderr 1.107 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_programbench.C_J7ZU4X.json) Source Vals AI Original model label meta/muse_spark_1_1 Source version 1 Observed 2026-10-21T16:46:10.137Z Evidence SHA-256 `e9bbb4378c7a058441a21d456baf9316b96c2cf5b58d371a3c7c803f5000bd8c` Source locator `/tasks/almost/meta~1muse_spark_1_1` | | | | | | | | | [CorpFin v2 / exact pages](https://benchize.com/benchmarks/vals-ai%3Acorp_fin_v2%3Aexact_pages.md) | 66.43% | SE ±1.61 pp | 4 / 39 | $0.00947 | 18.68source duration | [Vals AI · 2](https://www.vals.ai/_astro/benchmark_view_corp_fin_v2.CectKwl1.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 66.434 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.009465 \| USD \| per_test \| Lower is better \| \| `latency` \| 18.683 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider Meta reasoning Not reported Reasoning effort xhigh Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.61 pp sourcePopulation 134 stderr 1.612 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_corp_fin_v2.CectKwl1.json) Source Vals AI Original model label meta/muse_spark_1_1 Source version 2 Observed 2026-10-21T16:45:57.998Z Evidence SHA-256 `7cf2217ab3728d5a84bffce82aab9fc52d7f810b0cbe8c65b387df47f3d31c9b` Source locator `/tasks/exact_pages/meta~1muse_spark_1_1` | | | | | | | | | [Vals Multimodal Index / vibe code bench](https://benchize.com/benchmarks/vals-ai%3Avals_multimodal_index%3Avibe_code_bench.md) | 72.41% | SE ±5.17 pp | 11 / 20 | $0.92 | 615.43source duration | [Vals AI · 1.2](https://www.vals.ai/_astro/benchmark_view_vals_multimodal_index.virVpWMs.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 72.412 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.923283 \| USD \| per_test \| Lower is better \| \| `latency` \| 615.428 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider Meta reasoning Not reported Reasoning effort xhigh Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±5.17 pp sourcePopulation 33 stderr 5.168 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_vals_multimodal_index.virVpWMs.json) Source Vals AI Original model label meta/muse_spark_1_1 Source version 1.2 Observed 2026-10-21T16:45:51.881Z Evidence SHA-256 `077b22575f6e075b7b78157d000ec8f94d7ead409c7dcbcf04887630ed572a2d` Source locator `/tasks/vibe_code_bench/meta~1muse_spark_1_1` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / tax](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Atax.md) | 0% | SE ±0 pp | 3 / 49 | $0.71 | 698.21source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.714588 \| USD \| per_test \| Lower is better \| \| `latency` \| 698.208 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider Meta reasoning Not reported Reasoning effort xhigh Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 81 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label meta/muse_spark_1_1 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/tax/meta~1muse_spark_1_1` | | | | | | | | | [EMB / lbo](https://benchize.com/benchmarks/vals-ai%3Aemb%3Albo.md) | 49.77% | SE ±5.37 pp | 31 / 47 | $1.74 | 1,292.91source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 49.767 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 1.737996 \| USD \| per_test \| Lower is better \| \| `latency` \| 1292.905 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider Meta reasoning Not reported Reasoning effort xhigh Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±5.37 pp sourcePopulation 76 stderr 5.366 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) Source Vals AI Original model label meta/muse_spark_1_1 Source version 1 Observed 2026-10-21T16:45:57.495Z Evidence SHA-256 `d807aee620ea47152a1fca0257a8dd12a1be4512c76cc1744c1ae9e285180918` Source locator `/tasks/lbo/meta~1muse_spark_1_1` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / data privacy cybersecurity](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Adata_privacy_cybersecurity.md) | 0% | SE ±0 pp | 13 / 49 | $0.84 | 654.12source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.83785 \| USD \| per_test \| Lower is better \| \| `latency` \| 654.12 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider Meta reasoning Not reported Reasoning effort xhigh Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 81 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label meta/muse_spark_1_1 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/data_privacy_cybersecurity/meta~1muse_spark_1_1` | | | | | | | | | [LegalBench / interpretation tasks](https://benchize.com/benchmarks/vals-ai%3Alegal_bench%3Ainterpretation_tasks.md) | 82.88% | SE ±0.2 pp | 16 / 44 | $0.00777 | 14.79source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_bench.D6uTgA3b.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 82.876 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.00777 \| USD \| per_test \| Lower is better \| \| `latency` \| 14.792 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider Meta reasoning Not reported Reasoning effort xhigh Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.2 pp sourcePopulation 151 stderr 0.199 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_bench.D6uTgA3b.json) Source Vals AI Original model label meta/muse_spark_1_1 Source version 1 Observed 2026-10-21T16:45:53.390Z Evidence SHA-256 `c8486ba756baa5cd3955d035edff67440e8db23e69d62ad1a9e18d15b8c4bc52` Source locator `/tasks/interpretation_tasks/meta~1muse_spark_1_1` | | | | | | | | | [Finance Agent (v2) / adjustments](https://benchize.com/benchmarks/vals-ai%3Afabv2%3Aadjustments.md) | 49.66% | SE ±2.34 pp | 21 / 50 | $0.71 | 331.41source duration | [Vals AI · 2](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 49.66 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.707474 \| USD \| per_test \| Lower is better \| \| `latency` \| 331.405 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider Meta reasoning Not reported Reasoning effort xhigh Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.34 pp sourcePopulation 80 stderr 2.337 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) Source Vals AI Original model label meta/muse_spark_1_1 Source version 2 Observed 2026-10-21T16:45:57.035Z Evidence SHA-256 `f731b20a26c126725d125519f2b5841aa45254656dba859a9570d7ce7e5d8191` Source locator `/tasks/adjustments/meta~1muse_spark_1_1` | | | | | | | | | [MortgageTax / semantic extraction](https://benchize.com/benchmarks/vals-ai%3Amortgage_tax%3Asemantic_extraction.md) | 57.79% | SE ±1.39 pp | 21 / 31 | $0.02 | 18.94source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_mortgage_tax.DCd-oO5Y.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 57.79 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.019246 \| USD \| per_test \| Lower is better \| \| `latency` \| 18.935 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider Meta reasoning Not reported Reasoning effort xhigh Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.39 pp sourcePopulation 98 stderr 1.392 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_mortgage_tax.DCd-oO5Y.json) Source Vals AI Original model label meta/muse_spark_1_1 Source version 1 Observed 2026-10-21T16:45:58.500Z Evidence SHA-256 `e5c844561f97d119d66d70269e726ecfb68be79eae39acf14ead3e95422e648c` Source locator `/tasks/semantic_extraction/meta~1muse_spark_1_1` | | | | | | | | | [Code Migration / cli](https://benchize.com/benchmarks/vals-ai%3Acode-migration%3Acli.md) | 18.63% | SE ±2.08 pp | 31 / 47 | $5.54 | 2,739.51source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_code_migration.Dttzoqk3.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 18.629 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 5.536818 \| USD \| per_test \| Lower is better \| \| `latency` \| 2739.514 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider Meta reasoning Not reported Reasoning effort xhigh Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.08 pp sourcePopulation 79 stderr 2.08 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_code_migration.Dttzoqk3.json) Source Vals AI Original model label meta/muse_spark_1_1 Source version 1 Observed 2026-10-21T16:46:12.712Z Evidence SHA-256 `99194597f2c76c402195be47839409046e18bd30454616da6dd40b736a088fa4` Source locator `/tasks/cli/meta~1muse_spark_1_1` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / banking finance](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Abanking_finance.md) | 0% | SE ±0 pp | 5 / 49 | $0.66 | 553.18source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.664525 \| USD \| per_test \| Lower is better \| \| `latency` \| 553.184 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 256000 provider Meta reasoning Not reported Reasoning effort xhigh Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 81 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label meta/muse_spark_1_1 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/banking_finance/meta~1muse_spark_1_1` | | | | | | | | [Next page](https://benchize.com/models/muse-spark-1-1-xhigh.md?cursor=9b65a440daad599188e37a1ac182ce64561a3c68a18770182f2bd9e1de09b203) --- Cite as: Muse Spark 1.1Extra high — Benchize, https://benchize.com/models/muse-spark-1-1-xhigh (retrieved 2026-10-21)