# Muse Spark OpenExtra high > Source: Benchize (https://benchize.com) — https://benchize.com/models/muse-spark-open-xhigh. When citing or reusing this content, credit "Benchize" and link to https://benchize.com/models/muse-spark-open-xhigh. Meta Canonical URL: https://benchize.com/models/muse-spark-open-xhigh Last updated: Wed, 21 Oct 2026 18:52:20 GMT Reproduction and quotation must name Benchize, link to the canonical HTML URL, and retain individual source citations. Meta ## Benchmarks | Benchmark | Score | ± | Rank | Cost | Latency | Source | | | --- | --- | --- | --- | --- | --- | --- | --- | | [Terminal-Bench 4.0 / media](https://benchize.com/benchmarks/vals-ai%3Aterminal-bench-4%3Amedia.md) | 0% | SE ±0 pp | 24 / 36 | — | — | [Vals AI · 4.0](https://www.vals.ai/_astro/benchmark_view_terminal_bench_4.CAnGyzC_.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 131072 provider Together AI reasoning Not reported Reasoning effort xhigh Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 52 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_terminal_bench_4.CAnGyzC_.json) Source Vals AI Original model label meta/muse-spark-open Source version 4.0 Observed 2026-10-21T16:46:10.642Z Evidence SHA-256 `98f6964580e901f1aa143db664bd92aa558415f643569492545d67c47d7f519b` Source locator `/tasks/media/meta~1muse-spark-open` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / employment labor](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Aemployment_labor.md) | 0% | SE ±0 pp | 1 / 49 | $6.32 | 8,635.98source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 6.318117 \| USD \| per_test \| Lower is better \| \| `latency` \| 8635.982 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 131072 provider Together AI reasoning Not reported Reasoning effort xhigh Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 81 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label meta/muse-spark-open Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/employment_labor/meta~1muse-spark-open` | | | | | | | | | [EMB v1.1 / dcf](https://benchize.com/benchmarks/vals-ai%3Aemb-v1-1%3Adcf.md) | 49.91% | SE ±7.26 pp | 19 / 19 | $4.65 | 5,120.53source duration | [Vals AI · 1.1](https://www.vals.ai/_astro/benchmark_view_emb_v1_1.11L06J7L.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 49.913 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 4.653132 \| USD \| per_test \| Lower is better \| \| `latency` \| 5120.532 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 131072 provider Meta reasoning Not reported Reasoning effort xhigh Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±7.26 pp sourcePopulation 30 stderr 7.255 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb_v1_1.11L06J7L.json) Source Vals AI Original model label meta/muse-spark-open Source version 1.1 Observed 2026-10-21T16:45:55.456Z Evidence SHA-256 `eea2ef2c83a6dc7d553c7c6531c287710ea7ffbc56ee1ccec06c85b104b2408e` Source locator `/tasks/dcf/meta~1muse-spark-open` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / criteria pass rate healthcare life sciences](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__healthcare_life_sciences.md) | 34.98% | SE ±2.74 pp | 48 / 49 | $5.03 | 7,857.63source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 34.983 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 5.029791 \| USD \| per_test \| Lower is better \| \| `latency` \| 7857.632 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 131072 provider Together AI reasoning Not reported Reasoning effort xhigh Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.74 pp sourcePopulation 81 stderr 2.744 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label meta/muse-spark-open Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__healthcare_life_sciences/meta~1muse-spark-open` | | | | | | | | | [EMB v1.1 / lbo](https://benchize.com/benchmarks/vals-ai%3Aemb-v1-1%3Albo.md) | 48.33% | SE ±5.03 pp | 18 / 19 | $7.43 | 5,821.1source duration | [Vals AI · 1.1](https://www.vals.ai/_astro/benchmark_view_emb_v1_1.11L06J7L.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 48.325 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 7.431962 \| USD \| per_test \| Lower is better \| \| `latency` \| 5821.099 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 131072 provider Meta reasoning Not reported Reasoning effort xhigh Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±5.03 pp sourcePopulation 30 stderr 5.029 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb_v1_1.11L06J7L.json) Source Vals AI Original model label meta/muse-spark-open Source version 1.1 Observed 2026-10-21T16:45:55.456Z Evidence SHA-256 `eea2ef2c83a6dc7d553c7c6531c287710ea7ffbc56ee1ccec06c85b104b2408e` Source locator `/tasks/lbo/meta~1muse-spark-open` | | | | | | | | | [Tax Agent Bench / weighted pass current temporal analysis](https://benchize.com/benchmarks/vals-ai%3Atax_agent_bench%3Aweighted_pass__current_temporal_analysis.md) | 47.22% | SE ±5.77 pp | 37 / 46 | $0.29 | 760.5source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 47.221 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.288385 \| USD \| per_test \| Lower is better \| \| `latency` \| 760.495 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 131072 provider Together AI reasoning Not reported Reasoning effort xhigh Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±5.77 pp sourcePopulation 73 stderr 5.768 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) Source Vals AI Original model label meta/muse-spark-open Source version 1 Observed 2026-10-21T16:45:55.958Z Evidence SHA-256 `446db182b67efb7923b3187a4040ec3a3ea492b8ad699a2b6d708d714cc063ac` Source locator `/tasks/weighted_pass__current_temporal_analysis/meta~1muse-spark-open` | | | | | | | | | [Legal Research Bench / weighted pass health](https://benchize.com/benchmarks/vals-ai%3Alegal_research%3Aweighted_pass__health.md) | 90.8% | SE ±2.47 pp | 28 / 50 | $1.56 | 1,685.59source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 90.803 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 1.561613 \| USD \| per_test \| Lower is better \| \| `latency` \| 1685.594 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 131072 provider Together AI reasoning Not reported Reasoning effort xhigh Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.47 pp sourcePopulation 82 stderr 2.474 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) Source Vals AI Original model label meta/muse-spark-open Source version 1 Observed 2026-10-21T16:45:52.389Z Evidence SHA-256 `26c58dfd47d0d2dc86f33c6d056f30309e0ee63570e7cd752c593b385f9d862e` Source locator `/tasks/weighted_pass__health/meta~1muse-spark-open` | | | | | | | | | [Terminal-Bench 4.0 / software](https://benchize.com/benchmarks/vals-ai%3Aterminal-bench-4%3Asoftware.md) | 11.11% | SE ±0 pp | 28 / 36 | — | — | [Vals AI · 4.0](https://www.vals.ai/_astro/benchmark_view_terminal_bench_4.CAnGyzC_.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 11.111 \| percent \| reported \| Higher is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 131072 provider Together AI reasoning Not reported Reasoning effort xhigh Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 52 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_terminal_bench_4.CAnGyzC_.json) Source Vals AI Original model label meta/muse-spark-open Source version 4.0 Observed 2026-10-21T16:46:10.642Z Evidence SHA-256 `98f6964580e901f1aa143db664bd92aa558415f643569492545d67c47d7f519b` Source locator `/tasks/software/meta~1muse-spark-open` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / criteria pass rate emerging companies venture capital](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__emerging_companies_venture_capital.md) | 93.99% | SE ±1.56 pp | 5 / 49 | $7.1 | 11,588.73source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 93.991 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 7.104223 \| USD \| per_test \| Lower is better \| \| `latency` \| 11588.732 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 131072 provider Together AI reasoning Not reported Reasoning effort xhigh Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.56 pp sourcePopulation 81 stderr 1.56 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label meta/muse-spark-open Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__emerging_companies_venture_capital/meta~1muse-spark-open` | | | | | | | | | [EMB v1.1 / misc](https://benchize.com/benchmarks/vals-ai%3Aemb-v1-1%3Amisc.md) | 60.28% | SE ±3.14 pp | 16 / 19 | $4.42 | 5,020.41source duration | [Vals AI · 1.1](https://www.vals.ai/_astro/benchmark_view_emb_v1_1.11L06J7L.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 60.284 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 4.421871 \| USD \| per_test \| Lower is better \| \| `latency` \| 5020.414 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 131072 provider Meta reasoning Not reported Reasoning effort xhigh Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±3.14 pp sourcePopulation 30 stderr 3.136 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb_v1_1.11L06J7L.json) Source Vals AI Original model label meta/muse-spark-open Source version 1.1 Observed 2026-10-21T16:45:55.456Z Evidence SHA-256 `eea2ef2c83a6dc7d553c7c6531c287710ea7ffbc56ee1ccec06c85b104b2408e` Source locator `/tasks/misc/meta~1muse-spark-open` | | | | | | | | | [Tax Agent Bench / weighted pass](https://benchize.com/benchmarks/vals-ai%3Atax_agent_bench%3Aweighted_pass.md) | 52.07% | SE ±2.39 pp | 35 / 46 | $0.88 | 701.83source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 52.066 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.883559 \| USD \| per_test \| Lower is better \| \| `latency` \| 701.825 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 131072 provider Together AI reasoning Not reported Reasoning effort xhigh Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.39 pp sourcePopulation 73 stderr 2.387 tokenTotals {"cache_read_tokens":329680688,"cache_write_tokens":null,"input_tokens":413690950,"output_tokens":3571385,"reasoning_tokens":1667757} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) Source Vals AI Original model label meta/muse-spark-open Source version 1 Observed 2026-10-21T16:45:55.958Z Evidence SHA-256 `446db182b67efb7923b3187a4040ec3a3ea492b8ad699a2b6d708d714cc063ac` Source locator `/tasks/weighted_pass/meta~1muse-spark-open` | | | | | | | | | [Finance Agent (v2) / all pass](https://benchize.com/benchmarks/vals-ai%3Afabv2%3Aall_pass.md) | 50.59% | SE ±2.22 pp | 3 / 50 | $0.77 | 1,453.44source duration | [Vals AI · 2](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 50.593 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.770844 \| USD \| per_test \| Lower is better \| \| `latency` \| 1453.439 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 131072 provider Together AI reasoning Not reported Reasoning effort xhigh Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.22 pp sourcePopulation 80 stderr 2.224 tokenTotals {"cache_read_tokens":107422512,"cache_write_tokens":null,"input_tokens":334545744,"output_tokens":11026394,"reasoning_tokens":5948162} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) Source Vals AI Original model label meta/muse-spark-open Source version 2 Observed 2026-10-21T16:45:57.035Z Evidence SHA-256 `f731b20a26c126725d125519f2b5841aa45254656dba859a9570d7ce7e5d8191` Source locator `/tasks/all_pass/meta~1muse-spark-open` | | | | | | | | | [EMB v1.1 / template](https://benchize.com/benchmarks/vals-ai%3Aemb-v1-1%3Atemplate.md) | 61.81% | SE ±3.94 pp | 18 / 19 | $7.01 | 5,739.05source duration | [Vals AI · 1.1](https://www.vals.ai/_astro/benchmark_view_emb_v1_1.11L06J7L.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 61.805 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 7.009169 \| USD \| per_test \| Lower is better \| \| `latency` \| 5739.054 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 131072 provider Meta reasoning Not reported Reasoning effort xhigh Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±3.94 pp sourcePopulation 30 stderr 3.943 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb_v1_1.11L06J7L.json) Source Vals AI Original model label meta/muse-spark-open Source version 1.1 Observed 2026-10-21T16:45:55.456Z Evidence SHA-256 `eea2ef2c83a6dc7d553c7c6531c287710ea7ffbc56ee1ccec06c85b104b2408e` Source locator `/tasks/template/meta~1muse-spark-open` | | | | | | | | | [Finance Agent v2.1 / disclosure analysis](https://benchize.com/benchmarks/vals-ai%3Afabv2-1%3Adisclosure_analysis.md) | 76.79% | SE ±5.68 pp | 5 / 18 | $1.85 | 1,369.4source duration | [Vals AI · 2.1](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2_1.ZbMhy9xo.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 76.791 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 1.846256 \| USD \| per_test \| Lower is better \| \| `latency` \| 1369.397 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 131072 provider Together AI reasoning Not reported Reasoning effort xhigh Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±5.68 pp sourcePopulation 28 stderr 5.676 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2_1.ZbMhy9xo.json) Source Vals AI Original model label meta/muse-spark-open Source version 2.1 Observed 2026-10-21T16:45:54.954Z Evidence SHA-256 `f1b0c2d7dca4e73b52136862a9be059f7c8481c69f15d705ce90bf3bd9f78429` Source locator `/tasks/disclosure_analysis/meta~1muse-spark-open` | | | | | | | | | [EMB v1.1 / overall](https://benchize.com/benchmarks/vals-ai%3Aemb-v1-1%3Aoverall.md) | 61.98% | SE ±2.7 pp | 18 / 19 | $5.35 | 5,078.61source duration | [Vals AI · 1.1](https://www.vals.ai/_astro/benchmark_view_emb_v1_1.11L06J7L.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 61.983 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 5.348821 \| USD \| per_test \| Lower is better \| \| `latency` \| 5078.614 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 131072 provider Meta reasoning Not reported Reasoning effort xhigh Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.7 pp sourcePopulation 30 stderr 2.704 tokenTotals {"cache_read_tokens":642573184,"cache_write_tokens":null,"input_tokens":973042320,"output_tokens":8495845,"reasoning_tokens":4888586} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb_v1_1.11L06J7L.json) Source Vals AI Original model label meta/muse-spark-open Source version 1.1 Observed 2026-10-21T16:45:55.456Z Evidence SHA-256 `eea2ef2c83a6dc7d553c7c6531c287710ea7ffbc56ee1ccec06c85b104b2408e` Source locator `/tasks/overall/meta~1muse-spark-open` | | | | | | | | | [Tax Agent Bench / numbers calculations](https://benchize.com/benchmarks/vals-ai%3Atax_agent_bench%3Anumbers_calculations.md) | 38.67% | SE ±7.9 pp | 23 / 46 | $0.16 | 339.07source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 38.667 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.15679 \| USD \| per_test \| Lower is better \| \| `latency` \| 339.074 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 131072 provider Together AI reasoning Not reported Reasoning effort xhigh Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±7.9 pp sourcePopulation 73 stderr 7.904 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) Source Vals AI Original model label meta/muse-spark-open Source version 1 Observed 2026-10-21T16:45:55.958Z Evidence SHA-256 `446db182b67efb7923b3187a4040ec3a3ea492b8ad699a2b6d708d714cc063ac` Source locator `/tasks/numbers_calculations/meta~1muse-spark-open` | | | | | | | | | [Terminal-Bench 4.0 / hardware](https://benchize.com/benchmarks/vals-ai%3Aterminal-bench-4%3Ahardware.md) | 0% | SE ±0 pp | 21 / 36 | — | — | [Vals AI · 4.0](https://www.vals.ai/_astro/benchmark_view_terminal_bench_4.CAnGyzC_.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 131072 provider Together AI reasoning Not reported Reasoning effort xhigh Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 52 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_terminal_bench_4.CAnGyzC_.json) Source Vals AI Original model label meta/muse-spark-open Source version 4.0 Observed 2026-10-21T16:46:10.642Z Evidence SHA-256 `98f6964580e901f1aa143db664bd92aa558415f643569492545d67c47d7f519b` Source locator `/tasks/hardware/meta~1muse-spark-open` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / criteria pass rate international trade sanctions](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__international_trade_sanctions.md) | 76.19% | SE ±2.69 pp | 41 / 49 | $9.69 | 12,102.36source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 76.19 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 9.688157 \| USD \| per_test \| Lower is better \| \| `latency` \| 12102.358 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 131072 provider Together AI reasoning Not reported Reasoning effort xhigh Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.69 pp sourcePopulation 81 stderr 2.688 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label meta/muse-spark-open Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__international_trade_sanctions/meta~1muse-spark-open` | | | | | | | | | [Legal Research Bench / weighted pass business commercial](https://benchize.com/benchmarks/vals-ai%3Alegal_research%3Aweighted_pass__business_commercial.md) | 88.96% | SE ±2.65 pp | 11 / 50 | $1.14 | 1,523.33source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 88.955 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 1.137443 \| USD \| per_test \| Lower is better \| \| `latency` \| 1523.334 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 131072 provider Together AI reasoning Not reported Reasoning effort xhigh Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.65 pp sourcePopulation 82 stderr 2.648 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) Source Vals AI Original model label meta/muse-spark-open Source version 1 Observed 2026-10-21T16:45:52.389Z Evidence SHA-256 `26c58dfd47d0d2dc86f33c6d056f30309e0ee63570e7cd752c593b385f9d862e` Source locator `/tasks/weighted_pass__business_commercial/meta~1muse-spark-open` | | | | | | | | | [Tax Agent Bench / weighted pass controversy precedence analysis](https://benchize.com/benchmarks/vals-ai%3Atax_agent_bench%3Aweighted_pass__controversy_precedence_analysis.md) | 45.67% | SE ±6.66 pp | 34 / 46 | $1.14 | 826.16source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 45.667 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 1.13869 \| USD \| per_test \| Lower is better \| \| `latency` \| 826.163 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 131072 provider Together AI reasoning Not reported Reasoning effort xhigh Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±6.66 pp sourcePopulation 73 stderr 6.659 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) Source Vals AI Original model label meta/muse-spark-open Source version 1 Observed 2026-10-21T16:45:55.958Z Evidence SHA-256 `446db182b67efb7923b3187a4040ec3a3ea492b8ad699a2b6d708d714cc063ac` Source locator `/tasks/weighted_pass__controversy_precedence_analysis/meta~1muse-spark-open` | | | | | | | | | [Code Migration / code quality](https://benchize.com/benchmarks/vals-ai%3Acode-migration%3Acode_quality.md) | 2.46% | SE ±0.05 pp | 47 / 47 | $9.04 | 6,790.7source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_code_migration.Dttzoqk3.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 2.463 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 9.042387 \| USD \| per_test \| Lower is better \| \| `latency` \| 6790.697 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 131072 provider Together AI reasoning Not reported Reasoning effort xhigh Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.05 pp sourcePopulation 79 stderr 0.047 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_code_migration.Dttzoqk3.json) Source Vals AI Original model label meta/muse-spark-open Source version 1 Observed 2026-10-21T16:46:12.712Z Evidence SHA-256 `99194597f2c76c402195be47839409046e18bd30454616da6dd40b736a088fa4` Source locator `/tasks/code_quality/meta~1muse-spark-open` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / criteria pass rate bankruptcy restructuring](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__bankruptcy_restructuring.md) | 76.82% | SE ±2.49 pp | 30 / 49 | $4.16 | 6,747.6source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 76.817 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 4.156653 \| USD \| per_test \| Lower is better \| \| `latency` \| 6747.604 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 131072 provider Together AI reasoning Not reported Reasoning effort xhigh Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.49 pp sourcePopulation 81 stderr 2.487 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label meta/muse-spark-open Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__bankruptcy_restructuring/meta~1muse-spark-open` | | | | | | | | | [Legal Research Bench / weighted pass](https://benchize.com/benchmarks/vals-ai%3Alegal_research%3Aweighted_pass.md) | 85.99% | SE ±1.39 pp | 10 / 50 | $1.56 | 1,763.02source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 85.989 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 1.559385 \| USD \| per_test \| Lower is better \| \| `latency` \| 1763.023 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 131072 provider Together AI reasoning Not reported Reasoning effort xhigh Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.39 pp sourcePopulation 82 stderr 1.394 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) Source Vals AI Original model label meta/muse-spark-open Source version 1 Observed 2026-10-21T16:45:52.389Z Evidence SHA-256 `26c58dfd47d0d2dc86f33c6d056f30309e0ee63570e7cd752c593b385f9d862e` Source locator `/tasks/weighted_pass/meta~1muse-spark-open` | | | | | | | | | [Legal Research Bench / health](https://benchize.com/benchmarks/vals-ai%3Alegal_research%3Ahealth.md) | 53.33% | SE ±9.26 pp | 24 / 50 | $1.56 | 1,685.59source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 53.333 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 1.561613 \| USD \| per_test \| Lower is better \| \| `latency` \| 1685.594 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 131072 provider Together AI reasoning Not reported Reasoning effort xhigh Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±9.26 pp sourcePopulation 82 stderr 9.264 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) Source Vals AI Original model label meta/muse-spark-open Source version 1 Observed 2026-10-21T16:45:52.389Z Evidence SHA-256 `26c58dfd47d0d2dc86f33c6d056f30309e0ee63570e7cd752c593b385f9d862e` Source locator `/tasks/health/meta~1muse-spark-open` | | | | | | | | | [Finance Agent (v2) / adjustments](https://benchize.com/benchmarks/vals-ai%3Afabv2%3Aadjustments.md) | 57.18% | SE ±6.97 pp | 3 / 50 | $0.84 | 2,513.75source duration | [Vals AI · 2](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 57.182 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.837871 \| USD \| per_test \| Lower is better \| \| `latency` \| 2513.754 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 131072 provider Together AI reasoning Not reported Reasoning effort xhigh Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±6.97 pp sourcePopulation 80 stderr 6.967 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) Source Vals AI Original model label meta/muse-spark-open Source version 2 Observed 2026-10-21T16:45:57.035Z Evidence SHA-256 `f731b20a26c126725d125519f2b5841aa45254656dba859a9570d7ce7e5d8191` Source locator `/tasks/adjustments/meta~1muse-spark-open` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / environmental esg](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Aenvironmental_esg.md) | 0% | SE ±0 pp | 1 / 49 | $12.21 | 11,761.1source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 12.206644 \| USD \| per_test \| Lower is better \| \| `latency` \| 11761.102 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 131072 provider Together AI reasoning Not reported Reasoning effort xhigh Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 81 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label meta/muse-spark-open Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/environmental_esg/meta~1muse-spark-open` | | | | | | | | | [Terminal-Bench 4.0 / ml](https://benchize.com/benchmarks/vals-ai%3Aterminal-bench-4%3Aml.md) | 3.03% | SE ±3.03 pp | 30 / 36 | — | — | [Vals AI · 4.0](https://www.vals.ai/_astro/benchmark_view_terminal_bench_4.CAnGyzC_.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 3.03 \| percent \| reported \| Higher is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 131072 provider Together AI reasoning Not reported Reasoning effort xhigh Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±3.03 pp sourcePopulation 52 stderr 3.03 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_terminal_bench_4.CAnGyzC_.json) Source Vals AI Original model label meta/muse-spark-open Source version 4.0 Observed 2026-10-21T16:46:10.642Z Evidence SHA-256 `98f6964580e901f1aa143db664bd92aa558415f643569492545d67c47d7f519b` Source locator `/tasks/ml/meta~1muse-spark-open` | | | | | | | | | [Finance Agent (v2) / market analysis](https://benchize.com/benchmarks/vals-ai%3Afabv2%3Amarket_analysis.md) | 71.13% | SE ±6.38 pp | 7 / 50 | $0.55 | 943.04source duration | [Vals AI · 2](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 71.134 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.549596 \| USD \| per_test \| Lower is better \| \| `latency` \| 943.044 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 131072 provider Together AI reasoning Not reported Reasoning effort xhigh Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±6.38 pp sourcePopulation 80 stderr 6.381 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) Source Vals AI Original model label meta/muse-spark-open Source version 2 Observed 2026-10-21T16:45:57.035Z Evidence SHA-256 `f731b20a26c126725d125519f2b5841aa45254656dba859a9570d7ce7e5d8191` Source locator `/tasks/market_analysis/meta~1muse-spark-open` | | | | | | | | | [Legal Research Bench / criminal](https://benchize.com/benchmarks/vals-ai%3Alegal_research%3Acriminal.md) | 35.09% | SE ±6.38 pp | 19 / 50 | $2.57 | 2,183.68source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 35.088 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 2.568801 \| USD \| per_test \| Lower is better \| \| `latency` \| 2183.678 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 131072 provider Together AI reasoning Not reported Reasoning effort xhigh Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±6.38 pp sourcePopulation 82 stderr 6.377 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) Source Vals AI Original model label meta/muse-spark-open Source version 1 Observed 2026-10-21T16:45:52.389Z Evidence SHA-256 `26c58dfd47d0d2dc86f33c6d056f30309e0ee63570e7cd752c593b385f9d862e` Source locator `/tasks/criminal/meta~1muse-spark-open` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / litigation dispute resolution](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Alitigation_dispute_resolution.md) | 0% | SE ±0 pp | 6 / 49 | $8.3 | 6,544.25source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 8.301988 \| USD \| per_test \| Lower is better \| \| `latency` \| 6544.254 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 131072 provider Together AI reasoning Not reported Reasoning effort xhigh Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 81 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label meta/muse-spark-open Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/litigation_dispute_resolution/meta~1muse-spark-open` | | | | | | | | | [Tax Agent Bench / weighted pass numbers calculations](https://benchize.com/benchmarks/vals-ai%3Atax_agent_bench%3Aweighted_pass__numbers_calculations.md) | 68.91% | SE ±5.04 pp | 35 / 46 | $0.16 | 339.07source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 68.911 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.15679 \| USD \| per_test \| Lower is better \| \| `latency` \| 339.074 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 131072 provider Together AI reasoning Not reported Reasoning effort xhigh Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±5.04 pp sourcePopulation 73 stderr 5.042 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) Source Vals AI Original model label meta/muse-spark-open Source version 1 Observed 2026-10-21T16:45:55.958Z Evidence SHA-256 `446db182b67efb7923b3187a4040ec3a3ea492b8ad699a2b6d708d714cc063ac` Source locator `/tasks/weighted_pass__numbers_calculations/meta~1muse-spark-open` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / white collar defense investigations](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Awhite_collar_defense_investigations.md) | 0% | SE ±0 pp | 1 / 49 | $3.52 | 7,655.6source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 3.523605 \| USD \| per_test \| Lower is better \| \| `latency` \| 7655.598 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 131072 provider Together AI reasoning Not reported Reasoning effort xhigh Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 81 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label meta/muse-spark-open Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/white_collar_defense_investigations/meta~1muse-spark-open` | | | | | | | | | [DesignArena Website generation](https://benchize.com/benchmarks/designarena%3Awebsite.md) | 1,317elo | — | 9 / 123 | — | — | [designarena · public](https://www.designarena.ai/api/leaderboard#board=website) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `elo` \| 1317 \| elo \| website \| Higher is better \| ### Configuration arenaType models category website inputModality text Reasoning effort xhigh sourceModelVersion muse-spark-open thinkingEnabled true variationName public ### Reported statistics averageGenerationTimeMs 278552 battles 308 losses 134 standardError Not reported winRate 56.5 wins 174 ### Source [Original source](https://www.designarena.ai/api/leaderboard#board=website) Source designarena Original model label Muse Spark Open (xhigh) Source version public Observed 2026-10-21T16:37:57.757Z Evidence SHA-256 `51873e97776a3c5ae706b24cc2c60384793dd18398df970e213a8e7c276d8ac0` Source locator `/data/10` | | | | | | | | | [EMB / lbo](https://benchize.com/benchmarks/vals-ai%3Aemb%3Albo.md) | 50.7% | SE ±5.14 pp | 28 / 47 | $7.43 | 5,821.1source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 50.698 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 7.431962 \| USD \| per_test \| Lower is better \| \| `latency` \| 5821.099 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 131072 provider Together AI reasoning Not reported Reasoning effort xhigh Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±5.14 pp sourcePopulation 76 stderr 5.141 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) Source Vals AI Original model label meta/muse-spark-open Source version 1 Observed 2026-10-21T16:45:57.495Z Evidence SHA-256 `d807aee620ea47152a1fca0257a8dd12a1be4512c76cc1744c1ae9e285180918` Source locator `/tasks/lbo/meta~1muse-spark-open` | | | | | | | | | [Finance Agent (v2) / overall](https://benchize.com/benchmarks/vals-ai%3Afabv2%3Aoverall.md) | 61.34% | SE ±2.09 pp | 3 / 50 | $0.77 | 1,453.44source duration | [Vals AI · 2](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 61.336 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.770844 \| USD \| per_test \| Lower is better \| \| `latency` \| 1453.439 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 131072 provider Together AI reasoning Not reported Reasoning effort xhigh Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.09 pp sourcePopulation 80 stderr 2.088 tokenTotals {"cache_read_tokens":107422512,"cache_write_tokens":null,"input_tokens":334545744,"output_tokens":11026394,"reasoning_tokens":5948162} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) Source Vals AI Original model label meta/muse-spark-open Source version 2 Observed 2026-10-21T16:45:57.035Z Evidence SHA-256 `f731b20a26c126725d125519f2b5841aa45254656dba859a9570d7ce7e5d8191` Source locator `/tasks/overall/meta~1muse-spark-open` | | | | | | | | | [DesignArena Browser game generation](https://benchize.com/benchmarks/designarena%3Agame-dev.md) | 1,300elo | — | 23 / 117 | — | — | [designarena · public](https://www.designarena.ai/api/leaderboard#board=game-dev) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `elo` \| 1300 \| elo \| game-dev \| Higher is better \| ### Configuration arenaType models category gamedev inputModality Not reported Reasoning effort xhigh sourceModelVersion muse-spark-open thinkingEnabled true variationName public ### Reported statistics averageGenerationTimeMs 247334 battles 236 losses 108 standardError Not reported winRate 54.2 wins 128 ### Source [Original source](https://www.designarena.ai/api/leaderboard#board=game-dev) Source designarena Original model label Muse Spark Open (xhigh) Source version public Observed 2026-10-21T16:38:01.100Z Evidence SHA-256 `53db6a89112ce8cdf644568c666d4b8309aece764cfcedb291ffd52ae3934e30` Source locator `/data/27` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / criteria pass rate environmental esg](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__environmental_esg.md) | 19.36% | SE ±2.37 pp | 49 / 49 | $12.21 | 11,761.1source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 19.355 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 12.206644 \| USD \| per_test \| Lower is better \| \| `latency` \| 11761.102 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 131072 provider Together AI reasoning Not reported Reasoning effort xhigh Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.37 pp sourcePopulation 81 stderr 2.37 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label meta/muse-spark-open Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__environmental_esg/meta~1muse-spark-open` | | | | | | | | | [EMB v1.1 / comps](https://benchize.com/benchmarks/vals-ai%3Aemb-v1-1%3Acomps.md) | 56.04% | SE ±6.92 pp | 19 / 19 | $6.33 | 4,497.27source duration | [Vals AI · 1.1](https://www.vals.ai/_astro/benchmark_view_emb_v1_1.11L06J7L.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 56.044 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 6.33156 \| USD \| per_test \| Lower is better \| \| `latency` \| 4497.271 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 131072 provider Meta reasoning Not reported Reasoning effort xhigh Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±6.92 pp sourcePopulation 30 stderr 6.916 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb_v1_1.11L06J7L.json) Source Vals AI Original model label meta/muse-spark-open Source version 1.1 Observed 2026-10-21T16:45:55.456Z Evidence SHA-256 `eea2ef2c83a6dc7d553c7c6531c287710ea7ffbc56ee1ccec06c85b104b2408e` Source locator `/tasks/comps/meta~1muse-spark-open` | | | | | | | | | [Finance Agent v2.1 / all pass](https://benchize.com/benchmarks/vals-ai%3Afabv2-1%3Aall_pass.md) | 48.96% | SE ±2.2 pp | 11 / 18 | $2.76 | 1,612.35source duration | [Vals AI · 2.1](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2_1.ZbMhy9xo.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 48.957 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 2.758047 \| USD \| per_test \| Lower is better \| \| `latency` \| 1612.347 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 131072 provider Together AI reasoning Not reported Reasoning effort xhigh Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.2 pp sourcePopulation 28 stderr 2.204 tokenTotals {"cache_read_tokens":2388431856,"cache_write_tokens":null,"input_tokens":3043998798,"output_tokens":14268191,"reasoning_tokens":7831933} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2_1.ZbMhy9xo.json) Source Vals AI Original model label meta/muse-spark-open Source version 2.1 Observed 2026-10-21T16:45:54.954Z Evidence SHA-256 `f1b0c2d7dca4e73b52136862a9be059f7c8481c69f15d705ce90bf3bd9f78429` Source locator `/tasks/all_pass/meta~1muse-spark-open` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / emerging companies venture capital](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Aemerging_companies_venture_capital.md) | 20% | SE ±20 pp | 6 / 49 | $7.1 | 11,588.73source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 20 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 7.104223 \| USD \| per_test \| Lower is better \| \| `latency` \| 11588.732 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 131072 provider Together AI reasoning Not reported Reasoning effort xhigh Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±20 pp sourcePopulation 81 stderr 20 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label meta/muse-spark-open Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/emerging_companies_venture_capital/meta~1muse-spark-open` | | | | | | | | | [Finance Agent v2.1 / adjustments](https://benchize.com/benchmarks/vals-ai%3Afabv2-1%3Aadjustments.md) | 63% | SE ±6.8 pp | 8 / 18 | $2.52 | 1,335.69source duration | [Vals AI · 2.1](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2_1.ZbMhy9xo.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 62.995 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 2.51918 \| USD \| per_test \| Lower is better \| \| `latency` \| 1335.692 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 131072 provider Together AI reasoning Not reported Reasoning effort xhigh Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±6.8 pp sourcePopulation 28 stderr 6.801 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2_1.ZbMhy9xo.json) Source Vals AI Original model label meta/muse-spark-open Source version 2.1 Observed 2026-10-21T16:45:54.954Z Evidence SHA-256 `f1b0c2d7dca4e73b52136862a9be059f7c8481c69f15d705ce90bf3bd9f78429` Source locator `/tasks/adjustments/meta~1muse-spark-open` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / criteria pass rate intellectual property](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__intellectual_property.md) | 59.01% | SE ±2.93 pp | 45 / 49 | $2.18 | 3,662.96source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 59.011 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 2.179175 \| USD \| per_test \| Lower is better \| \| `latency` \| 3662.956 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 131072 provider Together AI reasoning Not reported Reasoning effort xhigh Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.93 pp sourcePopulation 81 stderr 2.929 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label meta/muse-spark-open Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__intellectual_property/meta~1muse-spark-open` | | | | | | | | | [Legal Research Bench / weighted pass administrative regulatory](https://benchize.com/benchmarks/vals-ai%3Alegal_research%3Aweighted_pass__administrative_regulatory.md) | 87.94% | SE ±2.52 pp | 13 / 50 | $1.42 | 1,671.25source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 87.942 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 1.423111 \| USD \| per_test \| Lower is better \| \| `latency` \| 1671.25 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 131072 provider Together AI reasoning Not reported Reasoning effort xhigh Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.52 pp sourcePopulation 82 stderr 2.515 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) Source Vals AI Original model label meta/muse-spark-open Source version 1 Observed 2026-10-21T16:45:52.389Z Evidence SHA-256 `26c58dfd47d0d2dc86f33c6d056f30309e0ee63570e7cd752c593b385f9d862e` Source locator `/tasks/weighted_pass__administrative_regulatory/meta~1muse-spark-open` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / arbitration international dispute resolution](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Aarbitration_international_dispute_resolution.md) | 20% | SE ±20 pp | 1 / 49 | $6.39 | 5,196.48source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 20 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 6.387045 \| USD \| per_test \| Lower is better \| \| `latency` \| 5196.484 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 131072 provider Together AI reasoning Not reported Reasoning effort xhigh Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±20 pp sourcePopulation 81 stderr 20 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label meta/muse-spark-open Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/arbitration_international_dispute_resolution/meta~1muse-spark-open` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / capital markets](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acapital_markets.md) | 20% | SE ±20 pp | 4 / 49 | $2.18 | 3,364.68source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 20 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 2.177918 \| USD \| per_test \| Lower is better \| \| `latency` \| 3364.68 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 131072 provider Together AI reasoning Not reported Reasoning effort xhigh Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±20 pp sourcePopulation 81 stderr 20 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label meta/muse-spark-open Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/capital_markets/meta~1muse-spark-open` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / criteria pass rate](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate.md) | 63.85% | SE ±0.58 pp | 41 / 49 | $6.83 | 8,090.07source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 63.849 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 6.834878 \| USD \| per_test \| Lower is better \| \| `latency` \| 8090.065 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 131072 provider Together AI reasoning Not reported Reasoning effort xhigh Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.58 pp sourcePopulation 81 stderr 0.581 tokenTotals {"cache_read_tokens":595452080,"cache_write_tokens":null,"input_tokens":1146205590,"output_tokens":9982506,"reasoning_tokens":5173489} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label meta/muse-spark-open Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate/meta~1muse-spark-open` | | | | | | | | | [Finance Agent v2.1 / general qualitative analysis](https://benchize.com/benchmarks/vals-ai%3Afabv2-1%3Ageneral_qualitative_analysis.md) | 75.29% | SE ±5.48 pp | 2 / 18 | $2.01 | 1,581.95source duration | [Vals AI · 2.1](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2_1.ZbMhy9xo.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 75.291 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 2.006504 \| USD \| per_test \| Lower is better \| \| `latency` \| 1581.951 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 131072 provider Together AI reasoning Not reported Reasoning effort xhigh Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±5.48 pp sourcePopulation 28 stderr 5.479 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2_1.ZbMhy9xo.json) Source Vals AI Original model label meta/muse-spark-open Source version 2.1 Observed 2026-10-21T16:45:54.954Z Evidence SHA-256 `f1b0c2d7dca4e73b52136862a9be059f7c8481c69f15d705ce90bf3bd9f78429` Source locator `/tasks/general_qualitative_analysis/meta~1muse-spark-open` | | | | | | | | | [Finance Agent v2.1 / earnings analysis](https://benchize.com/benchmarks/vals-ai%3Afabv2-1%3Aearnings_analysis.md) | 81.26% | SE ±5.47 pp | 15 / 18 | $1.22 | 1,037.85source duration | [Vals AI · 2.1](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2_1.ZbMhy9xo.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 81.263 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 1.221736 \| USD \| per_test \| Lower is better \| \| `latency` \| 1037.852 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 131072 provider Together AI reasoning Not reported Reasoning effort xhigh Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±5.47 pp sourcePopulation 28 stderr 5.468 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2_1.ZbMhy9xo.json) Source Vals AI Original model label meta/muse-spark-open Source version 2.1 Observed 2026-10-21T16:45:54.954Z Evidence SHA-256 `f1b0c2d7dca4e73b52136862a9be059f7c8481c69f15d705ce90bf3bd9f78429` Source locator `/tasks/earnings_analysis/meta~1muse-spark-open` | | | | | | | | | [Code Migration v1.1 / code quality](https://benchize.com/benchmarks/vals-ai%3Acode-migration-v1-1%3Acode_quality.md) | 2.4% | SE ±0.04 pp | 17 / 17 | $3.77 | 5,021.25source duration | [Vals AI · 1.1](https://www.vals.ai/_astro/benchmark_view_code_migration_v1_1.bJHk6HEY.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 2.398 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 3.767449 \| USD \| per_test \| Lower is better \| \| `latency` \| 5021.245 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 131072 provider Meta reasoning Not reported Reasoning effort xhigh Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.04 pp sourcePopulation 26 stderr 0.039 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_code_migration_v1_1.bJHk6HEY.json) Source Vals AI Original model label meta/muse-spark-open Source version 1.1 Observed 2026-10-21T16:46:09.631Z Evidence SHA-256 `352faac603c73f05cc01e89214fb4adcafa05ea7432ea71ea9903a6b5fe36b08` Source locator `/tasks/code_quality/meta~1muse-spark-open` | | | | | | | | | [Terminal-Bench 4.0 / operations](https://benchize.com/benchmarks/vals-ai%3Aterminal-bench-4%3Aoperations.md) | 7.41% | SE ±3.7 pp | 24 / 36 | — | — | [Vals AI · 4.0](https://www.vals.ai/_astro/benchmark_view_terminal_bench_4.CAnGyzC_.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 7.407 \| percent \| reported \| Higher is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 131072 provider Together AI reasoning Not reported Reasoning effort xhigh Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±3.7 pp sourcePopulation 52 stderr 3.704 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_terminal_bench_4.CAnGyzC_.json) Source Vals AI Original model label meta/muse-spark-open Source version 4.0 Observed 2026-10-21T16:46:10.642Z Evidence SHA-256 `98f6964580e901f1aa143db664bd92aa558415f643569492545d67c47d7f519b` Source locator `/tasks/operations/meta~1muse-spark-open` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / criteria pass rate corporate governance](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__corporate_governance.md) | 78.77% | SE ±2.4 pp | 34 / 49 | $10.58 | 9,835.31source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 78.767 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 10.580113 \| USD \| per_test \| Lower is better \| \| `latency` \| 9835.314 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 131072 provider Together AI reasoning Not reported Reasoning effort xhigh Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.4 pp sourcePopulation 81 stderr 2.397 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label meta/muse-spark-open Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__corporate_governance/meta~1muse-spark-open` | | | | | | | | | [Code Migration v1.1 / cobol](https://benchize.com/benchmarks/vals-ai%3Acode-migration-v1-1%3Acobol.md) | 62.5% | SE ±11.77 pp | 15 / 17 | $1.88 | 3,103.49source duration | [Vals AI · 1.1](https://www.vals.ai/_astro/benchmark_view_code_migration_v1_1.bJHk6HEY.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 62.5 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 1.884248 \| USD \| per_test \| Lower is better \| \| `latency` \| 3103.489 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 131072 provider Meta reasoning Not reported Reasoning effort xhigh Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±11.77 pp sourcePopulation 26 stderr 11.767 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_code_migration_v1_1.bJHk6HEY.json) Source Vals AI Original model label meta/muse-spark-open Source version 1.1 Observed 2026-10-21T16:46:09.631Z Evidence SHA-256 `352faac603c73f05cc01e89214fb4adcafa05ea7432ea71ea9903a6b5fe36b08` Source locator `/tasks/cobol/meta~1muse-spark-open` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / criteria pass rate funds asset management](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__funds_asset_management.md) | 35.86% | SE ±2.59 pp | 45 / 49 | $8.73 | 9,037.94source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 35.86 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 8.728109 \| USD \| per_test \| Lower is better \| \| `latency` \| 9037.942 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 131072 provider Together AI reasoning Not reported Reasoning effort xhigh Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.59 pp sourcePopulation 81 stderr 2.593 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label meta/muse-spark-open Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__funds_asset_management/meta~1muse-spark-open` | | | | | | | | | [Code Migration / overall](https://benchize.com/benchmarks/vals-ai%3Acode-migration%3Aoverall.md) | 21.17% | SE ±3.47 pp | 37 / 47 | $9.04 | 6,790.7source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_code_migration.Dttzoqk3.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 21.171 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 9.042387 \| USD \| per_test \| Lower is better \| \| `latency` \| 6790.697 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 131072 provider Together AI reasoning Not reported Reasoning effort xhigh Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±3.47 pp sourcePopulation 79 stderr 3.472 tokenTotals {"cache_read_tokens":1937882464,"cache_write_tokens":null,"input_tokens":2648676598,"output_tokens":14008508,"reasoning_tokens":8828852} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_code_migration.Dttzoqk3.json) Source Vals AI Original model label meta/muse-spark-open Source version 1 Observed 2026-10-21T16:46:12.712Z Evidence SHA-256 `99194597f2c76c402195be47839409046e18bd30454616da6dd40b736a088fa4` Source locator `/tasks/overall/meta~1muse-spark-open` | | | | | | | | | [Finance Agent v2.1 / market analysis](https://benchize.com/benchmarks/vals-ai%3Afabv2-1%3Amarket_analysis.md) | 56.64% | SE ±7.03 pp | 15 / 18 | $4.56 | 1,287.19source duration | [Vals AI · 2.1](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2_1.ZbMhy9xo.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 56.642 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 4.555468 \| USD \| per_test \| Lower is better \| \| `latency` \| 1287.19 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 131072 provider Together AI reasoning Not reported Reasoning effort xhigh Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±7.03 pp sourcePopulation 28 stderr 7.026 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2_1.ZbMhy9xo.json) Source Vals AI Original model label meta/muse-spark-open Source version 2.1 Observed 2026-10-21T16:45:54.954Z Evidence SHA-256 `f1b0c2d7dca4e73b52136862a9be059f7c8481c69f15d705ce90bf3bd9f78429` Source locator `/tasks/market_analysis/meta~1muse-spark-open` | | | | | | | | | [Terminal-Bench 4.0 / security](https://benchize.com/benchmarks/vals-ai%3Aterminal-bench-4%3Asecurity.md) | 6.67% | SE ±6.67 pp | 24 / 36 | — | — | [Vals AI · 4.0](https://www.vals.ai/_astro/benchmark_view_terminal_bench_4.CAnGyzC_.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 6.667 \| percent \| reported \| Higher is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 131072 provider Together AI reasoning Not reported Reasoning effort xhigh Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±6.67 pp sourcePopulation 52 stderr 6.667 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_terminal_bench_4.CAnGyzC_.json) Source Vals AI Original model label meta/muse-spark-open Source version 4.0 Observed 2026-10-21T16:46:10.642Z Evidence SHA-256 `98f6964580e901f1aa143db664bd92aa558415f643569492545d67c47d7f519b` Source locator `/tasks/security/meta~1muse-spark-open` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / corporate governance](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acorporate_governance.md) | 40% | SE ±24.5 pp | 1 / 49 | $10.58 | 9,835.31source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 40 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 10.580113 \| USD \| per_test \| Lower is better \| \| `latency` \| 9835.314 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 131072 provider Together AI reasoning Not reported Reasoning effort xhigh Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±24.5 pp sourcePopulation 81 stderr 24.495 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label meta/muse-spark-open Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/corporate_governance/meta~1muse-spark-open` | | | | | | | | | [EMB / comps](https://benchize.com/benchmarks/vals-ai%3Aemb%3Acomps.md) | 40.99% | SE ±8.4 pp | 40 / 47 | $6.33 | 4,497.27source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 40.991 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 6.33156 \| USD \| per_test \| Lower is better \| \| `latency` \| 4497.271 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 131072 provider Together AI reasoning Not reported Reasoning effort xhigh Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±8.4 pp sourcePopulation 76 stderr 8.397 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) Source Vals AI Original model label meta/muse-spark-open Source version 1 Observed 2026-10-21T16:45:57.495Z Evidence SHA-256 `d807aee620ea47152a1fca0257a8dd12a1be4512c76cc1744c1ae9e285180918` Source locator `/tasks/comps/meta~1muse-spark-open` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / criteria pass rate immigration](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__immigration.md) | 36% | SE ±2.9 pp | 46 / 49 | $3.36 | 6,677.02source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 36 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 3.360409 \| USD \| per_test \| Lower is better \| \| `latency` \| 6677.02 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 131072 provider Together AI reasoning Not reported Reasoning effort xhigh Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.9 pp sourcePopulation 81 stderr 2.9 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label meta/muse-spark-open Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__immigration/meta~1muse-spark-open` | | | | | | | | | [EMB v1.1 / scratch](https://benchize.com/benchmarks/vals-ai%3Aemb-v1-1%3Ascratch.md) | 61.72% | SE ±2.7 pp | 17 / 19 | $3.69 | 4,418.17source duration | [Vals AI · 1.1](https://www.vals.ai/_astro/benchmark_view_emb_v1_1.11L06J7L.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 61.72 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 3.688473 \| USD \| per_test \| Lower is better \| \| `latency` \| 4418.173 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 131072 provider Meta reasoning Not reported Reasoning effort xhigh Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.7 pp sourcePopulation 30 stderr 2.699 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb_v1_1.11L06J7L.json) Source Vals AI Original model label meta/muse-spark-open Source version 1.1 Observed 2026-10-21T16:45:55.456Z Evidence SHA-256 `eea2ef2c83a6dc7d553c7c6531c287710ea7ffbc56ee1ccec06c85b104b2408e` Source locator `/tasks/scratch/meta~1muse-spark-open` | | | | | | | | | [Finance Agent v2.1 / comparables](https://benchize.com/benchmarks/vals-ai%3Afabv2-1%3Acomparables.md) | 54.44% | SE ±6.59 pp | 13 / 18 | $1.6 | 2,030.11source duration | [Vals AI · 2.1](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2_1.ZbMhy9xo.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 54.442 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 1.599423 \| USD \| per_test \| Lower is better \| \| `latency` \| 2030.108 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 131072 provider Together AI reasoning Not reported Reasoning effort xhigh Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±6.59 pp sourcePopulation 28 stderr 6.588 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2_1.ZbMhy9xo.json) Source Vals AI Original model label meta/muse-spark-open Source version 2.1 Observed 2026-10-21T16:45:54.954Z Evidence SHA-256 `f1b0c2d7dca4e73b52136862a9be059f7c8481c69f15d705ce90bf3bd9f78429` Source locator `/tasks/comparables/meta~1muse-spark-open` | | | | | | | | | [Legal Research Bench / weighted pass civil litigation](https://benchize.com/benchmarks/vals-ai%3Alegal_research%3Aweighted_pass__civil_litigation.md) | 88.25% | SE ±3.24 pp | 17 / 50 | $1.59 | 1,273.91source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 88.253 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 1.590079 \| USD \| per_test \| Lower is better \| \| `latency` \| 1273.914 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 131072 provider Together AI reasoning Not reported Reasoning effort xhigh Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±3.24 pp sourcePopulation 82 stderr 3.238 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) Source Vals AI Original model label meta/muse-spark-open Source version 1 Observed 2026-10-21T16:45:52.389Z Evidence SHA-256 `26c58dfd47d0d2dc86f33c6d056f30309e0ee63570e7cd752c593b385f9d862e` Source locator `/tasks/weighted_pass__civil_litigation/meta~1muse-spark-open` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / corporate ma](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acorporate_ma.md) | 20% | SE ±20 pp | 1 / 49 | $4.07 | 5,887.68source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 20 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 4.068228 \| USD \| per_test \| Lower is better \| \| `latency` \| 5887.684 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 131072 provider Together AI reasoning Not reported Reasoning effort xhigh Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±20 pp sourcePopulation 81 stderr 20 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label meta/muse-spark-open Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/corporate_ma/meta~1muse-spark-open` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / intellectual property](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Aintellectual_property.md) | 20% | SE ±20 pp | 1 / 49 | $2.18 | 3,662.96source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 20 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 2.179175 \| USD \| per_test \| Lower is better \| \| `latency` \| 3662.956 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 131072 provider Together AI reasoning Not reported Reasoning effort xhigh Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±20 pp sourcePopulation 81 stderr 20 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label meta/muse-spark-open Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/intellectual_property/meta~1muse-spark-open` | | | | | | | | | [Legal Research Bench / civil litigation](https://benchize.com/benchmarks/vals-ai%3Alegal_research%3Acivil_litigation.md) | 25.93% | SE ±8.59 pp | 32 / 50 | $1.59 | 1,273.91source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 25.926 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 1.590079 \| USD \| per_test \| Lower is better \| \| `latency` \| 1273.914 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 131072 provider Together AI reasoning Not reported Reasoning effort xhigh Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±8.59 pp sourcePopulation 82 stderr 8.594 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) Source Vals AI Original model label meta/muse-spark-open Source version 1 Observed 2026-10-21T16:45:52.389Z Evidence SHA-256 `26c58dfd47d0d2dc86f33c6d056f30309e0ee63570e7cd752c593b385f9d862e` Source locator `/tasks/civil_litigation/meta~1muse-spark-open` | | | | | | | | | [Finance Agent v2.1 / precedents](https://benchize.com/benchmarks/vals-ai%3Afabv2-1%3Aprecedents.md) | 39.26% | SE ±7.05 pp | 9 / 18 | $3.95 | 2,050.6source duration | [Vals AI · 2.1](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2_1.ZbMhy9xo.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 39.255 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 3.945511 \| USD \| per_test \| Lower is better \| \| `latency` \| 2050.598 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 131072 provider Together AI reasoning Not reported Reasoning effort xhigh Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±7.05 pp sourcePopulation 28 stderr 7.053 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2_1.ZbMhy9xo.json) Source Vals AI Original model label meta/muse-spark-open Source version 2.1 Observed 2026-10-21T16:45:54.954Z Evidence SHA-256 `f1b0c2d7dca4e73b52136862a9be059f7c8481c69f15d705ce90bf3bd9f78429` Source locator `/tasks/precedents/meta~1muse-spark-open` | | | | | | | | | [Legal Research Bench / constitutional civil rights](https://benchize.com/benchmarks/vals-ai%3Alegal_research%3Aconstitutional_civil_rights.md) | 44.74% | SE ±8.17 pp | 4 / 50 | $0.9 | 1,835.59source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 44.737 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.898996 \| USD \| per_test \| Lower is better \| \| `latency` \| 1835.592 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 131072 provider Together AI reasoning Not reported Reasoning effort xhigh Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±8.17 pp sourcePopulation 82 stderr 8.174 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) Source Vals AI Original model label meta/muse-spark-open Source version 1 Observed 2026-10-21T16:45:52.389Z Evidence SHA-256 `26c58dfd47d0d2dc86f33c6d056f30309e0ee63570e7cd752c593b385f9d862e` Source locator `/tasks/constitutional_civil_rights/meta~1muse-spark-open` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / criteria pass rate corporate ma](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__corporate_ma.md) | 67.63% | SE ±2.65 pp | 42 / 49 | $4.07 | 5,887.68source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 67.628 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 4.068228 \| USD \| per_test \| Lower is better \| \| `latency` \| 5887.684 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 131072 provider Together AI reasoning Not reported Reasoning effort xhigh Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.65 pp sourcePopulation 81 stderr 2.653 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label meta/muse-spark-open Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__corporate_ma/meta~1muse-spark-open` | | | | | | | | | [Vibe Code Bench v1.1 / overall](https://benchize.com/benchmarks/vals-ai%3Avibe-code%3Aoverall.md) | 76.19% | SE ±3.32 pp | 25 / 59 | $1.66 | 1,869.17source duration | [Vals AI · 1.1](https://www.vals.ai/_astro/benchmark_view_vibe_code_bench.DcEBMbCA.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 76.187 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 1.655609 \| USD \| per_test \| Lower is better \| \| `latency` \| 1869.172 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness OpenHands maxOutputTokens 131072 provider Together AI reasoning Not reported Reasoning effort xhigh Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±3.32 pp sourcePopulation 116 stderr 3.317 tokenTotals {"cache_read_tokens":189265312,"cache_write_tokens":null,"input_tokens":225685852,"output_tokens":2085879,"reasoning_tokens":952456} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_vibe_code_bench.DcEBMbCA.json) Source Vals AI Original model label meta/muse-spark-open Source version 1.1 Observed 2026-10-21T16:46:08.625Z Evidence SHA-256 `0e8c23f521a0f751c5592952620e197b376cade41c78578461b7aa5740a173e7` Source locator `/tasks/overall/meta~1muse-spark-open` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / international trade sanctions](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Ainternational_trade_sanctions.md) | 20% | SE ±20 pp | 1 / 49 | $9.69 | 12,102.36source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 20 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 9.688157 \| USD \| per_test \| Lower is better \| \| `latency` \| 12102.358 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 131072 provider Together AI reasoning Not reported Reasoning effort xhigh Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±20 pp sourcePopulation 81 stderr 20 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label meta/muse-spark-open Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/international_trade_sanctions/meta~1muse-spark-open` | | | | | | | | | [Legal Research Bench / weighted pass immigration](https://benchize.com/benchmarks/vals-ai%3Alegal_research%3Aweighted_pass__immigration.md) | 76.75% | SE ±8.47 pp | 34 / 50 | $0.57 | 1,537.67source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 76.745 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.570169 \| USD \| per_test \| Lower is better \| \| `latency` \| 1537.666 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 131072 provider Together AI reasoning Not reported Reasoning effort xhigh Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±8.47 pp sourcePopulation 82 stderr 8.465 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) Source Vals AI Original model label meta/muse-spark-open Source version 1 Observed 2026-10-21T16:45:52.389Z Evidence SHA-256 `26c58dfd47d0d2dc86f33c6d056f30309e0ee63570e7cd752c593b385f9d862e` Source locator `/tasks/weighted_pass__immigration/meta~1muse-spark-open` | | | | | | | | | [Tax Agent Bench / controversy precedence analysis](https://benchize.com/benchmarks/vals-ai%3Atax_agent_bench%3Acontroversy_precedence_analysis.md) | 10.33% | SE ±4.97 pp | 17 / 46 | $1.14 | 826.16source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 10.333 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 1.13869 \| USD \| per_test \| Lower is better \| \| `latency` \| 826.163 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 131072 provider Together AI reasoning Not reported Reasoning effort xhigh Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±4.97 pp sourcePopulation 73 stderr 4.971 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) Source Vals AI Original model label meta/muse-spark-open Source version 1 Observed 2026-10-21T16:45:55.958Z Evidence SHA-256 `446db182b67efb7923b3187a4040ec3a3ea492b8ad699a2b6d708d714cc063ac` Source locator `/tasks/controversy_precedence_analysis/meta~1muse-spark-open` | | | | | | | | | [EMB v1.1 / dataroom](https://benchize.com/benchmarks/vals-ai%3Aemb-v1-1%3Adataroom.md) | 79.54% | SE ±5.95 pp | 18 / 19 | $6.15 | 5,754.88source duration | [Vals AI · 1.1](https://www.vals.ai/_astro/benchmark_view_emb_v1_1.11L06J7L.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 79.535 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 6.145752 \| USD \| per_test \| Lower is better \| \| `latency` \| 5754.881 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 131072 provider Meta reasoning Not reported Reasoning effort xhigh Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±5.95 pp sourcePopulation 30 stderr 5.952 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb_v1_1.11L06J7L.json) Source Vals AI Original model label meta/muse-spark-open Source version 1.1 Observed 2026-10-21T16:45:55.456Z Evidence SHA-256 `eea2ef2c83a6dc7d553c7c6531c287710ea7ffbc56ee1ccec06c85b104b2408e` Source locator `/tasks/dataroom/meta~1muse-spark-open` | | | | | | | | | [Legal Research Bench / weighted pass constitutional civil rights](https://benchize.com/benchmarks/vals-ai%3Alegal_research%3Aweighted_pass__constitutional_civil_rights.md) | 89.18% | SE ±2.52 pp | 4 / 50 | $0.9 | 1,835.59source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 89.18 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.898996 \| USD \| per_test \| Lower is better \| \| `latency` \| 1835.592 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 131072 provider Together AI reasoning Not reported Reasoning effort xhigh Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.52 pp sourcePopulation 82 stderr 2.515 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) Source Vals AI Original model label meta/muse-spark-open Source version 1 Observed 2026-10-21T16:45:52.389Z Evidence SHA-256 `26c58dfd47d0d2dc86f33c6d056f30309e0ee63570e7cd752c593b385f9d862e` Source locator `/tasks/weighted_pass__constitutional_civil_rights/meta~1muse-spark-open` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / healthcare life sciences](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Ahealthcare_life_sciences.md) | 20% | SE ±20 pp | 5 / 49 | $5.03 | 7,857.63source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 20 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 5.029791 \| USD \| per_test \| Lower is better \| \| `latency` \| 7857.632 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 131072 provider Together AI reasoning Not reported Reasoning effort xhigh Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±20 pp sourcePopulation 81 stderr 20 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label meta/muse-spark-open Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/healthcare_life_sciences/meta~1muse-spark-open` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / criteria pass rate real estate](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__real_estate.md) | 57.43% | SE ±2.67 pp | 40 / 49 | $5.39 | 3,705.41source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 57.434 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 5.388792 \| USD \| per_test \| Lower is better \| \| `latency` \| 3705.408 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 131072 provider Together AI reasoning Not reported Reasoning effort xhigh Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.67 pp sourcePopulation 81 stderr 2.674 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label meta/muse-spark-open Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__real_estate/meta~1muse-spark-open` | | | | | | | | | [EMB v1.1 / ma](https://benchize.com/benchmarks/vals-ai%3Aemb-v1-1%3Ama.md) | 71.13% | SE ±5.32 pp | 17 / 19 | $3.86 | 5,014.64source duration | [Vals AI · 1.1](https://www.vals.ai/_astro/benchmark_view_emb_v1_1.11L06J7L.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 71.133 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 3.855837 \| USD \| per_test \| Lower is better \| \| `latency` \| 5014.642 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 131072 provider Meta reasoning Not reported Reasoning effort xhigh Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±5.32 pp sourcePopulation 30 stderr 5.319 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb_v1_1.11L06J7L.json) Source Vals AI Original model label meta/muse-spark-open Source version 1.1 Observed 2026-10-21T16:45:55.456Z Evidence SHA-256 `eea2ef2c83a6dc7d553c7c6531c287710ea7ffbc56ee1ccec06c85b104b2408e` Source locator `/tasks/ma/meta~1muse-spark-open` | | | | | | | | | [Finance Agent (v2) / disclosure analysis](https://benchize.com/benchmarks/vals-ai%3Afabv2%3Adisclosure_analysis.md) | 67.78% | SE ±6.41 pp | 10 / 50 | $0.9 | 1,521.21source duration | [Vals AI · 2](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 67.78 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.897488 \| USD \| per_test \| Lower is better \| \| `latency` \| 1521.214 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 131072 provider Together AI reasoning Not reported Reasoning effort xhigh Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±6.41 pp sourcePopulation 80 stderr 6.408 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) Source Vals AI Original model label meta/muse-spark-open Source version 2 Observed 2026-10-21T16:45:57.035Z Evidence SHA-256 `f731b20a26c126725d125519f2b5841aa45254656dba859a9570d7ce7e5d8191` Source locator `/tasks/disclosure_analysis/meta~1muse-spark-open` | | | | | | | | | [EMB / operating](https://benchize.com/benchmarks/vals-ai%3Aemb%3Aoperating.md) | 63.15% | SE ±6.53 pp | 22 / 47 | $4.2 | 4,388.69source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 63.149 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 4.195928 \| USD \| per_test \| Lower is better \| \| `latency` \| 4388.685 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 131072 provider Together AI reasoning Not reported Reasoning effort xhigh Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±6.53 pp sourcePopulation 76 stderr 6.528 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) Source Vals AI Original model label meta/muse-spark-open Source version 1 Observed 2026-10-21T16:45:57.495Z Evidence SHA-256 `d807aee620ea47152a1fca0257a8dd12a1be4512c76cc1744c1ae9e285180918` Source locator `/tasks/operating/meta~1muse-spark-open` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / banking finance](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Abanking_finance.md) | 0% | SE ±0 pp | 5 / 49 | $4.28 | 6,800.68source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 4.275836 \| USD \| per_test \| Lower is better \| \| `latency` \| 6800.68 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 131072 provider Together AI reasoning Not reported Reasoning effort xhigh Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 81 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label meta/muse-spark-open Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/banking_finance/meta~1muse-spark-open` | | | | | | | | | [Finance Agent v2.1 / overall](https://benchize.com/benchmarks/vals-ai%3Afabv2-1%3Aoverall.md) | 60.67% | SE ±2.06 pp | 12 / 18 | $2.76 | 1,612.35source duration | [Vals AI · 2.1](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2_1.ZbMhy9xo.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 60.665 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 2.758047 \| USD \| per_test \| Lower is better \| \| `latency` \| 1612.347 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 131072 provider Together AI reasoning Not reported Reasoning effort xhigh Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.06 pp sourcePopulation 28 stderr 2.056 tokenTotals {"cache_read_tokens":2388431856,"cache_write_tokens":null,"input_tokens":3043998798,"output_tokens":14268191,"reasoning_tokens":7831933} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2_1.ZbMhy9xo.json) Source Vals AI Original model label meta/muse-spark-open Source version 2.1 Observed 2026-10-21T16:45:54.954Z Evidence SHA-256 `f1b0c2d7dca4e73b52136862a9be059f7c8481c69f15d705ce90bf3bd9f78429` Source locator `/tasks/overall/meta~1muse-spark-open` | | | | | | | | | [Tax Agent Bench / fact pattern analysis](https://benchize.com/benchmarks/vals-ai%3Atax_agent_bench%3Afact_pattern_analysis.md) | 26.74% | SE ±5.4 pp | 25 / 46 | $2.53 | 995.72source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 26.744 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 2.528872 \| USD \| per_test \| Lower is better \| \| `latency` \| 995.717 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 131072 provider Together AI reasoning Not reported Reasoning effort xhigh Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±5.4 pp sourcePopulation 73 stderr 5.404 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) Source Vals AI Original model label meta/muse-spark-open Source version 1 Observed 2026-10-21T16:45:55.958Z Evidence SHA-256 `446db182b67efb7923b3187a4040ec3a3ea492b8ad699a2b6d708d714cc063ac` Source locator `/tasks/fact_pattern_analysis/meta~1muse-spark-open` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / criteria pass rate banking finance](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__banking_finance.md) | 77.7% | SE ±2.54 pp | 41 / 49 | $4.28 | 6,800.68source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 77.695 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 4.275836 \| USD \| per_test \| Lower is better \| \| `latency` \| 6800.68 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 131072 provider Together AI reasoning Not reported Reasoning effort xhigh Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.54 pp sourcePopulation 81 stderr 2.543 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label meta/muse-spark-open Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__banking_finance/meta~1muse-spark-open` | | | | | | | | | [Code Migration v1.1 / overall](https://benchize.com/benchmarks/vals-ai%3Acode-migration-v1-1%3Aoverall.md) | 26.89% | SE ±3.23 pp | 16 / 17 | $3.77 | 5,021.25source duration | [Vals AI · 1.1](https://www.vals.ai/_astro/benchmark_view_code_migration_v1_1.bJHk6HEY.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 26.893 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 3.767449 \| USD \| per_test \| Lower is better \| \| `latency` \| 5021.245 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 131072 provider Meta reasoning Not reported Reasoning effort xhigh Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±3.23 pp sourcePopulation 26 stderr 3.229 tokenTotals {"cache_read_tokens":1500265728,"cache_write_tokens":0,"input_tokens":1717177941,"output_tokens":11784542,"reasoning_tokens":5997374} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_code_migration_v1_1.bJHk6HEY.json) Source Vals AI Original model label meta/muse-spark-open Source version 1.1 Observed 2026-10-21T16:46:09.631Z Evidence SHA-256 `352faac603c73f05cc01e89214fb4adcafa05ea7432ea71ea9903a6b5fe36b08` Source locator `/tasks/overall/meta~1muse-spark-open` | | | | | | | | | [Tax Agent Bench / weighted pass rule source lookup](https://benchize.com/benchmarks/vals-ai%3Atax_agent_bench%3Aweighted_pass__rule_source_lookup.md) | 60.29% | SE ±6.07 pp | 14 / 46 | $0.19 | 430.4source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 60.287 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.193793 \| USD \| per_test \| Lower is better \| \| `latency` \| 430.403 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 131072 provider Together AI reasoning Not reported Reasoning effort xhigh Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±6.07 pp sourcePopulation 73 stderr 6.065 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) Source Vals AI Original model label meta/muse-spark-open Source version 1 Observed 2026-10-21T16:45:55.958Z Evidence SHA-256 `446db182b67efb7923b3187a4040ec3a3ea492b8ad699a2b6d708d714cc063ac` Source locator `/tasks/weighted_pass__rule_source_lookup/meta~1muse-spark-open` | | | | | | | | | [EMB v1.1 / operating](https://benchize.com/benchmarks/vals-ai%3Aemb-v1-1%3Aoperating.md) | 68.65% | SE ±5.16 pp | 13 / 19 | $4.2 | 4,388.69source duration | [Vals AI · 1.1](https://www.vals.ai/_astro/benchmark_view_emb_v1_1.11L06J7L.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 68.645 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 4.195928 \| USD \| per_test \| Lower is better \| \| `latency` \| 4388.685 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 131072 provider Meta reasoning Not reported Reasoning effort xhigh Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±5.16 pp sourcePopulation 30 stderr 5.159 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb_v1_1.11L06J7L.json) Source Vals AI Original model label meta/muse-spark-open Source version 1.1 Observed 2026-10-21T16:45:55.456Z Evidence SHA-256 `eea2ef2c83a6dc7d553c7c6531c287710ea7ffbc56ee1ccec06c85b104b2408e` Source locator `/tasks/operating/meta~1muse-spark-open` | | | | | | | | | [Tax Agent Bench / forms filings](https://benchize.com/benchmarks/vals-ai%3Atax_agent_bench%3Aforms_filings.md) | 11.67% | SE ±4.84 pp | 29 / 46 | $0.29 | 848.02source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 11.667 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.290359 \| USD \| per_test \| Lower is better \| \| `latency` \| 848.017 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 131072 provider Together AI reasoning Not reported Reasoning effort xhigh Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±4.84 pp sourcePopulation 73 stderr 4.844 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) Source Vals AI Original model label meta/muse-spark-open Source version 1 Observed 2026-10-21T16:45:55.958Z Evidence SHA-256 `446db182b67efb7923b3187a4040ec3a3ea492b8ad699a2b6d708d714cc063ac` Source locator `/tasks/forms_filings/meta~1muse-spark-open` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / criteria pass rate tax](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__tax.md) | 68.9% | SE ±2.68 pp | 37 / 49 | $6.12 | 7,929.84source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 68.896 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 6.121789 \| USD \| per_test \| Lower is better \| \| `latency` \| 7929.838 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 131072 provider Together AI reasoning Not reported Reasoning effort xhigh Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.68 pp sourcePopulation 81 stderr 2.682 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label meta/muse-spark-open Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__tax/meta~1muse-spark-open` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / antitrust competition](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Aantitrust_competition.md) | 20% | SE ±20 pp | 3 / 49 | $14.32 | 14,200.24source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 20 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 14.31638 \| USD \| per_test \| Lower is better \| \| `latency` \| 14200.244 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 131072 provider Together AI reasoning Not reported Reasoning effort xhigh Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±20 pp sourcePopulation 81 stderr 20 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label meta/muse-spark-open Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/antitrust_competition/meta~1muse-spark-open` | | | | | | | | | [EMB / overall](https://benchize.com/benchmarks/vals-ai%3Aemb%3Aoverall.md) | 53.67% | SE ±3.2 pp | 36 / 47 | $5.35 | 5,078.61source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 53.667 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 5.348821 \| USD \| per_test \| Lower is better \| \| `latency` \| 5078.614 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 131072 provider Together AI reasoning Not reported Reasoning effort xhigh Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±3.2 pp sourcePopulation 76 stderr 3.196 tokenTotals {"cache_read_tokens":642573184,"cache_write_tokens":null,"input_tokens":973042320,"output_tokens":8495845,"reasoning_tokens":4888586} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) Source Vals AI Original model label meta/muse-spark-open Source version 1 Observed 2026-10-21T16:45:57.495Z Evidence SHA-256 `d807aee620ea47152a1fca0257a8dd12a1be4512c76cc1744c1ae9e285180918` Source locator `/tasks/overall/meta~1muse-spark-open` | | | | | | | | | [Terminal-Bench 4.0 / science](https://benchize.com/benchmarks/vals-ai%3Aterminal-bench-4%3Ascience.md) | 7.14% | SE ±0 pp | 31 / 36 | — | — | [Vals AI · 4.0](https://www.vals.ai/_astro/benchmark_view_terminal_bench_4.CAnGyzC_.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 7.143 \| percent \| reported \| Higher is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 131072 provider Together AI reasoning Not reported Reasoning effort xhigh Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 52 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_terminal_bench_4.CAnGyzC_.json) Source Vals AI Original model label meta/muse-spark-open Source version 4.0 Observed 2026-10-21T16:46:10.642Z Evidence SHA-256 `98f6964580e901f1aa143db664bd92aa558415f643569492545d67c47d7f519b` Source locator `/tasks/science/meta~1muse-spark-open` | | | | | | | | | [Finance Agent v2.1 / financial modeling](https://benchize.com/benchmarks/vals-ai%3Afabv2-1%3Afinancial_modeling.md) | 23.42% | SE ±5.75 pp | 17 / 18 | $6.23 | 3,287.51source duration | [Vals AI · 2.1](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2_1.ZbMhy9xo.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 23.42 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 6.232657 \| USD \| per_test \| Lower is better \| \| `latency` \| 3287.51 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 131072 provider Together AI reasoning Not reported Reasoning effort xhigh Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±5.75 pp sourcePopulation 28 stderr 5.749 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2_1.ZbMhy9xo.json) Source Vals AI Original model label meta/muse-spark-open Source version 2.1 Observed 2026-10-21T16:45:54.954Z Evidence SHA-256 `f1b0c2d7dca4e73b52136862a9be059f7c8481c69f15d705ce90bf3bd9f78429` Source locator `/tasks/financial_modeling/meta~1muse-spark-open` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / criteria pass rate trusts estates private client](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__trusts_estates_private_client.md) | 75.57% | SE ±2.66 pp | 34 / 49 | $5.2 | 9,800.96source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 75.573 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 5.198573 \| USD \| per_test \| Lower is better \| \| `latency` \| 9800.956 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 131072 provider Together AI reasoning Not reported Reasoning effort xhigh Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.66 pp sourcePopulation 81 stderr 2.66 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label meta/muse-spark-open Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__trusts_estates_private_client/meta~1muse-spark-open` | | | | | | | | | [EMB / dcf](https://benchize.com/benchmarks/vals-ai%3Aemb%3Adcf.md) | 47.54% | SE ±8.25 pp | 32 / 47 | $4.65 | 5,120.53source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 47.537 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 4.653132 \| USD \| per_test \| Lower is better \| \| `latency` \| 5120.532 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 131072 provider Together AI reasoning Not reported Reasoning effort xhigh Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±8.25 pp sourcePopulation 76 stderr 8.245 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) Source Vals AI Original model label meta/muse-spark-open Source version 1 Observed 2026-10-21T16:45:57.495Z Evidence SHA-256 `d807aee620ea47152a1fca0257a8dd12a1be4512c76cc1744c1ae9e285180918` Source locator `/tasks/dcf/meta~1muse-spark-open` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / insurance](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Ainsurance.md) | 0% | SE ±0 pp | 2 / 49 | $6.89 | 7,735.15source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 6.893238 \| USD \| per_test \| Lower is better \| \| `latency` \| 7735.152 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 131072 provider Together AI reasoning Not reported Reasoning effort xhigh Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 81 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label meta/muse-spark-open Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/insurance/meta~1muse-spark-open` | | | | | | | | | [Legal Research Bench / administrative regulatory](https://benchize.com/benchmarks/vals-ai%3Alegal_research%3Aadministrative_regulatory.md) | 50% | SE ±6.2 pp | 18 / 50 | $1.42 | 1,671.25source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 50 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 1.423111 \| USD \| per_test \| Lower is better \| \| `latency` \| 1671.25 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 131072 provider Together AI reasoning Not reported Reasoning effort xhigh Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±6.2 pp sourcePopulation 82 stderr 6.202 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) Source Vals AI Original model label meta/muse-spark-open Source version 1 Observed 2026-10-21T16:45:52.389Z Evidence SHA-256 `26c58dfd47d0d2dc86f33c6d056f30309e0ee63570e7cd752c593b385f9d862e` Source locator `/tasks/administrative_regulatory/meta~1muse-spark-open` | | | | | | | | | [Terminal-Bench 4.0 / overall](https://benchize.com/benchmarks/vals-ai%3Aterminal-bench-4%3Aoverall.md) | 6.57% | SE ±0.51 pp | 30 / 36 | $7.26 | 7,466.4source duration | [Vals AI · 4.0](https://www.vals.ai/_astro/benchmark_view_terminal_bench_4.CAnGyzC_.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 6.566 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 7.264062 \| USD \| per_test \| Lower is better \| \| `latency` \| 7466.403 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 131072 provider Together AI reasoning Not reported Reasoning effort xhigh Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.51 pp sourcePopulation 52 stderr 0.505 tokenTotals {"cache_read_tokens":633327467,"cache_write_tokens":0,"input_tokens":916170962,"output_tokens":7264611,"reasoning_tokens":4188707} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_terminal_bench_4.CAnGyzC_.json) Source Vals AI Original model label meta/muse-spark-open Source version 4.0 Observed 2026-10-21T16:46:10.642Z Evidence SHA-256 `98f6964580e901f1aa143db664bd92aa558415f643569492545d67c47d7f519b` Source locator `/tasks/overall/meta~1muse-spark-open` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / criteria pass rate white collar defense investigations](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__white_collar_defense_investigations.md) | 56.23% | SE ±2.88 pp | 42 / 49 | $3.52 | 7,655.6source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 56.229 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 3.523605 \| USD \| per_test \| Lower is better \| \| `latency` \| 7655.598 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 131072 provider Together AI reasoning Not reported Reasoning effort xhigh Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.88 pp sourcePopulation 81 stderr 2.884 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label meta/muse-spark-open Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__white_collar_defense_investigations/meta~1muse-spark-open` | | | | | | | | | [Tax Agent Bench / current temporal analysis](https://benchize.com/benchmarks/vals-ai%3Atax_agent_bench%3Acurrent_temporal_analysis.md) | 28% | SE ±6.37 pp | 32 / 46 | $0.29 | 760.5source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 28 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.288385 \| USD \| per_test \| Lower is better \| \| `latency` \| 760.495 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 131072 provider Together AI reasoning Not reported Reasoning effort xhigh Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±6.37 pp sourcePopulation 73 stderr 6.368 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) Source Vals AI Original model label meta/muse-spark-open Source version 1 Observed 2026-10-21T16:45:55.958Z Evidence SHA-256 `446db182b67efb7923b3187a4040ec3a3ea492b8ad699a2b6d708d714cc063ac` Source locator `/tasks/current_temporal_analysis/meta~1muse-spark-open` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / data privacy cybersecurity](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Adata_privacy_cybersecurity.md) | 40% | SE ±24.5 pp | 1 / 49 | $12.05 | 14,677.24source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 40 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 12.052447 \| USD \| per_test \| Lower is better \| \| `latency` \| 14677.238 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 131072 provider Together AI reasoning Not reported Reasoning effort xhigh Temperature 1 topP 1 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±24.5 pp sourcePopulation 81 stderr 24.495 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label meta/muse-spark-open Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/data_privacy_cybersecurity/meta~1muse-spark-open` | | | | | | | | [Next page](https://benchize.com/models/muse-spark-open-xhigh.md?cursor=902af373711ee194e650e19d7f45e1abc26b9c0a7d00eb11abaa1cd11493ac42) --- Cite as: Muse Spark OpenExtra high — Benchize, https://benchize.com/models/muse-spark-open-xhigh (retrieved 2026-10-21)