# Claude Opus 4.8Max > Source: Benchize (https://benchize.com) — https://benchize.com/models/claude-opus-4-8-max. When citing or reusing this content, credit "Benchize" and link to https://benchize.com/models/claude-opus-4-8-max. Anthropic Canonical URL: https://benchize.com/models/claude-opus-4-8-max Last updated: Wed, 21 Oct 2026 18:52:20 GMT Reproduction and quotation must name Benchize, link to the canonical HTML URL, and retain individual source citations. Anthropic ## Benchmarks | Benchmark | Score | ± | Rank | Cost | Latency | Source | | | --- | --- | --- | --- | --- | --- | --- | --- | | [Terminal-Bench 2.0 / easy](https://benchize.com/benchmarks/vals-ai%3Aterminal-bench-2%3Aeasy.md) | 91.67% | SE ±8.33 pp | 7 / 23 | $11.8 | 10,511.91source duration | [Vals AI · 2.0](https://www.vals.ai/_astro/benchmark_view_terminal_bench_2.T48tPR4O.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 91.667 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 11.799576 \| USD \| per_test \| Lower is better \| \| `latency` \| 10511.909 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±8.33 pp sourcePopulation 67 stderr 8.333 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_terminal_bench_2.T48tPR4O.json) Source Vals AI Original model label anthropic/claude-opus-4-8 Source version 2.0 Observed 2026-10-21T16:46:14.732Z Evidence SHA-256 `38e640d995259778e531f048b2f2c7b3b281cb05b56be643ddc9dc53eb85adf7` Source locator `/tasks/easy/anthropic~1claude-opus-4-8` | | | | | | | | | [Public Benefits Bench v1 / overall](https://benchize.com/benchmarks/vals-ai%3Apublic-benefits-bench-v1%3Aoverall.md) | 62.11% | SE ±1.26 pp | 2 / 8 | $1.32 | 597.69source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_snap_v1.BPa0Hrre.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 62.111 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 1.316771 \| USD \| per_test \| Lower is better \| \| `latency` \| 597.691 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.26 pp sourcePopulation 13 stderr 1.262 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_snap_v1.BPa0Hrre.json) Source Vals AI Original model label anthropic/claude-opus-4-8 Source version 1 Observed 2026-10-21T16:46:18.363Z Evidence SHA-256 `b85e2b3cbdf03e13e2ffdb27ce4f108003a548d705a24ae26a0a115e2027475d` Source locator `/tasks/overall/anthropic~1claude-opus-4-8` | | | | | | | | | [FrontierSWE](https://benchize.com/benchmarks/frontierswe%3Adominance.md) | 66.7% | — | 5 / 6 | — | — | [Moonshot AI · sha256:0e806bbad76dc8e8dbc899a86051e3406bdd5995fe6c6aa1b2176dde4fb28e6c](https://www.kimi.ai/blog/kimi-k3) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `dominance` \| 66.7 \| percent \| dominance_2026-07-16 \| Higher is better \| ### Configuration normalization dominance normalizationDate 2026-07-16 Reasoning effort max thinkingEnabled true ### Source Score publisher [Moonshot AI](https://www.kimi.ai/blog/kimi-k3) Collected via [moonshot k3](https://www.kimi.ai/blog/kimi-k3) Publisher reference Referenced by the collection source; the upstream result has not been independently captured. Benchmark [https://www.kimi.ai/blog/kimi-k3](https://www.kimi.ai/blog/kimi-k3) [Collected via](https://www.kimi.ai/blog/kimi-k3) Source Moonshot AI Original model label Claude Opus 4.8 (max) Source version sha256:0e806bbad76dc8e8dbc899a86051e3406bdd5995fe6c6aa1b2176dde4fb28e6c Observed 2026-10-21T16:31:44.130Z Evidence SHA-256 `0e806bbad76dc8e8dbc899a86051e3406bdd5995fe6c6aa1b2176dde4fb28e6c` Source locator `/flight/sheet/data/3/scores/3` | | | | | | | | | [BrokenArXiv 05/2026](https://benchize.com/benchmarks/matharena%3Aarxiv_false--may.md) | 23% | 95% CI 14.75–31.25% | 11 / 20 | 5.78usd per problem | — | [matharena · arxiv_false--may](https://matharena.ai/competition_tables/arxiv_false--may) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 23.00 \| percent \| competition \| Higher is better \| \| `cost_per_problem` \| 5.78 \| usd_per_problem \| competition \| Lower is better \| ### Configuration competition arxiv_false--may predicted false Reasoning effort max sourceModelName Claude-Opus-4.8 (max) sourceModelVersion anthropic/opus_48_max thinkingEnabled true ### Reported statistics Uncertainty 95% confidence interval: 14.75–31.25% confidenceInterval95 8.25 problems 50 ### Source [Original source](https://matharena.ai/competition_tables/arxiv_false--may) Source matharena Original model label Claude-Opus-4.8 (max) Source version arxiv_false--may Observed 2026-10-21T16:52:24.785Z Evidence SHA-256 `b317fca30109e8c78ed5d2b835fe227b8cfbee3b6f9cb6d7aaeadca866660969` Source locator `/table/rows/13` | | | | | | | | | [Tax Agent Bench / overall](https://benchize.com/benchmarks/vals-ai%3Atax_agent_bench%3Aoverall.md) | 30% | SE ±3.3 pp | 17 / 46 | $1.69 | 479.2source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 29.999 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 1.68964 \| USD \| per_test \| Lower is better \| \| `latency` \| 479.2 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±3.3 pp sourcePopulation 73 stderr 3.303 tokenTotals {"cache_read_tokens":39455480,"cache_write_tokens":19099407,"input_tokens":58571818,"output_tokens":7476677,"reasoning_tokens":5120560} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) Source Vals AI Original model label anthropic/claude-opus-4-8 Source version 1 Observed 2026-10-21T16:45:55.958Z Evidence SHA-256 `446db182b67efb7923b3187a4040ec3a3ea492b8ad699a2b6d708d714cc063ac` Source locator `/tasks/overall/anthropic~1claude-opus-4-8` | | | | | | | | | [AA-Briefcase (Elo-score)](https://benchize.com/benchmarks/aa-briefcase.md) | 1,354elo | — | 4 / 6 | — | — | [Moonshot AI · sha256:0e806bbad76dc8e8dbc899a86051e3406bdd5995fe6c6aa1b2176dde4fb28e6c](https://www.kimi.ai/blog/kimi-k3) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `elo` \| 1354 \| elo \| reported \| Higher is better \| ### Configuration Reasoning effort max thinkingEnabled true ### Source Score publisher [Moonshot AI](https://www.kimi.ai/blog/kimi-k3) Collected via [moonshot k3](https://www.kimi.ai/blog/kimi-k3) Publisher reference Referenced by the collection source; the upstream result has not been independently captured. Benchmark [https://www.kimi.ai/blog/kimi-k3](https://www.kimi.ai/blog/kimi-k3) [Collected via](https://www.kimi.ai/blog/kimi-k3) Source Moonshot AI Original model label Claude Opus 4.8 (max) Source version sha256:0e806bbad76dc8e8dbc899a86051e3406bdd5995fe6c6aa1b2176dde4fb28e6c Observed 2026-10-21T16:31:44.130Z Evidence SHA-256 `0e806bbad76dc8e8dbc899a86051e3406bdd5995fe6c6aa1b2176dde4fb28e6c` Source locator `/flight/sheet/data/15/scores/3` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / intellectual property](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Aintellectual_property.md) | 0% | SE ±0 pp | 6 / 49 | $9.53 | 1,413.28source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 9.530409 \| USD \| per_test \| Lower is better \| \| `latency` \| 1413.284 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 81 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label anthropic/claude-opus-4-8 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/intellectual_property/anthropic~1claude-opus-4-8` | | | | | | | | | [AutomationBench](https://benchize.com/benchmarks/zapier%3Aautomationbench.md) | 27.2% | — | 4 / 6 | — | — | [Moonshot AI · sha256:0e806bbad76dc8e8dbc899a86051e3406bdd5995fe6c6aa1b2176dde4fb28e6c](https://www.kimi.ai/blog/kimi-k3) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 27.2 \| percent \| reported \| Higher is better \| ### Configuration Reasoning effort max subset public taskCount 600 thinkingEnabled true ### Source Score publisher [Moonshot AI](https://www.kimi.ai/blog/kimi-k3) Collected via [moonshot k3](https://www.kimi.ai/blog/kimi-k3) Publisher reference Referenced by the collection source; the upstream result has not been independently captured. Benchmark [https://www.kimi.ai/blog/kimi-k3](https://www.kimi.ai/blog/kimi-k3) [Collected via](https://www.kimi.ai/blog/kimi-k3) Source Moonshot AI Original model label Claude Opus 4.8 (max) Source version sha256:0e806bbad76dc8e8dbc899a86051e3406bdd5995fe6c6aa1b2176dde4fb28e6c Observed 2026-10-21T16:31:44.130Z Evidence SHA-256 `0e806bbad76dc8e8dbc899a86051e3406bdd5995fe6c6aa1b2176dde4fb28e6c` Source locator `/flight/sheet/data/13/scores/3` | | | | | | | | | [Job Bench](https://benchize.com/benchmarks/jobbench.md) | 48.4% | — | 3 / 6 | — | — | [Moonshot AI · sha256:0e806bbad76dc8e8dbc899a86051e3406bdd5995fe6c6aa1b2176dde4fb28e6c](https://www.kimi.ai/blog/kimi-k3) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 48.4 \| percent \| reported \| Higher is better \| ### Configuration Reasoning effort max thinkingEnabled true ### Source Score publisher [Moonshot AI](https://www.kimi.ai/blog/kimi-k3) Collected via [moonshot k3](https://www.kimi.ai/blog/kimi-k3) Publisher reference Referenced by the collection source; the upstream result has not been independently captured. Benchmark [https://www.kimi.ai/blog/kimi-k3](https://www.kimi.ai/blog/kimi-k3) [Collected via](https://www.kimi.ai/blog/kimi-k3) Source Moonshot AI Original model label Claude Opus 4.8 (max) Source version sha256:0e806bbad76dc8e8dbc899a86051e3406bdd5995fe6c6aa1b2176dde4fb28e6c Observed 2026-10-21T16:31:44.130Z Evidence SHA-256 `0e806bbad76dc8e8dbc899a86051e3406bdd5995fe6c6aa1b2176dde4fb28e6c` Source locator `/flight/sheet/data/14/scores/3` | | | | | | | | | [Humanity's Sixth Sense](https://benchize.com/benchmarks/scale%3Ahss.md) | 23.5source score | CI 20.3–26.7 | 21 / 24 | — | — | [scale](https://labs.scale.com/leaderboard/hss) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 23.5 \| source_score \| reported \| Higher is better \| ### Configuration Reasoning effort max Model configuration Claude-Opus-4.8 (max) sourceModelVersion Not reported thinkingEnabled true ### Reported statistics Uncertainty Confidence interval: 20.3–26.7 contaminationWarning Not reported rank 24 reportedConfidenceHalfWidth 3.2 ### Source [Original source](https://labs.scale.com/leaderboard/hss) Source scale Original model label Claude-Opus-4.8 (max) Observed 2026-10-21T16:35:26.953Z Evidence SHA-256 `0b7d37655e8886c99f8325ed125eafaa4f89312cd44e1a783ac0c49b45db842b` Source locator `/rows/23` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / tax](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Atax.md) | 0% | SE ±0 pp | 3 / 49 | $9.86 | 1,692.9source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 9.85827 \| USD \| per_test \| Lower is better \| \| `latency` \| 1692.904 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 81 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label anthropic/claude-opus-4-8 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/tax/anthropic~1claude-opus-4-8` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / insurance](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Ainsurance.md) | 0% | SE ±0 pp | 2 / 49 | $9.12 | 1,460.75source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 9.118767 \| USD \| per_test \| Lower is better \| \| `latency` \| 1460.754 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 81 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label anthropic/claude-opus-4-8 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/insurance/anthropic~1claude-opus-4-8` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / corporate governance](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acorporate_governance.md) | 0% | SE ±0 pp | 11 / 49 | $11.14 | 1,412.71source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 11.140007 \| USD \| per_test \| Lower is better \| \| `latency` \| 1412.708 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 81 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label anthropic/claude-opus-4-8 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/corporate_governance/anthropic~1claude-opus-4-8` | | | | | | | | | [Terminal-Bench-Science / overall](https://benchize.com/benchmarks/vals-ai%3Aterminal-bench-science%3Aoverall.md) | 4.29% | SE ±2.44 pp | 18 / 30 | $23.14 | 6,906.54source duration | [Vals AI · 0.1](https://www.vals.ai/_astro/benchmark_view_terminal_bench_science.o-HR8Nyt.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 4.286 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 23.141733 \| USD \| per_test \| Lower is better \| \| `latency` \| 6906.538 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Mini-SWE-agent maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.44 pp sourcePopulation 41 stderr 2.438 tokenTotals {"cache_read_tokens":1772026517,"cache_write_tokens":37411842,"input_tokens":3618892862,"output_tokens":34489714,"reasoning_tokens":14489581} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_terminal_bench_science.o-HR8Nyt.json) Source Vals AI Original model label anthropic/claude-opus-4-8 Source version 0.1 Observed 2026-10-21T16:46:03.529Z Evidence SHA-256 `a30a8618deaecf018ea1c8c9f675576cdd9ab79742d05788067db49afcc5f91b` Source locator `/tasks/overall/anthropic~1claude-opus-4-8` | | | | | | | | | [GPQA Diamond / diamond zero shot cot](https://benchize.com/benchmarks/vals-ai%3Agpqa%3Adiamond_zero_shot_cot.md) | 92.93% | SE ±3.25 pp | 9 / 40 | $0.34 | 136.37source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_gpqa.ghZZBpyU.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 92.929 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.340421 \| USD \| per_test \| Lower is better \| \| `latency` \| 136.373 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±3.25 pp sourcePopulation 138 stderr 3.25 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_gpqa.ghZZBpyU.json) Source Vals AI Original model label anthropic/claude-opus-4-8 Source version 1 Observed 2026-10-21T16:46:06.040Z Evidence SHA-256 `951eb5224097733a9d09d06c14908ae2980e2e640e539f2c284877c5092d7077` Source locator `/tasks/diamond_zero_shot_cot/anthropic~1claude-opus-4-8` | | | | | | | | | [MMMU-Pro w/ python](https://benchize.com/benchmarks/mmmu-pro-python.md) | 82.7% | — | 5 / 5 | — | — | [Moonshot AI · sha256:0e806bbad76dc8e8dbc899a86051e3406bdd5995fe6c6aa1b2176dde4fb28e6c](https://www.kimi.ai/blog/kimi-k3) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 82.7 \| percent \| reported \| Higher is better \| ### Configuration imageOrder original_images_prepended Reasoning effort max runs 3 thinkingEnabled true tools ["python"] toolsEnabled true ### Reported statistics runCount 3 ### Source Score publisher [Moonshot AI](https://www.kimi.ai/blog/kimi-k3) Collected via [moonshot k3](https://www.kimi.ai/blog/kimi-k3) Publisher reference Referenced by the collection source; the upstream result has not been independently captured. Benchmark [https://www.kimi.ai/blog/kimi-k3](https://www.kimi.ai/blog/kimi-k3) [Collected via](https://www.kimi.ai/blog/kimi-k3) Source Moonshot AI Original model label Claude Opus 4.8 (max) Source version sha256:0e806bbad76dc8e8dbc899a86051e3406bdd5995fe6c6aa1b2176dde4fb28e6c Observed 2026-10-21T16:31:44.130Z Evidence SHA-256 `0e806bbad76dc8e8dbc899a86051e3406bdd5995fe6c6aa1b2176dde4fb28e6c` Source locator `/flight/sheet/data/24/scores/3` | | | | | | | | | [Vals Multimodal Index / overall](https://benchize.com/benchmarks/vals-ai%3Avals_multimodal_index%3Aoverall.md) | 70.89% | SE ±0.74 pp | 4 / 20 | $5.46 | 979.85source duration | [Vals AI · 1.2](https://www.vals.ai/_astro/benchmark_view_vals_multimodal_index.virVpWMs.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 70.889 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 5.455981 \| USD \| per_test \| Lower is better \| \| `latency` \| 979.847 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.74 pp sourcePopulation 33 stderr 0.744 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_vals_multimodal_index.virVpWMs.json) Source Vals AI Original model label anthropic/claude-opus-4-8 Source version 1.2 Observed 2026-10-21T16:45:51.881Z Evidence SHA-256 `077b22575f6e075b7b78157d000ec8f94d7ead409c7dcbcf04887630ed572a2d` Source locator `/tasks/overall/anthropic~1claude-opus-4-8` | | | | | | | | | [CharXiv (RQ) w/ python](https://benchize.com/benchmarks/charxiv-rq-python.md) | 89.9% | — | 3 / 5 | — | — | [Moonshot AI · sha256:0e806bbad76dc8e8dbc899a86051e3406bdd5995fe6c6aa1b2176dde4fb28e6c](https://www.kimi.ai/blog/kimi-k3) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 89.9 \| percent \| reported \| Higher is better \| ### Configuration Reasoning effort max runs 3 subset RQ thinkingEnabled true tools ["python"] toolsEnabled true ### Reported statistics runCount 3 ### Source Score publisher [Moonshot AI](https://www.kimi.ai/blog/kimi-k3) Collected via [moonshot k3](https://www.kimi.ai/blog/kimi-k3) Publisher reference Referenced by the collection source; the upstream result has not been independently captured. Benchmark [https://www.kimi.ai/blog/kimi-k3](https://www.kimi.ai/blog/kimi-k3) [Collected via](https://www.kimi.ai/blog/kimi-k3) Source Moonshot AI Original model label Claude Opus 4.8 (max) Source version sha256:0e806bbad76dc8e8dbc899a86051e3406bdd5995fe6c6aa1b2176dde4fb28e6c Observed 2026-10-21T16:31:44.130Z Evidence SHA-256 `0e806bbad76dc8e8dbc899a86051e3406bdd5995fe6c6aa1b2176dde4fb28e6c` Source locator `/flight/sheet/data/26/scores/3` | | | | | | | | | [BrokenArXiv 06/2026](https://benchize.com/benchmarks/matharena%3Aarxiv_false--june.md) | 41.67% | 95% CI 34.08–49.26% | 12 / 21 | 5.55usd per problem | — | [matharena · arxiv_false--june](https://matharena.ai/competition_tables/arxiv_false--june) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 41.67 \| percent \| competition \| Higher is better \| \| `cost_per_problem` \| 5.55 \| usd_per_problem \| competition \| Lower is better \| ### Configuration competition arxiv_false--june predicted false Reasoning effort max sourceModelName Claude-Opus-4.8 (max) sourceModelVersion anthropic/opus_48_max thinkingEnabled true ### Reported statistics Uncertainty 95% confidence interval: 34.08–49.26% confidenceInterval95 7.59 problems 54 ### Source [Original source](https://matharena.ai/competition_tables/arxiv_false--june) Source matharena Original model label Claude-Opus-4.8 (max) Source version arxiv_false--june Observed 2026-10-21T16:52:25.323Z Evidence SHA-256 `f6d7e2280d05706643d46ae7786bd3fad6d912e4893781c3f62c73deebf38459` Source locator `/table/rows/16` | | | | | | | | | [HLE-Full](https://benchize.com/benchmarks/hle-full.md) | 49.8% | — | 2 / 5 | — | — | [Moonshot AI · sha256:0e806bbad76dc8e8dbc899a86051e3406bdd5995fe6c6aa1b2176dde4fb28e6c](https://www.kimi.ai/blog/kimi-k3) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 49.8 \| percent \| reported \| Higher is better \| ### Configuration Reasoning effort max sourceScoreFootnote * thinkingEnabled true toolsEnabled false ### Source Score publisher [Moonshot AI](https://www.kimi.ai/blog/kimi-k3) Collected via [moonshot k3](https://www.kimi.ai/blog/kimi-k3) Publisher reference Referenced by the collection source; the upstream result has not been independently captured. Benchmark [https://www.kimi.ai/blog/kimi-k3](https://www.kimi.ai/blog/kimi-k3) [Collected via](https://www.kimi.ai/blog/kimi-k3) Source Moonshot AI Original model label Claude Opus 4.8 (max) Source version sha256:0e806bbad76dc8e8dbc899a86051e3406bdd5995fe6c6aa1b2176dde4fb28e6c Observed 2026-10-21T16:31:44.130Z Evidence SHA-256 `0e806bbad76dc8e8dbc899a86051e3406bdd5995fe6c6aa1b2176dde4fb28e6c` Source locator `/flight/sheet/data/21/scores/3` | | | | | | | | | [EMB / ma](https://benchize.com/benchmarks/vals-ai%3Aemb%3Ama.md) | 80.09% | SE ±3.26 pp | 13 / 47 | $9.45 | 2,301.31source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 80.091 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 9.449638 \| USD \| per_test \| Lower is better \| \| `latency` \| 2301.312 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±3.26 pp sourcePopulation 76 stderr 3.258 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) Source Vals AI Original model label anthropic/claude-opus-4-8 Source version 1 Observed 2026-10-21T16:45:57.495Z Evidence SHA-256 `d807aee620ea47152a1fca0257a8dd12a1be4512c76cc1744c1ae9e285180918` Source locator `/tasks/ma/anthropic~1claude-opus-4-8` | | | | | | | | | [MMLU Pro / philosophy](https://benchize.com/benchmarks/vals-ai%3Ammlu_pro%3Aphilosophy.md) | 86.57% | SE ±1.53 pp | 14 / 40 | $0.03 | 12.12source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_mmlu_pro.y416pXDi.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 86.573 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.031561 \| USD \| per_test \| Lower is better \| \| `latency` \| 12.119 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.53 pp sourcePopulation 138 stderr 1.526 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_mmlu_pro.y416pXDi.json) Source Vals AI Original model label anthropic/claude-opus-4-8 Source version 1 Observed 2026-10-21T16:46:06.503Z Evidence SHA-256 `dab025507f46c716dd8d145e90ec0a737d9c56219853f41e4b1cc3c5e8c222b4` Source locator `/tasks/philosophy/anthropic~1claude-opus-4-8` | | | | | | | | | [MMLU Pro / health](https://benchize.com/benchmarks/vals-ai%3Ammlu_pro%3Ahealth.md) | 85.09% | SE ±1.25 pp | 4 / 40 | $0.04 | 17.11source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_mmlu_pro.y416pXDi.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 85.086 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.040556 \| USD \| per_test \| Lower is better \| \| `latency` \| 17.113 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.25 pp sourcePopulation 138 stderr 1.246 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_mmlu_pro.y416pXDi.json) Source Vals AI Original model label anthropic/claude-opus-4-8 Source version 1 Observed 2026-10-21T16:46:06.503Z Evidence SHA-256 `dab025507f46c716dd8d145e90ec0a737d9c56219853f41e4b1cc3c5e8c222b4` Source locator `/tasks/health/anthropic~1claude-opus-4-8` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / international trade sanctions](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Ainternational_trade_sanctions.md) | 0% | SE ±0 pp | 4 / 49 | $7.69 | 1,470.57source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 7.692416 \| USD \| per_test \| Lower is better \| \| `latency` \| 1470.566 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 81 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label anthropic/claude-opus-4-8 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/international_trade_sanctions/anthropic~1claude-opus-4-8` | | | | | | | | | [Tax Agent Bench / current temporal analysis](https://benchize.com/benchmarks/vals-ai%3Atax_agent_bench%3Acurrent_temporal_analysis.md) | 46.52% | SE ±9.24 pp | 12 / 46 | $1.95 | 584.04source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 46.524 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 1.945154 \| USD \| per_test \| Lower is better \| \| `latency` \| 584.037 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±9.24 pp sourcePopulation 73 stderr 9.237 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) Source Vals AI Original model label anthropic/claude-opus-4-8 Source version 1 Observed 2026-10-21T16:45:55.958Z Evidence SHA-256 `446db182b67efb7923b3187a4040ec3a3ea492b8ad699a2b6d708d714cc063ac` Source locator `/tasks/current_temporal_analysis/anthropic~1claude-opus-4-8` | | | | | | | | | [Kimi Code Bench 2.0 (Internal)](https://benchize.com/benchmarks/kimi-code-bench-2.md) | 71.7% | — | 3 / 6 | — | — | [Moonshot AI · sha256:0e806bbad76dc8e8dbc899a86051e3406bdd5995fe6c6aa1b2176dde4fb28e6c](https://www.kimi.ai/blog/kimi-k3) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 71.7 \| percent \| reported \| Higher is better \| ### Configuration benchmarkVisibility internal harness Claude Code Reasoning effort max thinkingEnabled true ### Source Score publisher [Moonshot AI](https://www.kimi.ai/blog/kimi-k3) Collected via [moonshot k3](https://www.kimi.ai/blog/kimi-k3) Publisher reference Referenced by the collection source; the upstream result has not been independently captured. Benchmark [https://www.kimi.ai/blog/kimi-k3](https://www.kimi.ai/blog/kimi-k3) [Collected via](https://www.kimi.ai/blog/kimi-k3) Source Moonshot AI Original model label Claude Opus 4.8 (max) Source version sha256:0e806bbad76dc8e8dbc899a86051e3406bdd5995fe6c6aa1b2176dde4fb28e6c Observed 2026-10-21T16:31:44.130Z Evidence SHA-256 `0e806bbad76dc8e8dbc899a86051e3406bdd5995fe6c6aa1b2176dde4fb28e6c` Source locator `/flight/sheet/data/7/scores/3` | | | | | | | | | [SWE Marathon](https://benchize.com/benchmarks/swe-marathon.md) | 40% | — | 2 / 6 | — | — | [Moonshot AI · sha256:0e806bbad76dc8e8dbc899a86051e3406bdd5995fe6c6aa1b2176dde4fb28e6c](https://www.kimi.ai/blog/kimi-k3) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 40.0 \| percent \| reported \| Higher is better \| ### Configuration datasetAdaptation H20-calibrated harness Claude Code Reasoning effort max thinkingEnabled true ### Source Score publisher [Moonshot AI](https://www.kimi.ai/blog/kimi-k3) Collected via [moonshot k3](https://www.kimi.ai/blog/kimi-k3) Publisher reference Referenced by the collection source; the upstream result has not been independently captured. Benchmark [https://www.kimi.ai/blog/kimi-k3](https://www.kimi.ai/blog/kimi-k3) [Collected via](https://www.kimi.ai/blog/kimi-k3) Source Moonshot AI Original model label Claude Opus 4.8 (max) Source version sha256:0e806bbad76dc8e8dbc899a86051e3406bdd5995fe6c6aa1b2176dde4fb28e6c Observed 2026-10-21T16:31:44.130Z Evidence SHA-256 `0e806bbad76dc8e8dbc899a86051e3406bdd5995fe6c6aa1b2176dde4fb28e6c` Source locator `/flight/sheet/data/4/scores/3` | | | | | | | | | [MMMU Pro / overall](https://benchize.com/benchmarks/vals-ai%3Ammmu%3Aoverall.md) | 86.59% | SE ±0.82 pp | 12 / 28 | $0.12 | 49.04source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_mmmu.BBln8x5x.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 86.59 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.120663 \| USD \| per_test \| Lower is better \| \| `latency` \| 49.044 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.82 pp sourcePopulation 93 stderr 0.819 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_mmmu.BBln8x5x.json) Source Vals AI Original model label anthropic/claude-opus-4-8 Source version 1 Observed 2026-10-21T16:46:07.000Z Evidence SHA-256 `29395186eddd2c70601228c584a14b3c1d8f9dba4b68256561b46b0dd03c635a` Source locator `/tasks/overall/anthropic~1claude-opus-4-8` | | | | | | | | | [Terminal-Bench 4.0 / operations](https://benchize.com/benchmarks/vals-ai%3Aterminal-bench-4%3Aoperations.md) | 11.11% | SE ±11.11 pp | 20 / 36 | — | — | [Vals AI · 4.0](https://www.vals.ai/_astro/benchmark_view_terminal_bench_4.CAnGyzC_.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 11.111 \| percent \| reported \| Higher is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±11.11 pp sourcePopulation 52 stderr 11.111 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_terminal_bench_4.CAnGyzC_.json) Source Vals AI Original model label anthropic/claude-opus-4-8 Source version 4.0 Observed 2026-10-21T16:46:10.642Z Evidence SHA-256 `98f6964580e901f1aa143db664bd92aa558415f643569492545d67c47d7f519b` Source locator `/tasks/operations/anthropic~1claude-opus-4-8` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / criteria pass rate funds asset management](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__funds_asset_management.md) | 84.55% | SE ±1.95 pp | 13 / 49 | $12.06 | 1,687.57source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 84.548 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 12.061978 \| USD \| per_test \| Lower is better \| \| `latency` \| 1687.568 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.95 pp sourcePopulation 81 stderr 1.954 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label anthropic/claude-opus-4-8 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__funds_asset_management/anthropic~1claude-opus-4-8` | | | | | | | | | [ProgramBench / almost](https://benchize.com/benchmarks/vals-ai%3Aprogrambench%3Aalmost.md) | 15.5% | SE ±2.57 pp | 14 / 40 | $31.27 | 5,523.1source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_programbench.C_J7ZU4X.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 15.5 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 31.267424 \| USD \| per_test \| Lower is better \| \| `latency` \| 5523.099 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.57 pp sourcePopulation 64 stderr 2.565 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_programbench.C_J7ZU4X.json) Source Vals AI Original model label anthropic/claude-opus-4-8 Source version 1 Observed 2026-10-21T16:46:10.137Z Evidence SHA-256 `e9bbb4378c7a058441a21d456baf9316b96c2cf5b58d371a3c7c803f5000bd8c` Source locator `/tasks/almost/anthropic~1claude-opus-4-8` | | | | | | | | | [Time Horizon Index: KSP / overall](https://benchize.com/benchmarks/vals-ai%3Atime_horizon_index%3Aoverall.md) | 11.83% | SE ±0 pp | 8 / 10 | $1,499.32 | 0source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_ksp_bench.CDDTSam7.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 11.833 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 1499.319504 \| USD \| per_test \| Lower is better \| \| `latency` \| 0 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 13 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_ksp_bench.CDDTSam7.json) Source Vals AI Original model label anthropic/claude-opus-4-8 Source version 1 Observed 2026-10-21T16:46:16.730Z Evidence SHA-256 `be251b72ecceae7912b9577f9b3a3512740c308a34f81ac823952e919adae50f` Source locator `/tasks/overall/anthropic~1claude-opus-4-8` | | | | | | | | | [Legal Research Bench / weighted pass constitutional civil rights](https://benchize.com/benchmarks/vals-ai%3Alegal_research%3Aweighted_pass__constitutional_civil_rights.md) | 87.02% | SE ±2.63 pp | 11 / 50 | $2.46 | 609.12source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 87.016 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 2.462184 \| USD \| per_test \| Lower is better \| \| `latency` \| 609.116 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.63 pp sourcePopulation 82 stderr 2.634 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) Source Vals AI Original model label anthropic/claude-opus-4-8 Source version 1 Observed 2026-10-21T16:45:52.389Z Evidence SHA-256 `26c58dfd47d0d2dc86f33c6d056f30309e0ee63570e7cd752c593b385f9d862e` Source locator `/tasks/weighted_pass__constitutional_civil_rights/anthropic~1claude-opus-4-8` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / litigation dispute resolution](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Alitigation_dispute_resolution.md) | 0% | SE ±0 pp | 6 / 49 | $10.41 | 1,464.8source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 10.4087 \| USD \| per_test \| Lower is better \| \| `latency` \| 1464.802 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 81 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label anthropic/claude-opus-4-8 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/litigation_dispute_resolution/anthropic~1claude-opus-4-8` | | | | | | | | | [MMLU Pro / engineering](https://benchize.com/benchmarks/vals-ai%3Ammlu_pro%3Aengineering.md) | 87.72% | SE ±1.05 pp | 11 / 40 | $0.18 | 72.12source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_mmlu_pro.y416pXDi.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 87.719 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.177437 \| USD \| per_test \| Lower is better \| \| `latency` \| 72.117 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.05 pp sourcePopulation 138 stderr 1.054 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_mmlu_pro.y416pXDi.json) Source Vals AI Original model label anthropic/claude-opus-4-8 Source version 1 Observed 2026-10-21T16:46:06.503Z Evidence SHA-256 `dab025507f46c716dd8d145e90ec0a737d9c56219853f41e4b1cc3c5e8c222b4` Source locator `/tasks/engineering/anthropic~1claude-opus-4-8` | | | | | | | | | [Terminal-Bench-Science / earth sciences](https://benchize.com/benchmarks/vals-ai%3Aterminal-bench-science%3Aearth_sciences.md) | 0% | SE ±0 pp | 14 / 30 | — | — | [Vals AI · 0.1](https://www.vals.ai/_astro/benchmark_view_terminal_bench_science.o-HR8Nyt.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| ### Configuration computeEffort max harness Mini-SWE-agent maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 41 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_terminal_bench_science.o-HR8Nyt.json) Source Vals AI Original model label anthropic/claude-opus-4-8 Source version 0.1 Observed 2026-10-21T16:46:03.529Z Evidence SHA-256 `a30a8618deaecf018ea1c8c9f675576cdd9ab79742d05788067db49afcc5f91b` Source locator `/tasks/earth_sciences/anthropic~1claude-opus-4-8` | | | | | | | | | [GPQA Diamond / diamond few shot cot](https://benchize.com/benchmarks/vals-ai%3Agpqa%3Adiamond_few_shot_cot.md) | 91.92% | SE ±2.28 pp | 12 / 40 | $0.34 | 131.38source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_gpqa.ghZZBpyU.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 91.919 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.3381 \| USD \| per_test \| Lower is better \| \| `latency` \| 131.379 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.28 pp sourcePopulation 138 stderr 2.277 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_gpqa.ghZZBpyU.json) Source Vals AI Original model label anthropic/claude-opus-4-8 Source version 1 Observed 2026-10-21T16:46:06.040Z Evidence SHA-256 `951eb5224097733a9d09d06c14908ae2980e2e640e539f2c284877c5092d7077` Source locator `/tasks/diamond_few_shot_cot/anthropic~1claude-opus-4-8` | | | | | | | | | [Time Horizon Index: KSP / mean progress to date](https://benchize.com/benchmarks/vals-ai%3Atime_horizon_index%3Amean_progress_to_date.md) | 7.74% | SE ±0 pp | 8 / 10 | $1,499.32 | 0source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_ksp_bench.CDDTSam7.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 7.739 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 1499.319504 \| USD \| per_test \| Lower is better \| \| `latency` \| 0 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 13 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_ksp_bench.CDDTSam7.json) Source Vals AI Original model label anthropic/claude-opus-4-8 Source version 1 Observed 2026-10-21T16:46:16.730Z Evidence SHA-256 `be251b72ecceae7912b9577f9b3a3512740c308a34f81ac823952e919adae50f` Source locator `/tasks/mean_progress_to_date/anthropic~1claude-opus-4-8` | | | | | | | | | [Public Benefits Bench v1 / multi turn](https://benchize.com/benchmarks/vals-ai%3Apublic-benefits-bench-v1%3Amulti_turn.md) | 58.46% | SE ±1.28 pp | 2 / 8 | $0.59 | 311.36source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_snap_v1.BPa0Hrre.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 58.457 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.592054 \| USD \| per_test \| Lower is better \| \| `latency` \| 311.358 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.28 pp sourcePopulation 13 stderr 1.282 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_snap_v1.BPa0Hrre.json) Source Vals AI Original model label anthropic/claude-opus-4-8 Source version 1 Observed 2026-10-21T16:46:18.363Z Evidence SHA-256 `b85e2b3cbdf03e13e2ffdb27ce4f108003a548d705a24ae26a0a115e2027475d` Source locator `/tasks/multi_turn/anthropic~1claude-opus-4-8` | | | | | | | | | [Terminal-Bench-Science / physical sciences](https://benchize.com/benchmarks/vals-ai%3Aterminal-bench-science%3Aphysical_sciences.md) | 11.77% | SE ±8.06 pp | 13 / 30 | — | — | [Vals AI · 0.1](https://www.vals.ai/_astro/benchmark_view_terminal_bench_science.o-HR8Nyt.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 11.765 \| percent \| reported \| Higher is better \| ### Configuration computeEffort max harness Mini-SWE-agent maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±8.06 pp sourcePopulation 41 stderr 8.055 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_terminal_bench_science.o-HR8Nyt.json) Source Vals AI Original model label anthropic/claude-opus-4-8 Source version 0.1 Observed 2026-10-21T16:46:03.529Z Evidence SHA-256 `a30a8618deaecf018ea1c8c9f675576cdd9ab79742d05788067db49afcc5f91b` Source locator `/tasks/physical_sciences/anthropic~1claude-opus-4-8` | | | | | | | | | [Legal Research Bench / weighted pass administrative regulatory](https://benchize.com/benchmarks/vals-ai%3Alegal_research%3Aweighted_pass__administrative_regulatory.md) | 87.37% | SE ±2.26 pp | 20 / 50 | $2.73 | 636.54source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 87.365 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 2.731524 \| USD \| per_test \| Lower is better \| \| `latency` \| 636.543 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.26 pp sourcePopulation 82 stderr 2.257 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) Source Vals AI Original model label anthropic/claude-opus-4-8 Source version 1 Observed 2026-10-21T16:45:52.389Z Evidence SHA-256 `26c58dfd47d0d2dc86f33c6d056f30309e0ee63570e7cd752c593b385f9d862e` Source locator `/tasks/weighted_pass__administrative_regulatory/anthropic~1claude-opus-4-8` | | | | | | | | | [Legal Research Bench / weighted pass criminal](https://benchize.com/benchmarks/vals-ai%3Alegal_research%3Aweighted_pass__criminal.md) | 87.75% | SE ±2.44 pp | 5 / 50 | $2.85 | 755.76source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 87.747 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 2.852274 \| USD \| per_test \| Lower is better \| \| `latency` \| 755.757 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.44 pp sourcePopulation 82 stderr 2.443 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) Source Vals AI Original model label anthropic/claude-opus-4-8 Source version 1 Observed 2026-10-21T16:45:52.389Z Evidence SHA-256 `26c58dfd47d0d2dc86f33c6d056f30309e0ee63570e7cd752c593b385f9d862e` Source locator `/tasks/weighted_pass__criminal/anthropic~1claude-opus-4-8` | | | | | | | | | [Vals Multimodal Index / sage](https://benchize.com/benchmarks/vals-ai%3Avals_multimodal_index%3Asage.md) | 54.79% | SE ±3.35 pp | 2 / 20 | $0.47 | 185.59source duration | [Vals AI · 1.2](https://www.vals.ai/_astro/benchmark_view_vals_multimodal_index.virVpWMs.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 54.788 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.47466 \| USD \| per_test \| Lower is better \| \| `latency` \| 185.594 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±3.35 pp sourcePopulation 33 stderr 3.347 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_vals_multimodal_index.virVpWMs.json) Source Vals AI Original model label anthropic/claude-opus-4-8 Source version 1.2 Observed 2026-10-21T16:45:51.881Z Evidence SHA-256 `077b22575f6e075b7b78157d000ec8f94d7ead409c7dcbcf04887630ed572a2d` Source locator `/tasks/sage/anthropic~1claude-opus-4-8` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / emerging companies venture capital](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Aemerging_companies_venture_capital.md) | 20% | SE ±20 pp | 6 / 49 | $8.69 | 1,500source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 20 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 8.692762 \| USD \| per_test \| Lower is better \| \| `latency` \| 1499.998 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±20 pp sourcePopulation 81 stderr 20 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label anthropic/claude-opus-4-8 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/emerging_companies_venture_capital/anthropic~1claude-opus-4-8` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / criteria pass rate](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate.md) | 87.86% | SE ±0.39 pp | 15 / 49 | $10.22 | 1,447.79source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 87.856 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 10.222944 \| USD \| per_test \| Lower is better \| \| `latency` \| 1447.794 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.39 pp sourcePopulation 81 stderr 0.393 tokenTotals {"cache_read_tokens":1089433052,"cache_write_tokens":53824786,"input_tokens":1143265902,"output_tokens":13823660,"reasoning_tokens":null} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label anthropic/claude-opus-4-8 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate/anthropic~1claude-opus-4-8` | | | | | | | | | [BrokenArXiv 04/2026](https://benchize.com/benchmarks/matharena%3Aarxiv_false--april.md) | 34.43% | 95% CI 26–42.86% | 5 / 17 | 5.18usd per problem | — | [matharena · arxiv_false--april](https://matharena.ai/competition_tables/arxiv_false--april) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 34.43 \| percent \| competition \| Higher is better \| \| `cost_per_problem` \| 5.18 \| usd_per_problem \| competition \| Lower is better \| ### Configuration competition arxiv_false--april predicted false Reasoning effort max sourceModelName Claude-Opus-4.8 (max) sourceModelVersion anthropic/opus_48_max thinkingEnabled true ### Reported statistics Uncertainty 95% confidence interval: 26–42.86% confidenceInterval95 8.43 problems 61 ### Source [Original source](https://matharena.ai/competition_tables/arxiv_false--april) Source matharena Original model label Claude-Opus-4.8 (max) Source version arxiv_false--april Observed 2026-10-21T16:52:24.402Z Evidence SHA-256 `e543f14dac0ba9663e8104a87738726c9848b89171a53fdca89b7baa4a892399` Source locator `/table/rows/6` | | | | | | | | | [ProgramBench / partial](https://benchize.com/benchmarks/vals-ai%3Aprogrambench%3Apartial.md) | 71.95% | SE ±1.83 pp | 15 / 40 | $31.27 | 5,523.1source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_programbench.C_J7ZU4X.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 71.945 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 31.267424 \| USD \| per_test \| Lower is better \| \| `latency` \| 5523.099 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.83 pp sourcePopulation 64 stderr 1.828 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_programbench.C_J7ZU4X.json) Source Vals AI Original model label anthropic/claude-opus-4-8 Source version 1 Observed 2026-10-21T16:46:10.137Z Evidence SHA-256 `e9bbb4378c7a058441a21d456baf9316b96c2cf5b58d371a3c7c803f5000bd8c` Source locator `/tasks/partial/anthropic~1claude-opus-4-8` | | | | | | | | | [HLE-Full w/ tools](https://benchize.com/benchmarks/hle-full-tools.md) | 57.9% | — | 3 / 5 | — | — | [Moonshot AI · sha256:0e806bbad76dc8e8dbc899a86051e3406bdd5995fe6c6aa1b2176dde4fb28e6c](https://www.kimi.ai/blog/kimi-k3) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 57.9 \| percent \| reported \| Higher is better \| ### Configuration Reasoning effort max sourceScoreFootnote * thinkingEnabled true toolsEnabled true ### Source Score publisher [Moonshot AI](https://www.kimi.ai/blog/kimi-k3) Collected via [moonshot k3](https://www.kimi.ai/blog/kimi-k3) Publisher reference Referenced by the collection source; the upstream result has not been independently captured. Benchmark [https://www.kimi.ai/blog/kimi-k3](https://www.kimi.ai/blog/kimi-k3) [Collected via](https://www.kimi.ai/blog/kimi-k3) Source Moonshot AI Original model label Claude Opus 4.8 (max) Source version sha256:0e806bbad76dc8e8dbc899a86051e3406bdd5995fe6c6aa1b2176dde4fb28e6c Observed 2026-10-21T16:31:44.130Z Evidence SHA-256 `0e806bbad76dc8e8dbc899a86051e3406bdd5995fe6c6aa1b2176dde4fb28e6c` Source locator `/flight/sheet/data/22/scores/3` | | | | | | | | | [Legal Research Bench / overall](https://benchize.com/benchmarks/vals-ai%3Alegal_research%3Aoverall.md) | 43.75% | SE ±3.45 pp | 11 / 50 | $2.82 | 714.04source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 43.75 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 2.820942 \| USD \| per_test \| Lower is better \| \| `latency` \| 714.036 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±3.45 pp sourcePopulation 82 stderr 3.448 tokenTotals {"cache_read_tokens":98212592,"cache_write_tokens":38178928,"input_tokens":136489421,"output_tokens":12617848,"reasoning_tokens":0} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) Source Vals AI Original model label anthropic/claude-opus-4-8 Source version 1 Observed 2026-10-21T16:45:52.389Z Evidence SHA-256 `26c58dfd47d0d2dc86f33c6d056f30309e0ee63570e7cd752c593b385f9d862e` Source locator `/tasks/overall/anthropic~1claude-opus-4-8` | | | | | | | | | [APEX-Agents](https://benchize.com/benchmarks/apex-agents.md) | 39.4% | — | 4 / 6 | — | — | [Moonshot AI · sha256:0e806bbad76dc8e8dbc899a86051e3406bdd5995fe6c6aa1b2176dde4fb28e6c](https://www.kimi.ai/blog/kimi-k3) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 39.4 \| percent \| reported \| Higher is better \| ### Configuration Reasoning effort max thinkingEnabled true ### Source Score publisher [Moonshot AI](https://www.kimi.ai/blog/kimi-k3) Collected via [moonshot k3](https://www.kimi.ai/blog/kimi-k3) Publisher reference Referenced by the collection source; the upstream result has not been independently captured. Benchmark [https://www.kimi.ai/blog/kimi-k3](https://www.kimi.ai/blog/kimi-k3) [Collected via](https://www.kimi.ai/blog/kimi-k3) Source Moonshot AI Original model label Claude Opus 4.8 (max) Source version sha256:0e806bbad76dc8e8dbc899a86051e3406bdd5995fe6c6aa1b2176dde4fb28e6c Observed 2026-10-21T16:31:44.130Z Evidence SHA-256 `0e806bbad76dc8e8dbc899a86051e3406bdd5995fe6c6aa1b2176dde4fb28e6c` Source locator `/flight/sheet/data/16/scores/3` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / criteria pass rate international trade sanctions](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__international_trade_sanctions.md) | 85.71% | SE ±2.21 pp | 22 / 49 | $7.69 | 1,470.57source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 85.714 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 7.692416 \| USD \| per_test \| Lower is better \| \| `latency` \| 1470.566 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.21 pp sourcePopulation 81 stderr 2.209 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label anthropic/claude-opus-4-8 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__international_trade_sanctions/anthropic~1claude-opus-4-8` | | | | | | | | | [Terminal-Bench-Science / mathematical sciences](https://benchize.com/benchmarks/vals-ai%3Aterminal-bench-science%3Amathematical_sciences.md) | 5.88% | SE ±5.88 pp | 15 / 30 | — | — | [Vals AI · 0.1](https://www.vals.ai/_astro/benchmark_view_terminal_bench_science.o-HR8Nyt.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 5.882 \| percent \| reported \| Higher is better \| ### Configuration computeEffort max harness Mini-SWE-agent maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±5.88 pp sourcePopulation 41 stderr 5.882 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_terminal_bench_science.o-HR8Nyt.json) Source Vals AI Original model label anthropic/claude-opus-4-8 Source version 0.1 Observed 2026-10-21T16:46:03.529Z Evidence SHA-256 `a30a8618deaecf018ea1c8c9f675576cdd9ab79742d05788067db49afcc5f91b` Source locator `/tasks/mathematical_sciences/anthropic~1claude-opus-4-8` | | | | | | | | | [Kangaroo 2025 3-4](https://benchize.com/benchmarks/matharena%3Akangaroo--kangaroo_2025_3-4.md) | 60.42% | 95% CI 46.59–74.25% | 12 / 16 | 0.84usd per problem | — | [matharena · kangaroo--kangaroo_2025_3-4](https://matharena.ai/competition_tables/kangaroo--kangaroo_2025_3-4) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 60.42 \| percent \| competition \| Higher is better \| \| `cost_per_problem` \| 0.84 \| usd_per_problem \| competition \| Lower is better \| ### Configuration competition kangaroo--kangaroo_2025_3-4 predicted false Reasoning effort max sourceModelName Claude-Opus-4.8 (max) sourceModelVersion anthropic/opus_48_max thinkingEnabled true ### Reported statistics Uncertainty 95% confidence interval: 46.59–74.25% confidenceInterval95 13.83 problems 24 ### Source [Original source](https://matharena.ai/competition_tables/kangaroo--kangaroo_2025_3-4) Source matharena Original model label Claude-Opus-4.8 (max) Source version kangaroo--kangaroo_2025_3-4 Observed 2026-10-21T16:52:32.012Z Evidence SHA-256 `38fec99c55752fef6703c9ed49d9e1e204b235f2bd78d5ee47725dac9e5da89c` Source locator `/table/rows/16` | | | | | | | | | [Public Benefits Bench v1.1 / overall](https://benchize.com/benchmarks/vals-ai%3Apublic-benefits-bench%3Aoverall.md) | 68.13% | SE ±1.21 pp | 9 / 34 | $1.89 | 767.85source duration | [Vals AI · 1.1](https://www.vals.ai/_astro/benchmark_view_snap.CUcEPb0v.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 68.133 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 1.889795 \| USD \| per_test \| Lower is better \| \| `latency` \| 767.85 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.21 pp sourcePopulation 50 stderr 1.212 tokenTotals {"cache_read_tokens":130953865,"cache_write_tokens":21982282,"input_tokens":152942625,"output_tokens":9270173,"reasoning_tokens":null} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_snap.CUcEPb0v.json) Source Vals AI Original model label anthropic/claude-opus-4-8 Source version 1.1 Observed 2026-10-21T16:46:17.865Z Evidence SHA-256 `6f091831ab607a26d65d2ade18cba7ce345ea3a67297b8c378e8a06ce398360d` Source locator `/tasks/overall/anthropic~1claude-opus-4-8` | | | | | | | | | [HMMT Feb 2026](https://benchize.com/benchmarks/matharena%3Ahmmt--hmmt_feb_2026.md) | 95.45% | 95% CI 90.42–100.48% | 5 / 21 | 0.76usd per problem | — | [matharena · hmmt--hmmt_feb_2026](https://matharena.ai/competition_tables/hmmt--hmmt_feb_2026) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 95.45 \| percent \| competition \| Higher is better \| \| `cost_per_problem` \| 0.76 \| usd_per_problem \| competition \| Lower is better \| ### Configuration competition hmmt--hmmt_feb_2026 predicted false Reasoning effort max sourceModelName Claude-Opus-4.8 (max) sourceModelVersion anthropic/opus_48_max thinkingEnabled true ### Reported statistics Uncertainty 95% confidence interval: 90.42–100.48% confidenceInterval95 5.03 problems 33 ### Source [Original source](https://matharena.ai/competition_tables/hmmt--hmmt_feb_2026) Source matharena Original model label Claude-Opus-4.8 (max) Source version hmmt--hmmt_feb_2026 Observed 2026-10-21T16:52:38.351Z Evidence SHA-256 `0dce922c33a54d29d4a1872a42d74077b491915c80d8433f4b379ea0b78bb979` Source locator `/table/rows/6` | | | | | | | | | [Finance Agent (v2) / market analysis](https://benchize.com/benchmarks/vals-ai%3Afabv2%3Amarket_analysis.md) | 67.22% | SE ±0.47 pp | 13 / 50 | $2.02 | 414.76source duration | [Vals AI · 2](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 67.219 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 2.017384 \| USD \| per_test \| Lower is better \| \| `latency` \| 414.757 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.47 pp sourcePopulation 80 stderr 0.47 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) Source Vals AI Original model label anthropic/claude-opus-4-8 Source version 2 Observed 2026-10-21T16:45:57.035Z Evidence SHA-256 `f731b20a26c126725d125519f2b5841aa45254656dba859a9570d7ce7e5d8191` Source locator `/tasks/market_analysis/anthropic~1claude-opus-4-8` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / criteria pass rate energy natural resources](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__energy_natural_resources.md) | 97.03% | SE ±0.98 pp | 2 / 49 | $17.56 | 1,738.25source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 97.03 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 17.557582 \| USD \| per_test \| Lower is better \| \| `latency` \| 1738.252 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.98 pp sourcePopulation 81 stderr 0.977 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label anthropic/claude-opus-4-8 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__energy_natural_resources/anthropic~1claude-opus-4-8` | | | | | | | | | [EMB / template](https://benchize.com/benchmarks/vals-ai%3Aemb%3Atemplate.md) | 71.14% | SE ±3.7 pp | 13 / 47 | $12.96 | 2,724.16source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 71.143 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 12.960012 \| USD \| per_test \| Lower is better \| \| `latency` \| 2724.158 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±3.7 pp sourcePopulation 76 stderr 3.696 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) Source Vals AI Original model label anthropic/claude-opus-4-8 Source version 1 Observed 2026-10-21T16:45:57.495Z Evidence SHA-256 `d807aee620ea47152a1fca0257a8dd12a1be4512c76cc1744c1ae9e285180918` Source locator `/tasks/template/anthropic~1claude-opus-4-8` | | | | | | | | | [Legal Research Bench / weighted pass immigration](https://benchize.com/benchmarks/vals-ai%3Alegal_research%3Aweighted_pass__immigration.md) | 78.35% | SE ±8.4 pp | 28 / 50 | $3.44 | 728.3source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 78.35 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 3.441413 \| USD \| per_test \| Lower is better \| \| `latency` \| 728.3 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±8.4 pp sourcePopulation 82 stderr 8.403 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) Source Vals AI Original model label anthropic/claude-opus-4-8 Source version 1 Observed 2026-10-21T16:45:52.389Z Evidence SHA-256 `26c58dfd47d0d2dc86f33c6d056f30309e0ee63570e7cd752c593b385f9d862e` Source locator `/tasks/weighted_pass__immigration/anthropic~1claude-opus-4-8` | | | | | | | | | [SkillsBench / no skills](https://benchize.com/benchmarks/vals-ai%3Askillsbench%3Ano_skills.md) | 43.67% | SE ±4.82 pp | 9 / 20 | $5.34 | 1,012.31source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_skillsbench.BE2TXB-M.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 43.674 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 5.343307 \| USD \| per_test \| Lower is better \| \| `latency` \| 1012.31 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness OpenHands maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±4.82 pp sourcePopulation 35 stderr 4.816 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_skillsbench.BE2TXB-M.json) Source Vals AI Original model label anthropic/claude-opus-4-8 Source version 1 Observed 2026-10-21T16:46:11.147Z Evidence SHA-256 `aaad242949d81b7c7c61c36d4895d4460e9146760c1e6b4b38d235a16043bc61` Source locator `/tasks/no_skills/anthropic~1claude-opus-4-8` | | | | | | | | | [DeepSearchQA (f1-score)](https://benchize.com/benchmarks/deepsearchqa.md) | 93.1% | — | 3 / 3 | — | — | [Moonshot AI · sha256:0e806bbad76dc8e8dbc899a86051e3406bdd5995fe6c6aa1b2176dde4fb28e6c](https://www.kimi.ai/blog/kimi-k3) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `f1` \| 93.1 \| percent \| reported \| Higher is better \| ### Configuration Reasoning effort max thinkingEnabled true ### Source Score publisher [Moonshot AI](https://www.kimi.ai/blog/kimi-k3) Collected via [moonshot k3](https://www.kimi.ai/blog/kimi-k3) Publisher reference Referenced by the collection source; the upstream result has not been independently captured. Benchmark [https://www.kimi.ai/blog/kimi-k3](https://www.kimi.ai/blog/kimi-k3) [Collected via](https://www.kimi.ai/blog/kimi-k3) Source Moonshot AI Original model label Claude Opus 4.8 (max) Source version sha256:0e806bbad76dc8e8dbc899a86051e3406bdd5995fe6c6aa1b2176dde4fb28e6c Observed 2026-10-21T16:31:44.130Z Evidence SHA-256 `0e806bbad76dc8e8dbc899a86051e3406bdd5995fe6c6aa1b2176dde4fb28e6c` Source locator `/flight/sheet/data/10/scores/3` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / overall](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Aoverall.md) | 9.58% | SE ±2.15 pp | 11 / 49 | $10.22 | 1,447.79source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 9.583 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 10.222944 \| USD \| per_test \| Lower is better \| \| `latency` \| 1447.794 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.15 pp sourcePopulation 81 stderr 2.148 tokenTotals {"cache_read_tokens":1089433052,"cache_write_tokens":53824786,"input_tokens":1143265902,"output_tokens":13823660,"reasoning_tokens":null} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label anthropic/claude-opus-4-8 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/overall/anthropic~1claude-opus-4-8` | | | | | | | | | [Kangaroo 2025 7-8](https://benchize.com/benchmarks/matharena%3Akangaroo--kangaroo_2025_7-8.md) | 96.67% | 95% CI 92.13–101.21% | 1 / 15 | 0.48usd per problem | — | [matharena · kangaroo--kangaroo_2025_7-8](https://matharena.ai/competition_tables/kangaroo--kangaroo_2025_7-8) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 96.67 \| percent \| competition \| Higher is better \| \| `cost_per_problem` \| 0.48 \| usd_per_problem \| competition \| Lower is better \| ### Configuration competition kangaroo--kangaroo_2025_7-8 predicted false Reasoning effort max sourceModelName Claude-Opus-4.8 (max) sourceModelVersion anthropic/opus_48_max thinkingEnabled true ### Reported statistics Uncertainty 95% confidence interval: 92.13–101.21% confidenceInterval95 4.54 problems 30 ### Source [Original source](https://matharena.ai/competition_tables/kangaroo--kangaroo_2025_7-8) Source matharena Original model label Claude-Opus-4.8 (max) Source version kangaroo--kangaroo_2025_7-8 Observed 2026-10-21T16:52:33.033Z Evidence SHA-256 `197887eb5244aa6bce3570bd9eb25bbb0bb6ec95568339536d94e2859f0ee8c1` Source locator `/table/rows/1` | | | | | | | | | [EMB / lbo](https://benchize.com/benchmarks/vals-ai%3Aemb%3Albo.md) | 54.38% | SE ±5.35 pp | 23 / 47 | $17.13 | 3,411.87source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 54.384 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 17.127734 \| USD \| per_test \| Lower is better \| \| `latency` \| 3411.868 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±5.35 pp sourcePopulation 76 stderr 5.352 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) Source Vals AI Original model label anthropic/claude-opus-4-8 Source version 1 Observed 2026-10-21T16:45:57.495Z Evidence SHA-256 `d807aee620ea47152a1fca0257a8dd12a1be4512c76cc1744c1ae9e285180918` Source locator `/tasks/lbo/anthropic~1claude-opus-4-8` | | | | | | | | | [ArXivMath 04/2026](https://benchize.com/benchmarks/matharena%3Aarxiv--april.md) | 62.5% | 95% CI 53.84–71.16% | 7 / 19 | 3.5usd per problem | — | [matharena · arxiv--april](https://matharena.ai/competition_tables/arxiv--april) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 62.50 \| percent \| competition \| Higher is better \| \| `cost_per_problem` \| 3.50 \| usd_per_problem \| competition \| Lower is better \| ### Configuration competition arxiv--april predicted false Reasoning effort max sourceModelName Claude-Opus-4.8 (max) sourceModelVersion anthropic/opus_48_max thinkingEnabled true ### Reported statistics Uncertainty 95% confidence interval: 53.84–71.16% confidenceInterval95 8.66 problems 40 ### Source [Original source](https://matharena.ai/competition_tables/arxiv--april) Source matharena Original model label Claude-Opus-4.8 (max) Source version arxiv--april Observed 2026-10-21T16:52:28.840Z Evidence SHA-256 `5139c835884bfab38f18d23a248814af4680927310bc5cd2a86ea2eab779e0a7` Source locator `/table/rows/7` | | | | | | | | | [Tax Agent Bench / rule source lookup](https://benchize.com/benchmarks/vals-ai%3Atax_agent_bench%3Arule_source_lookup.md) | 36.67% | SE ±8.95 pp | 22 / 46 | $1.29 | 342.93source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 36.667 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 1.290724 \| USD \| per_test \| Lower is better \| \| `latency` \| 342.933 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±8.95 pp sourcePopulation 73 stderr 8.949 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) Source Vals AI Original model label anthropic/claude-opus-4-8 Source version 1 Observed 2026-10-21T16:45:55.958Z Evidence SHA-256 `446db182b67efb7923b3187a4040ec3a3ea492b8ad699a2b6d708d714cc063ac` Source locator `/tasks/rule_source_lookup/anthropic~1claude-opus-4-8` | | | | | | | | | [CorpFin v2 / exact pages](https://benchize.com/benchmarks/vals-ai%3Acorp_fin_v2%3Aexact_pages.md) | 64.34% | SE ±1.64 pp | 8 / 39 | $0.04 | 14.11source duration | [Vals AI · 2](https://www.vals.ai/_astro/benchmark_view_corp_fin_v2.CectKwl1.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 64.336 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.039715 \| USD \| per_test \| Lower is better \| \| `latency` \| 14.109 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.64 pp sourcePopulation 134 stderr 1.635 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_corp_fin_v2.CectKwl1.json) Source Vals AI Original model label anthropic/claude-opus-4-8 Source version 2 Observed 2026-10-21T16:45:57.998Z Evidence SHA-256 `7cf2217ab3728d5a84bffce82aab9fc52d7f810b0cbe8c65b387df47f3d31c9b` Source locator `/tasks/exact_pages/anthropic~1claude-opus-4-8` | | | | | | | | | [Terminal-Bench 2.1 / overall](https://benchize.com/benchmarks/vals-ai%3Aterminal-bench-2-1%3Aoverall.md) | 71.91% | SE ±0.65 pp | 21 / 43 | $2.41 | 929.9source duration | [Vals AI · 2.1](https://www.vals.ai/_astro/benchmark_view_terminal_bench_2_1.CdyoObsA.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 71.91 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 2.409772 \| USD \| per_test \| Lower is better \| \| `latency` \| 929.902 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.65 pp sourcePopulation 76 stderr 0.649 tokenTotals {"cache_read_tokens":76658876,"cache_write_tokens":7553615,"input_tokens":84371822,"output_tokens":5125339,"reasoning_tokens":0} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_terminal_bench_2_1.CdyoObsA.json) Source Vals AI Original model label anthropic/claude-opus-4-8 Source version 2.1 Observed 2026-10-21T16:46:13.216Z Evidence SHA-256 `a5306805cdf86495bdcceff6b2c8534a3768b9853f5077895dab975b809b054f` Source locator `/tasks/overall/anthropic~1claude-opus-4-8` | | | | | | | | | [SAGE / linear algebra](https://benchize.com/benchmarks/vals-ai%3Asage%3Alinear_algebra.md) | 78.25% | SE ±4.69 pp | 1 / 41 | $0.41 | 116.99source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_sage.RRwIIzO5.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 78.253 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.40882 \| USD \| per_test \| Lower is better \| \| `latency` \| 116.985 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 30000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±4.69 pp sourcePopulation 91 stderr 4.694 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_sage.RRwIIzO5.json) Source Vals AI Original model label anthropic/claude-opus-4-8 Source version 1 Observed 2026-10-21T16:46:07.515Z Evidence SHA-256 `cc5fb3fecdb0382d492ff167a42ad84b9c5c3883373c78ed87baaa972b8b6f7c` Source locator `/tasks/linear_algebra/anthropic~1claude-opus-4-8` | | | | | | | | | [Public Benefits Bench v1 / neither](https://benchize.com/benchmarks/vals-ai%3Apublic-benefits-bench-v1%3Aneither.md) | 53.11% | SE ±1.3 pp | 2 / 8 | $0.27 | 148.61source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_snap_v1.BPa0Hrre.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 53.112 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.269121 \| USD \| per_test \| Lower is better \| \| `latency` \| 148.613 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.3 pp sourcePopulation 13 stderr 1.298 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_snap_v1.BPa0Hrre.json) Source Vals AI Original model label anthropic/claude-opus-4-8 Source version 1 Observed 2026-10-21T16:46:18.363Z Evidence SHA-256 `b85e2b3cbdf03e13e2ffdb27ce4f108003a548d705a24ae26a0a115e2027475d` Source locator `/tasks/neither/anthropic~1claude-opus-4-8` | | | | | | | | | [CharXiv (RQ)](https://benchize.com/benchmarks/charxiv-rq.md) | 80.5% | — | 5 / 5 | — | — | [Moonshot AI · sha256:0e806bbad76dc8e8dbc899a86051e3406bdd5995fe6c6aa1b2176dde4fb28e6c](https://www.kimi.ai/blog/kimi-k3) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 80.5 \| percent \| reported \| Higher is better \| ### Configuration Reasoning effort max runs 3 subset RQ thinkingEnabled true toolsEnabled false ### Reported statistics runCount 3 ### Source Score publisher [Moonshot AI](https://www.kimi.ai/blog/kimi-k3) Collected via [moonshot k3](https://www.kimi.ai/blog/kimi-k3) Publisher reference Referenced by the collection source; the upstream result has not been independently captured. Benchmark [https://www.kimi.ai/blog/kimi-k3](https://www.kimi.ai/blog/kimi-k3) [Collected via](https://www.kimi.ai/blog/kimi-k3) Source Moonshot AI Original model label Claude Opus 4.8 (max) Source version sha256:0e806bbad76dc8e8dbc899a86051e3406bdd5995fe6c6aa1b2176dde4fb28e6c Observed 2026-10-21T16:31:44.130Z Evidence SHA-256 `0e806bbad76dc8e8dbc899a86051e3406bdd5995fe6c6aa1b2176dde4fb28e6c` Source locator `/flight/sheet/data/25/scores/3` | | | | | | | | | [Vals Multimodal Index / corp fin v2](https://benchize.com/benchmarks/vals-ai%3Avals_multimodal_index%3Acorp_fin_v2.md) | 65.97% | SE ±1.62 pp | 13 / 20 | $0.68 | 20.38source duration | [Vals AI · 1.2](https://www.vals.ai/_astro/benchmark_view_vals_multimodal_index.virVpWMs.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 65.967 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.679358 \| USD \| per_test \| Lower is better \| \| `latency` \| 20.38 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.62 pp sourcePopulation 33 stderr 1.618 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_vals_multimodal_index.virVpWMs.json) Source Vals AI Original model label anthropic/claude-opus-4-8 Source version 1.2 Observed 2026-10-21T16:45:51.881Z Evidence SHA-256 `077b22575f6e075b7b78157d000ec8f94d7ead409c7dcbcf04887630ed572a2d` Source locator `/tasks/corp_fin_v2/anthropic~1claude-opus-4-8` | | | | | | | | | [AIME 2026](https://benchize.com/benchmarks/aime-2026.md) | 100% | 95% CI 100–100% | 1 / 21 | 0.54usd per problem | — | [matharena · aime--aime_2026](https://matharena.ai/competition_tables/aime--aime_2026) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 100.00 \| percent \| competition \| Higher is better \| \| `cost_per_problem` \| 0.54 \| usd_per_problem \| competition \| Lower is better \| ### Configuration competition aime--aime_2026 predicted false Reasoning effort max sourceModelName Claude-Opus-4.8 (max) sourceModelVersion anthropic/opus_48_max thinkingEnabled true ### Reported statistics Uncertainty 95% confidence interval: 100–100% confidenceInterval95 0.00 problems 30 ### Source [Original source](https://matharena.ai/competition_tables/aime--aime_2026) Source matharena Original model label Claude-Opus-4.8 (max) Source version aime--aime_2026 Observed 2026-10-21T16:52:37.816Z Evidence SHA-256 `58b3ac1e0805975eba566c493b8e0165d41eb7ca517218106e112529fb6342de` Source locator `/table/rows/1` | | | | | | | | | [Finance Agent (v2) / overall](https://benchize.com/benchmarks/vals-ai%3Afabv2%3Aoverall.md) | 53.92% | SE ±0.16 pp | 22 / 50 | $4.22 | 547.4source duration | [Vals AI · 2](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 53.918 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 4.218538 \| USD \| per_test \| Lower is better \| \| `latency` \| 547.397 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.16 pp sourcePopulation 80 stderr 0.159 tokenTotals {"cache_read_tokens":82013466,"cache_write_tokens":227923817,"input_tokens":309993285,"output_tokens":17301258,"reasoning_tokens":null} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) Source Vals AI Original model label anthropic/claude-opus-4-8 Source version 2 Observed 2026-10-21T16:45:57.035Z Evidence SHA-256 `f731b20a26c126725d125519f2b5841aa45254656dba859a9570d7ce7e5d8191` Source locator `/tasks/overall/anthropic~1claude-opus-4-8` | | | | | | | | | [GPQA Diamond](https://benchize.com/benchmarks/gpqa-diamond.md) | 91% | — | 6 / 6 | — | — | [Moonshot AI · sha256:0e806bbad76dc8e8dbc899a86051e3406bdd5995fe6c6aa1b2176dde4fb28e6c](https://www.kimi.ai/blog/kimi-k3) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 91.0 \| percent \| reported \| Higher is better \| ### Configuration Reasoning effort max thinkingEnabled true ### Source Score publisher [Moonshot AI](https://www.kimi.ai/blog/kimi-k3) Collected via [moonshot k3](https://www.kimi.ai/blog/kimi-k3) Publisher reference Referenced by the collection source; the upstream result has not been independently captured. Benchmark [https://www.kimi.ai/blog/kimi-k3](https://www.kimi.ai/blog/kimi-k3) [Collected via](https://www.kimi.ai/blog/kimi-k3) Source Moonshot AI Original model label Claude Opus 4.8 (max) Source version sha256:0e806bbad76dc8e8dbc899a86051e3406bdd5995fe6c6aa1b2176dde4fb28e6c Observed 2026-10-21T16:31:44.130Z Evidence SHA-256 `0e806bbad76dc8e8dbc899a86051e3406bdd5995fe6c6aa1b2176dde4fb28e6c` Source locator `/flight/sheet/data/20/scores/3` | | | | | | | | | [ProgramBench / overall](https://benchize.com/benchmarks/vals-ai%3Aprogrambench%3Aoverall.md) | 71.9% | — | 4 / 6 | — | — | [Moonshot AI · sha256:0e806bbad76dc8e8dbc899a86051e3406bdd5995fe6c6aa1b2176dde4fb28e6c](https://www.kimi.ai/blog/kimi-k3) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 71.9 \| percent \| reported \| Higher is better \| ### Configuration Reasoning effort max thinkingEnabled true ### Source Score publisher [Moonshot AI](https://www.kimi.ai/blog/kimi-k3) Collected via [moonshot k3](https://www.kimi.ai/blog/kimi-k3) Publisher reference Referenced by the collection source; the upstream result has not been independently captured. Benchmark [https://www.kimi.ai/blog/kimi-k3](https://www.kimi.ai/blog/kimi-k3) [Collected via](https://www.kimi.ai/blog/kimi-k3) Source Moonshot AI Original model label Claude Opus 4.8 (max) Source version sha256:0e806bbad76dc8e8dbc899a86051e3406bdd5995fe6c6aa1b2176dde4fb28e6c Observed 2026-10-21T16:31:44.130Z Evidence SHA-256 `0e806bbad76dc8e8dbc899a86051e3406bdd5995fe6c6aa1b2176dde4fb28e6c` Source locator `/flight/sheet/data/1/scores/3` | | | | | | | | | [Tax Agent Bench / weighted pass rule source lookup](https://benchize.com/benchmarks/vals-ai%3Atax_agent_bench%3Aweighted_pass__rule_source_lookup.md) | 54.71% | SE ±8.92 pp | 26 / 46 | $1.29 | 342.93source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 54.706 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 1.290724 \| USD \| per_test \| Lower is better \| \| `latency` \| 342.933 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±8.92 pp sourcePopulation 73 stderr 8.916 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) Source Vals AI Original model label anthropic/claude-opus-4-8 Source version 1 Observed 2026-10-21T16:45:55.958Z Evidence SHA-256 `446db182b67efb7923b3187a4040ec3a3ea492b8ad699a2b6d708d714cc063ac` Source locator `/tasks/weighted_pass__rule_source_lookup/anthropic~1claude-opus-4-8` | | | | | | | | | [Tax Agent Bench / controversy precedence analysis](https://benchize.com/benchmarks/vals-ai%3Atax_agent_bench%3Acontroversy_precedence_analysis.md) | 13.13% | SE ±6.22 pp | 15 / 46 | $1.99 | 562.15source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 13.133 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 1.989157 \| USD \| per_test \| Lower is better \| \| `latency` \| 562.149 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±6.22 pp sourcePopulation 73 stderr 6.22 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) Source Vals AI Original model label anthropic/claude-opus-4-8 Source version 1 Observed 2026-10-21T16:45:55.958Z Evidence SHA-256 `446db182b67efb7923b3187a4040ec3a3ea492b8ad699a2b6d708d714cc063ac` Source locator `/tasks/controversy_precedence_analysis/anthropic~1claude-opus-4-8` | | | | | | | | | [Code Migration / cobol](https://benchize.com/benchmarks/vals-ai%3Acode-migration%3Acobol.md) | 68.57% | SE ±15.03 pp | 19 / 47 | $15.8 | 3,536.15source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_code_migration.Dttzoqk3.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 68.571 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 15.800701 \| USD \| per_test \| Lower is better \| \| `latency` \| 3536.152 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±15.03 pp sourcePopulation 79 stderr 15.028 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_code_migration.Dttzoqk3.json) Source Vals AI Original model label anthropic/claude-opus-4-8 Source version 1 Observed 2026-10-21T16:46:12.712Z Evidence SHA-256 `99194597f2c76c402195be47839409046e18bd30454616da6dd40b736a088fa4` Source locator `/tasks/cobol/anthropic~1claude-opus-4-8` | | | | | | | | | [ArXivMath 02/2026](https://benchize.com/benchmarks/matharena%3Aarxiv--february.md) | 60.16% | 95% CI 51.68–68.64% | 4 / 20 | 4.68usd per problem | — | [matharena · arxiv--february](https://matharena.ai/competition_tables/arxiv--february) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 60.16 \| percent \| competition \| Higher is better \| \| `cost_per_problem` \| 4.68 \| usd_per_problem \| competition \| Lower is better \| ### Configuration competition arxiv--february predicted false Reasoning effort max sourceModelName Claude-Opus-4.8 (max) sourceModelVersion anthropic/opus_48_max thinkingEnabled true ### Reported statistics Uncertainty 95% confidence interval: 51.68–68.64% confidenceInterval95 8.48 problems 32 ### Source [Original source](https://matharena.ai/competition_tables/arxiv--february) Source matharena Original model label Claude-Opus-4.8 (max) Source version arxiv--february Observed 2026-10-21T16:52:27.963Z Evidence SHA-256 `3bc5b0a1eb91a0bdaa3d0942ebf82327a8664334b225ae3b2dc1d8b4f57efa51` Source locator `/table/rows/3` | | | | | | | | | [Vals Multimodal Index / finance agent](https://benchize.com/benchmarks/vals-ai%3Avals_multimodal_index%3Afinance_agent.md) | 53.95% | SE ±1.76 pp | 9 / 20 | $4.22 | 522.68source duration | [Vals AI · 1.2](https://www.vals.ai/_astro/benchmark_view_vals_multimodal_index.virVpWMs.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 53.949 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 4.218538 \| USD \| per_test \| Lower is better \| \| `latency` \| 522.678 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.76 pp sourcePopulation 33 stderr 1.764 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_vals_multimodal_index.virVpWMs.json) Source Vals AI Original model label anthropic/claude-opus-4-8 Source version 1.2 Observed 2026-10-21T16:45:51.881Z Evidence SHA-256 `077b22575f6e075b7b78157d000ec8f94d7ead409c7dcbcf04887630ed572a2d` Source locator `/tasks/finance_agent/anthropic~1claude-opus-4-8` | | | | | | | | | [MMLU Pro / history](https://benchize.com/benchmarks/vals-ai%3Ammlu_pro%3Ahistory.md) | 82.68% | SE ±1.94 pp | 9 / 40 | $0.05 | 12.47source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_mmlu_pro.y416pXDi.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 82.677 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.04551 \| USD \| per_test \| Lower is better \| \| `latency` \| 12.47 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.94 pp sourcePopulation 138 stderr 1.939 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_mmlu_pro.y416pXDi.json) Source Vals AI Original model label anthropic/claude-opus-4-8 Source version 1 Observed 2026-10-21T16:46:06.503Z Evidence SHA-256 `dab025507f46c716dd8d145e90ec0a737d9c56219853f41e4b1cc3c5e8c222b4` Source locator `/tasks/history/anthropic~1claude-opus-4-8` | | | | | | | | | [SWE-bench / <15 min fix](https://benchize.com/benchmarks/vals-ai%3Aswebench%3A%3C15%20min%20fix.md) | 92.78% | SE ±1.86 pp | 7 / 39 | $0.72 | 418.3source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_swebench.Ae8ni7h0.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 92.784 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.715925 \| USD \| per_test \| Lower is better \| \| `latency` \| 418.303 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.86 pp sourcePopulation 88 stderr 1.863 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_swebench.Ae8ni7h0.json) Source Vals AI Original model label anthropic/claude-opus-4-8 Source version 1 Observed 2026-10-21T16:46:13.711Z Evidence SHA-256 `7a4e2e8485283caac500127419f68ec4bd5414c837ade114fc6afa5b2d34ccaa` Source locator `/tasks/<15 min fix/anthropic~1claude-opus-4-8` | | | | | | | | | [Terminal-Bench 4.0 / software](https://benchize.com/benchmarks/vals-ai%3Aterminal-bench-4%3Asoftware.md) | 16.67% | SE ±5.56 pp | 24 / 36 | — | — | [Vals AI · 4.0](https://www.vals.ai/_astro/benchmark_view_terminal_bench_4.CAnGyzC_.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 16.667 \| percent \| reported \| Higher is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±5.56 pp sourcePopulation 52 stderr 5.556 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_terminal_bench_4.CAnGyzC_.json) Source Vals AI Original model label anthropic/claude-opus-4-8 Source version 4.0 Observed 2026-10-21T16:46:10.642Z Evidence SHA-256 `98f6964580e901f1aa143db664bd92aa558415f643569492545d67c47d7f519b` Source locator `/tasks/software/anthropic~1claude-opus-4-8` | | | | | | | | | [MLS Bench](https://benchize.com/benchmarks/mls-bench-lite.md) | 42.8% | — | 4 / 6 | — | — | [Moonshot AI · sha256:0e806bbad76dc8e8dbc899a86051e3406bdd5995fe6c6aa1b2176dde4fb28e6c](https://www.kimi.ai/blog/kimi-k3) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 42.8 \| percent \| reported \| Higher is better \| ### Configuration harness Claude Code Reasoning effort max thinkingEnabled true ### Source Score publisher [Moonshot AI](https://www.kimi.ai/blog/kimi-k3) Collected via [moonshot k3](https://www.kimi.ai/blog/kimi-k3) Publisher reference Referenced by the collection source; the upstream result has not been independently captured. Benchmark [https://www.kimi.ai/blog/kimi-k3](https://www.kimi.ai/blog/kimi-k3) [Collected via](https://www.kimi.ai/blog/kimi-k3) Source Moonshot AI Original model label Claude Opus 4.8 (max) Source version sha256:0e806bbad76dc8e8dbc899a86051e3406bdd5995fe6c6aa1b2176dde4fb28e6c Observed 2026-10-21T16:31:44.130Z Evidence SHA-256 `0e806bbad76dc8e8dbc899a86051e3406bdd5995fe6c6aa1b2176dde4fb28e6c` Source locator `/flight/sheet/data/6/scores/3` | | | | | | | | | [SpreadsheetBench v2-full](https://benchize.com/benchmarks/spreadsheetbench%3Av2-full.md) | 31.55% | — | 4 / 6 | — | — | [Moonshot AI · sha256:0e806bbad76dc8e8dbc899a86051e3406bdd5995fe6c6aa1b2176dde4fb28e6c](https://www.kimi.ai/blog/kimi-k3) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 31.55 \| percent \| reported \| Higher is better \| ### Configuration harness Claude Code Reasoning effort max sourceScoreFootnote * thinkingEnabled true ### Source Score publisher [Moonshot AI](https://www.kimi.ai/blog/kimi-k3) Collected via [moonshot k3](https://www.kimi.ai/blog/kimi-k3) Publisher reference Referenced by the collection source; the upstream result has not been independently captured. Benchmark [https://www.kimi.ai/blog/kimi-k3](https://www.kimi.ai/blog/kimi-k3) [Collected via](https://www.kimi.ai/blog/kimi-k3) Source Moonshot AI Original model label Claude Opus 4.8 (max) Source version sha256:0e806bbad76dc8e8dbc899a86051e3406bdd5995fe6c6aa1b2176dde4fb28e6c Observed 2026-10-21T16:31:44.130Z Evidence SHA-256 `0e806bbad76dc8e8dbc899a86051e3406bdd5995fe6c6aa1b2176dde4fb28e6c` Source locator `/flight/sheet/data/18/scores/3` | | | | | | | | | [LegalBench / interpretation tasks](https://benchize.com/benchmarks/vals-ai%3Alegal_bench%3Ainterpretation_tasks.md) | 82.7% | SE ±0.2 pp | 17 / 44 | $0.00753 | 2.25source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_bench.D6uTgA3b.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 82.703 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.007534 \| USD \| per_test \| Lower is better \| \| `latency` \| 2.245 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.2 pp sourcePopulation 151 stderr 0.202 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_bench.D6uTgA3b.json) Source Vals AI Original model label anthropic/claude-opus-4-8 Source version 1 Observed 2026-10-21T16:45:53.390Z Evidence SHA-256 `c8486ba756baa5cd3955d035edff67440e8db23e69d62ad1a9e18d15b8c4bc52` Source locator `/tasks/interpretation_tasks/anthropic~1claude-opus-4-8` | | | | | | | | | [Finance Agent (v2) / general qualitative analysis](https://benchize.com/benchmarks/vals-ai%3Afabv2%3Ageneral_qualitative_analysis.md) | 66.41% | SE ±2.13 pp | 21 / 50 | $5.27 | 842.8source duration | [Vals AI · 2](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 66.414 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 5.273958 \| USD \| per_test \| Lower is better \| \| `latency` \| 842.804 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.13 pp sourcePopulation 80 stderr 2.134 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) Source Vals AI Original model label anthropic/claude-opus-4-8 Source version 2 Observed 2026-10-21T16:45:57.035Z Evidence SHA-256 `f731b20a26c126725d125519f2b5841aa45254656dba859a9570d7ce7e5d8191` Source locator `/tasks/general_qualitative_analysis/anthropic~1claude-opus-4-8` | | | | | | | | | [LiveCodeBench / medium](https://benchize.com/benchmarks/vals-ai%3Alcb%3Amedium.md) | 89.82% | SE ±1.55 pp | 11 / 43 | $0.16 | 87.12source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_lcb.miyRUvVa.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 89.817 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.162481 \| USD \| per_test \| Lower is better \| \| `latency` \| 87.119 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.55 pp sourcePopulation 143 stderr 1.545 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_lcb.miyRUvVa.json) Source Vals AI Original model label anthropic/claude-opus-4-8 Source version 1 Observed 2026-10-21T16:46:14.220Z Evidence SHA-256 `1a7c7dabc9d95a0182886b1edc42623987cac2095a0d407a5e334f37acdfa2f5` Source locator `/tasks/medium/anthropic~1claude-opus-4-8` | | | | | | | | | [SWE-bench / >4 hours](https://benchize.com/benchmarks/vals-ai%3Aswebench%3A%3E4%20hours.md) | 66.67% | SE ±33.33 pp | 6 / 39 | $3.17 | 1,263.34source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_swebench.Ae8ni7h0.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 66.667 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 3.172757 \| USD \| per_test \| Lower is better \| \| `latency` \| 1263.337 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±33.33 pp sourcePopulation 88 stderr 33.333 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_swebench.Ae8ni7h0.json) Source Vals AI Original model label anthropic/claude-opus-4-8 Source version 1 Observed 2026-10-21T16:46:13.711Z Evidence SHA-256 `7a4e2e8485283caac500127419f68ec4bd5414c837ade114fc6afa5b2d34ccaa` Source locator `/tasks/>4 hours/anthropic~1claude-opus-4-8` | | | | | | | | | [Apex](https://benchize.com/benchmarks/matharena%3Aapex--apex_2025.md) | 81.25% | 95% CI 73.44–89.06% | 1 / 30 | 4.59usd per problem | — | [matharena · apex--apex_2025](https://matharena.ai/competition_tables/apex--apex_2025) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 81.25 \| percent \| competition \| Higher is better \| \| `cost_per_problem` \| 4.59 \| usd_per_problem \| competition \| Lower is better \| ### Configuration competition apex--apex_2025 predicted false Reasoning effort max sourceModelName Claude-Opus-4.8 (max) sourceModelVersion anthropic/opus_48_max thinkingEnabled true ### Reported statistics Uncertainty 95% confidence interval: 73.44–89.06% confidenceInterval95 7.81 problems 12 ### Source [Original source](https://matharena.ai/competition_tables/apex--apex_2025) Source matharena Original model label Claude-Opus-4.8 (max) Source version apex--apex_2025 Observed 2026-10-21T16:52:38.728Z Evidence SHA-256 `20b28370d9d7f5a15a19fc9e4226c9857cdb07bc42b32ccec508e687b7b9f3fc` Source locator `/table/rows/0` | | | | | | | | | [EMB / dataroom](https://benchize.com/benchmarks/vals-ai%3Aemb%3Adataroom.md) | 85.95% | SE ±3.72 pp | 6 / 47 | $8.99 | 2,182.51source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 85.948 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 8.994203 \| USD \| per_test \| Lower is better \| \| `latency` \| 2182.513 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±3.72 pp sourcePopulation 76 stderr 3.719 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) Source Vals AI Original model label anthropic/claude-opus-4-8 Source version 1 Observed 2026-10-21T16:45:57.495Z Evidence SHA-256 `d807aee620ea47152a1fca0257a8dd12a1be4512c76cc1744c1ae9e285180918` Source locator `/tasks/dataroom/anthropic~1claude-opus-4-8` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / arbitration international dispute resolution](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Aarbitration_international_dispute_resolution.md) | 0% | SE ±0 pp | 4 / 49 | $12.08 | 1,580.33source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 12.077574 \| USD \| per_test \| Lower is better \| \| `latency` \| 1580.326 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 81 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label anthropic/claude-opus-4-8 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/arbitration_international_dispute_resolution/anthropic~1claude-opus-4-8` | | | | | | | | | [EMB / comps](https://benchize.com/benchmarks/vals-ai%3Aemb%3Acomps.md) | 72.8% | SE ±4.2 pp | 12 / 47 | $12.89 | 2,442.71source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 72.797 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 12.886468 \| USD \| per_test \| Lower is better \| \| `latency` \| 2442.712 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±4.2 pp sourcePopulation 76 stderr 4.198 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) Source Vals AI Original model label anthropic/claude-opus-4-8 Source version 1 Observed 2026-10-21T16:45:57.495Z Evidence SHA-256 `d807aee620ea47152a1fca0257a8dd12a1be4512c76cc1744c1ae9e285180918` Source locator `/tasks/comps/anthropic~1claude-opus-4-8` | | | | | | | | | [LegalBench / rule tasks](https://benchize.com/benchmarks/vals-ai%3Alegal_bench%3Arule_tasks.md) | 79.98% | SE ±1.72 pp | 22 / 44 | $0.00658 | 4.08source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_bench.D6uTgA3b.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 79.976 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.006584 \| USD \| per_test \| Lower is better \| \| `latency` \| 4.075 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.72 pp sourcePopulation 151 stderr 1.72 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_bench.D6uTgA3b.json) Source Vals AI Original model label anthropic/claude-opus-4-8 Source version 1 Observed 2026-10-21T16:45:53.390Z Evidence SHA-256 `c8486ba756baa5cd3955d035edff67440e8db23e69d62ad1a9e18d15b8c4bc52` Source locator `/tasks/rule_tasks/anthropic~1claude-opus-4-8` | | | | | | | | | [ARC-AGI-1](https://benchize.com/benchmarks/arc-prize%3Av1_Semi_Private.md) | 92.5% | — | 47 / 213 | 2.33usd per task | — | [arc prize · v1](https://arcprize.org/media/data/leaderboard/v1.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 0.925 \| score_fraction \| v1_Semi_Private \| Higher is better \| \| `cost_per_task` \| 2.332 \| usd_per_task \| v1_Semi_Private \| Lower is better \| ### Configuration evaluationSplit v1_Semi_Private harness Not reported modelGroup anthropic-opus-4-8 modelType CoT Reasoning effort max sourceModelVersion anthropic-opus-4-8-max thinkingEnabled true ### Source [Original source](https://arcprize.org/media/data/leaderboard/v1.json) Source arc prize Original model label Claude Opus 4.8 (Max) Source version v1 Observed 2026-10-21T16:34:20.378Z Evidence SHA-256 `302d8267c0522bf450ec1deeac4d060e2cf2fb444ba0a4269014d9535de0bd1f` Source locator `/evaluations/120` | | | | | | | | | [SAGE / overall](https://benchize.com/benchmarks/vals-ai%3Asage%3Aoverall.md) | 54.79% | SE ±3.35 pp | 3 / 41 | $0.51 | 185.59source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_sage.RRwIIzO5.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 54.788 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.5055 \| USD \| per_test \| Lower is better \| \| `latency` \| 185.594 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 30000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±3.35 pp sourcePopulation 91 stderr 3.347 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_sage.RRwIIzO5.json) Source Vals AI Original model label anthropic/claude-opus-4-8 Source version 1 Observed 2026-10-21T16:46:07.515Z Evidence SHA-256 `cc5fb3fecdb0382d492ff167a42ad84b9c5c3883373c78ed87baaa972b8b6f7c` Source locator `/tasks/overall/anthropic~1claude-opus-4-8` | | | | | | | | | [ArXivMath 05/2026](https://benchize.com/benchmarks/matharena%3Aarxiv--may.md) | 65% | 95% CI 56.47–73.53% | 10 / 21 | 4.16usd per problem | — | [matharena · arxiv--may](https://matharena.ai/competition_tables/arxiv--may) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 65.00 \| percent \| competition \| Higher is better \| \| `cost_per_problem` \| 4.16 \| usd_per_problem \| competition \| Lower is better \| ### Configuration competition arxiv--may predicted false Reasoning effort max sourceModelName Claude-Opus-4.8 (max) sourceModelVersion anthropic/opus_48_max thinkingEnabled true ### Reported statistics Uncertainty 95% confidence interval: 56.47–73.53% confidenceInterval95 8.53 problems 40 ### Source [Original source](https://matharena.ai/competition_tables/arxiv--may) Source matharena Original model label Claude-Opus-4.8 (max) Source version arxiv--may Observed 2026-10-21T16:52:29.369Z Evidence SHA-256 `758e1bba77e36f4ced2a51b4bc0d46bd956ed94ef6f57e93e1a974b75e76a4ce` Source locator `/table/rows/10` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / criteria pass rate banking finance](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__banking_finance.md) | 87.73% | SE ±2 pp | 23 / 49 | $9.1 | 1,245.96source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 87.732 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 9.097239 \| USD \| per_test \| Lower is better \| \| `latency` \| 1245.96 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2 pp sourcePopulation 81 stderr 2.004 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label anthropic/claude-opus-4-8 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__banking_finance/anthropic~1claude-opus-4-8` | | | | | | | | | [FrontierCode 1 extended](https://benchize.com/benchmarks/frontiercode%3Av1%3Aextended.md) | 49.89% | — | 7 / 39 | $7.52 | — | [frontiercode](https://cognition.com/data/frontiercode-leaderboard/data.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 0.4989 \| score_fraction \| blocking_criteria_adjusted \| Higher is better \| \| `correct` \| 0.5393 \| score_fraction \| raw_rubric \| Higher is better \| \| `cost` \| 7.5247 \| USD \| per_rollout \| Lower is better \| ### Configuration harness claude-code Reasoning effort max sourceVersion v1 subset extended ### Reported statistics flaggedRate Not reported sampleSize 150 ### Source [Original source](https://cognition.com/data/frontiercode-leaderboard/data.json) Source frontiercode Original model label Claude Opus 4.8 Observed 2026-10-21T16:31:00.878Z Evidence SHA-256 `ff0040778b7bdf69c705ad7474f771fb5e53cfaf5cf8573b3a4614153543d75f` Source locator `/v1/data/Claude%20Opus%204.8/max/extended` | | | | | | | | [Next page](https://benchize.com/models/claude-opus-4-8-max.md?cursor=6191951f7185019ec31c3d41629a6db5315a6f5800ce8f42c93259f5df098cda) --- Cite as: Claude Opus 4.8Max — Benchize, https://benchize.com/models/claude-opus-4-8-max (retrieved 2026-10-21)