# Claude Sonnet 5.5Max > Source: Benchize (https://benchize.com) — https://benchize.com/models/claude-sonnet-5-5-max. When citing or reusing this content, credit "Benchize" and link to https://benchize.com/models/claude-sonnet-5-5-max. Anthropic Canonical URL: https://benchize.com/models/claude-sonnet-5-5-max Last updated: Wed, 21 Oct 2026 18:52:20 GMT Reproduction and quotation must name Benchize, link to the canonical HTML URL, and retain individual source citations. Anthropic ## Benchmarks | Benchmark | Score | ± | Rank | Cost | Latency | Source | | | --- | --- | --- | --- | --- | --- | --- | --- | | [HLAB-Vals-1.1 / emerging companies venture capital](https://benchize.com/benchmarks/vals-ai%3Ahlab%3Aemerging_companies_venture_capital.md) | 10% | SE ±10 pp | 8 / 14 | $16.61 | 4,124.45source duration | [Vals AI · 1.1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 10 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 16.606099 \| USD \| per_test \| Lower is better \| \| `latency` \| 4124.448 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±10 pp sourcePopulation 24 stderr 10 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) Source Vals AI Original model label anthropic/claude-sonnet-5-5 Source version 1.1 Observed 2026-10-21T16:45:52.890Z Evidence SHA-256 `14681f4a2310e0f257df9ec07c9c81e5dadc8f6f71634f02e174a02a8096d868` Source locator `/tasks/emerging_companies_venture_capital/anthropic~1claude-sonnet-5-5` | | | | | | | | | [SAGE / linear algebra](https://benchize.com/benchmarks/vals-ai%3Asage%3Alinear_algebra.md) | 70.79% | SE ±5.2 pp | 11 / 41 | $0.12 | 76.37source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_sage.RRwIIzO5.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 70.792 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.120698 \| USD \| per_test \| Lower is better \| \| `latency` \| 76.371 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±5.2 pp sourcePopulation 91 stderr 5.201 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_sage.RRwIIzO5.json) Source Vals AI Original model label anthropic/claude-sonnet-5-5 Source version 1 Observed 2026-10-21T16:46:07.515Z Evidence SHA-256 `cc5fb3fecdb0382d492ff167a42ad84b9c5c3883373c78ed87baaa972b8b6f7c` Source locator `/tasks/linear_algebra/anthropic~1claude-sonnet-5-5` | | | | | | | | | [Legal Research Bench / weighted pass](https://benchize.com/benchmarks/vals-ai%3Alegal_research%3Aweighted_pass.md) | 84.27% | SE ±1.74 pp | 23 / 50 | $14.83 | 5,588.99source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 84.274 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 14.826571 \| USD \| per_test \| Lower is better \| \| `latency` \| 5588.989 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.74 pp sourcePopulation 82 stderr 1.74 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) Source Vals AI Original model label anthropic/claude-sonnet-5-5 Source version 1 Observed 2026-10-21T16:45:52.389Z Evidence SHA-256 `26c58dfd47d0d2dc86f33c6d056f30309e0ee63570e7cd752c593b385f9d862e` Source locator `/tasks/weighted_pass/anthropic~1claude-sonnet-5-5` | | | | | | | | | [Code Migration / cli](https://benchize.com/benchmarks/vals-ai%3Acode-migration%3Acli.md) | 69.77% | SE ±2.52 pp | 1 / 47 | $94.64 | 13,638.87source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_code_migration.Dttzoqk3.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 69.766 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 94.636889 \| USD \| per_test \| Lower is better \| \| `latency` \| 13638.865 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.52 pp sourcePopulation 79 stderr 2.524 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_code_migration.Dttzoqk3.json) Source Vals AI Original model label anthropic/claude-sonnet-5-5 Source version 1 Observed 2026-10-21T16:46:12.712Z Evidence SHA-256 `99194597f2c76c402195be47839409046e18bd30454616da6dd40b736a088fa4` Source locator `/tasks/cli/anthropic~1claude-sonnet-5-5` | | | | | | | | | [IOI / ioi2024](https://benchize.com/benchmarks/vals-ai%3Aioi%3Aioi2024.md) | 85% | SE ±13.69 pp | 9 / 29 | $5.77 | 2,893.87source duration | [Vals AI · 2](https://www.vals.ai/_astro/benchmark_view_ioi.CuUolFNv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 85 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 5.767053 \| USD \| per_test \| Lower is better \| \| `latency` \| 2893.871 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±13.69 pp sourcePopulation 44 stderr 13.693 tokenTotals {"cache_read_tokens":50475623,"cache_write_tokens":4935394,"input_tokens":55411541,"output_tokens":1216766,"reasoning_tokens":1031007} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_ioi.CuUolFNv.json) Source Vals AI Original model label anthropic/claude-sonnet-5-5 Source version 2 Observed 2026-10-21T16:46:11.757Z Evidence SHA-256 `e1545b21b5b5c98d037aefa9d441b65859d07a10550f242d0e16162a49b8e8ab` Source locator `/tasks/ioi2024/anthropic~1claude-sonnet-5-5` | | | | | | | | | [HLAB-Vals-1.1 / criteria pass rate banking finance](https://benchize.com/benchmarks/vals-ai%3Ahlab%3Acriteria_pass_rate__banking_finance.md) | 93.1% | SE ±1.55 pp | 8 / 14 | $16.89 | 3,454.48source duration | [Vals AI · 1.1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 93.097 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 16.890636 \| USD \| per_test \| Lower is better \| \| `latency` \| 3454.478 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.55 pp sourcePopulation 24 stderr 1.551 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) Source Vals AI Original model label anthropic/claude-sonnet-5-5 Source version 1.1 Observed 2026-10-21T16:45:52.890Z Evidence SHA-256 `14681f4a2310e0f257df9ec07c9c81e5dadc8f6f71634f02e174a02a8096d868` Source locator `/tasks/criteria_pass_rate__banking_finance/anthropic~1claude-sonnet-5-5` | | | | | | | | | [EMB v1.1 / misc](https://benchize.com/benchmarks/vals-ai%3Aemb-v1-1%3Amisc.md) | 68.82% | SE ±7.35 pp | 6 / 19 | $6.8 | 1,941.55source duration | [Vals AI · 1.1](https://www.vals.ai/_astro/benchmark_view_emb_v1_1.11L06J7L.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 68.819 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 6.797643 \| USD \| per_test \| Lower is better \| \| `latency` \| 1941.547 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±7.35 pp sourcePopulation 30 stderr 7.35 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb_v1_1.11L06J7L.json) Source Vals AI Original model label anthropic/claude-sonnet-5-5 Source version 1.1 Observed 2026-10-21T16:45:55.456Z Evidence SHA-256 `eea2ef2c83a6dc7d553c7c6531c287710ea7ffbc56ee1ccec06c85b104b2408e` Source locator `/tasks/misc/anthropic~1claude-sonnet-5-5` | | | | | | | | | [Finance Agent v2.1 / overall](https://benchize.com/benchmarks/vals-ai%3Afabv2-1%3Aoverall.md) | 67.28% | SE ±2.02 pp | 3 / 18 | $6.95 | 2,066.22source duration | [Vals AI · 2.1](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2_1.ZbMhy9xo.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 67.275 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 6.950246 \| USD \| per_test \| Lower is better \| \| `latency` \| 2066.221 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.02 pp sourcePopulation 28 stderr 2.023 tokenTotals {"cache_read_tokens":579490734,"cache_write_tokens":626479999,"input_tokens":1206037903,"output_tokens":148942679,"reasoning_tokens":130899975} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2_1.ZbMhy9xo.json) Source Vals AI Original model label anthropic/claude-sonnet-5-5 Source version 2.1 Observed 2026-10-21T16:45:54.954Z Evidence SHA-256 `f1b0c2d7dca4e73b52136862a9be059f7c8481c69f15d705ce90bf3bd9f78429` Source locator `/tasks/overall/anthropic~1claude-sonnet-5-5` | | | | | | | | | [SWE-bench Multimodal / highlight js](https://benchize.com/benchmarks/vals-ai%3Aswebench-multimodal%3Ahighlight_js.md) | 51.28% | SE ±8.11 pp | 3 / 14 | — | — | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_swebench_multimodal.C7UOLrm7.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 51.282 \| percent \| reported \| Higher is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±8.11 pp sourcePopulation 20 stderr 8.108 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_swebench_multimodal.C7UOLrm7.json) Source Vals AI Original model label anthropic/claude-sonnet-5-5 Source version 1 Observed 2026-10-21T16:46:12.211Z Evidence SHA-256 `5302c42065b017d7deaa43d9e34be1ec69893b7ec4cceb6c07bfbabbd20b1f82` Source locator `/tasks/highlight_js/anthropic~1claude-sonnet-5-5` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / criteria pass rate structured finance securitization](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__structured_finance_securitization.md) | 64.13% | SE ±2.71 pp | 41 / 49 | $19.76 | 3,641.72source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 64.127 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 19.757045 \| USD \| per_test \| Lower is better \| \| `latency` \| 3641.718 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.71 pp sourcePopulation 81 stderr 2.707 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label anthropic/claude-sonnet-5-5 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__structured_finance_securitization/anthropic~1claude-sonnet-5-5` | | | | | | | | | [HLAB-Vals-1.1 / healthcare life sciences](https://benchize.com/benchmarks/vals-ai%3Ahlab%3Ahealthcare_life_sciences.md) | 20% | SE ±20 pp | 5 / 14 | $19.89 | 5,352.56source duration | [Vals AI · 1.1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 20 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 19.894952 \| USD \| per_test \| Lower is better \| \| `latency` \| 5352.558 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±20 pp sourcePopulation 24 stderr 20 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) Source Vals AI Original model label anthropic/claude-sonnet-5-5 Source version 1.1 Observed 2026-10-21T16:45:52.890Z Evidence SHA-256 `14681f4a2310e0f257df9ec07c9c81e5dadc8f6f71634f02e174a02a8096d868` Source locator `/tasks/healthcare_life_sciences/anthropic~1claude-sonnet-5-5` | | | | | | | | | [EMB / overall](https://benchize.com/benchmarks/vals-ai%3Aemb%3Aoverall.md) | 75.71% | SE ±2.44 pp | 3 / 47 | $8.79 | 2,204.99source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 75.713 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 8.792863 \| USD \| per_test \| Lower is better \| \| `latency` \| 2204.991 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.44 pp sourcePopulation 76 stderr 2.437 tokenTotals {"cache_read_tokens":2311427347,"cache_write_tokens":56507215,"input_tokens":2367951142,"output_tokens":29328538,"reasoning_tokens":22320203} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) Source Vals AI Original model label anthropic/claude-sonnet-5-5 Source version 1 Observed 2026-10-21T16:45:57.495Z Evidence SHA-256 `d807aee620ea47152a1fca0257a8dd12a1be4512c76cc1744c1ae9e285180918` Source locator `/tasks/overall/anthropic~1claude-sonnet-5-5` | | | | | | | | | [Terminal-Bench 4.0 / hardware](https://benchize.com/benchmarks/vals-ai%3Aterminal-bench-4%3Ahardware.md) | 53.33% | SE ±6.67 pp | 4 / 36 | — | — | [Vals AI · 4.0](https://www.vals.ai/_astro/benchmark_view_terminal_bench_4.CAnGyzC_.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 53.333 \| percent \| reported \| Higher is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±6.67 pp sourcePopulation 52 stderr 6.667 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_terminal_bench_4.CAnGyzC_.json) Source Vals AI Original model label anthropic/claude-sonnet-5-5 Source version 4.0 Observed 2026-10-21T16:46:10.642Z Evidence SHA-256 `98f6964580e901f1aa143db664bd92aa558415f643569492545d67c47d7f519b` Source locator `/tasks/hardware/anthropic~1claude-sonnet-5-5` | | | | | | | | | [Finance Agent (v2) / adjustments](https://benchize.com/benchmarks/vals-ai%3Afabv2%3Aadjustments.md) | 52.75% | SE ±2.12 pp | 13 / 50 | $6.3 | 1,831.95source duration | [Vals AI · 2](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 52.746 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 6.302773 \| USD \| per_test \| Lower is better \| \| `latency` \| 1831.95 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.12 pp sourcePopulation 80 stderr 2.119 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) Source Vals AI Original model label anthropic/claude-sonnet-5-5 Source version 2 Observed 2026-10-21T16:45:57.035Z Evidence SHA-256 `f731b20a26c126725d125519f2b5841aa45254656dba859a9570d7ce7e5d8191` Source locator `/tasks/adjustments/anthropic~1claude-sonnet-5-5` | | | | | | | | | [Terminal-Bench-Science / mathematical sciences](https://benchize.com/benchmarks/vals-ai%3Aterminal-bench-science%3Amathematical_sciences.md) | 58.82% | SE ±12.3 pp | 4 / 30 | — | — | [Vals AI · 0.1](https://www.vals.ai/_astro/benchmark_view_terminal_bench_science.o-HR8Nyt.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 58.824 \| percent \| reported \| Higher is better \| ### Configuration computeEffort max harness Mini-SWE-agent maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±12.3 pp sourcePopulation 41 stderr 12.304 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_terminal_bench_science.o-HR8Nyt.json) Source Vals AI Original model label anthropic/claude-sonnet-5-5 Source version 0.1 Observed 2026-10-21T16:46:03.529Z Evidence SHA-256 `a30a8618deaecf018ea1c8c9f675576cdd9ab79742d05788067db49afcc5f91b` Source locator `/tasks/mathematical_sciences/anthropic~1claude-sonnet-5-5` | | | | | | | | | [HLAB-Vals-1.1 / insurance](https://benchize.com/benchmarks/vals-ai%3Ahlab%3Ainsurance.md) | 0% | SE ±0 pp | 4 / 14 | $15.74 | 3,563.8source duration | [Vals AI · 1.1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 15.744184 \| USD \| per_test \| Lower is better \| \| `latency` \| 3563.796 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 24 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) Source Vals AI Original model label anthropic/claude-sonnet-5-5 Source version 1.1 Observed 2026-10-21T16:45:52.890Z Evidence SHA-256 `14681f4a2310e0f257df9ec07c9c81e5dadc8f6f71634f02e174a02a8096d868` Source locator `/tasks/insurance/anthropic~1claude-sonnet-5-5` | | | | | | | | | [CyberBench v1.1 / poc](https://benchize.com/benchmarks/benchmark%3Acyber%3Apoc.md) | 31.67% | SE ±6.01 pp | 21 / 28 | $2.32 | 988.93source duration | [Vals AI · 1.1](https://www.vals.ai/_astro/benchmark_view_cyberbench.DS6UOlu0.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 31.667 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 2.320498 \| USD \| per_test \| Lower is better \| \| `latency` \| 988.925 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±6.01 pp sourcePopulation 47 stderr 6.005 tokenTotals {"cache_read_tokens":409215714,"cache_write_tokens":5769851,"input_tokens":414993169,"output_tokens":2226472,"reasoning_tokens":1660391} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_cyberbench.DS6UOlu0.json) Source Vals AI Original model label anthropic/claude-sonnet-5-5 Source version 1.1 Observed 2026-10-21T16:46:15.238Z Evidence SHA-256 `943f593f929f8971a6b746269a2851b5febc2aad4e2f1066895f715033cdc7f1` Source locator `/tasks/poc/anthropic~1claude-sonnet-5-5` | | | | | | | | | [EMB / misc](https://benchize.com/benchmarks/vals-ai%3Aemb%3Amisc.md) | 69.08% | SE ±7.41 pp | 9 / 47 | $6.8 | 1,941.55source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 69.079 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 6.797643 \| USD \| per_test \| Lower is better \| \| `latency` \| 1941.547 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±7.41 pp sourcePopulation 76 stderr 7.409 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) Source Vals AI Original model label anthropic/claude-sonnet-5-5 Source version 1 Observed 2026-10-21T16:45:57.495Z Evidence SHA-256 `d807aee620ea47152a1fca0257a8dd12a1be4512c76cc1744c1ae9e285180918` Source locator `/tasks/misc/anthropic~1claude-sonnet-5-5` | | | | | | | | | [HLAB-Vals-1.1 / employment labor](https://benchize.com/benchmarks/vals-ai%3Ahlab%3Aemployment_labor.md) | 0% | SE ±0 pp | 1 / 14 | $17.26 | 4,200.75source duration | [Vals AI · 1.1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 17.260921 \| USD \| per_test \| Lower is better \| \| `latency` \| 4200.754 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 24 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) Source Vals AI Original model label anthropic/claude-sonnet-5-5 Source version 1.1 Observed 2026-10-21T16:45:52.890Z Evidence SHA-256 `14681f4a2310e0f257df9ec07c9c81e5dadc8f6f71634f02e174a02a8096d868` Source locator `/tasks/employment_labor/anthropic~1claude-sonnet-5-5` | | | | | | | | | [ProgramBench / overall](https://benchize.com/benchmarks/vals-ai%3Aprogrambench%3Aoverall.md) | 6.5% | SE ±1.75 pp | 3 / 40 | $35.86 | 7,602.08source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_programbench.C_J7ZU4X.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 6.5 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 35.860741 \| USD \| per_test \| Lower is better \| \| `latency` \| 7602.078 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.75 pp sourcePopulation 64 stderr 1.748 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_programbench.C_J7ZU4X.json) Source Vals AI Original model label anthropic/claude-sonnet-5-5 Source version 1 Observed 2026-10-21T16:46:10.137Z Evidence SHA-256 `e9bbb4378c7a058441a21d456baf9316b96c2cf5b58d371a3c7c803f5000bd8c` Source locator `/tasks/overall/anthropic~1claude-sonnet-5-5` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / corporate governance](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acorporate_governance.md) | 0% | SE ±0 pp | 11 / 49 | $16.23 | 3,464.23source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 16.225332 \| USD \| per_test \| Lower is better \| \| `latency` \| 3464.234 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 81 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label anthropic/claude-sonnet-5-5 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/corporate_governance/anthropic~1claude-sonnet-5-5` | | | | | | | | | [SAGE / PDEs](https://benchize.com/benchmarks/vals-ai%3Asage%3APDEs.md) | 41.27% | SE ±6.19 pp | 14 / 41 | $0.2 | 124.55source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_sage.RRwIIzO5.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 41.27 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.195077 \| USD \| per_test \| Lower is better \| \| `latency` \| 124.552 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±6.19 pp sourcePopulation 91 stderr 6.185 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_sage.RRwIIzO5.json) Source Vals AI Original model label anthropic/claude-sonnet-5-5 Source version 1 Observed 2026-10-21T16:46:07.515Z Evidence SHA-256 `cc5fb3fecdb0382d492ff167a42ad84b9c5c3883373c78ed87baaa972b8b6f7c` Source locator `/tasks/PDEs/anthropic~1claude-sonnet-5-5` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / immigration](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Aimmigration.md) | 0% | SE ±0 pp | 6 / 49 | $13.84 | 3,260.78source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 13.84088 \| USD \| per_test \| Lower is better \| \| `latency` \| 3260.778 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 81 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label anthropic/claude-sonnet-5-5 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/immigration/anthropic~1claude-sonnet-5-5` | | | | | | | | | [HLAB-Vals-1.1 / arbitration international dispute resolution](https://benchize.com/benchmarks/vals-ai%3Ahlab%3Aarbitration_international_dispute_resolution.md) | 0% | SE ±0 pp | 8 / 14 | $16.01 | 5,100.89source duration | [Vals AI · 1.1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 16.01159 \| USD \| per_test \| Lower is better \| \| `latency` \| 5100.89 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 24 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) Source Vals AI Original model label anthropic/claude-sonnet-5-5 Source version 1.1 Observed 2026-10-21T16:45:52.890Z Evidence SHA-256 `14681f4a2310e0f257df9ec07c9c81e5dadc8f6f71634f02e174a02a8096d868` Source locator `/tasks/arbitration_international_dispute_resolution/anthropic~1claude-sonnet-5-5` | | | | | | | | | [HLAB-Vals-1.1 / tax](https://benchize.com/benchmarks/vals-ai%3Ahlab%3Atax.md) | 0% | SE ±0 pp | 7 / 14 | $16.09 | 4,877.43source duration | [Vals AI · 1.1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 16.089025 \| USD \| per_test \| Lower is better \| \| `latency` \| 4877.428 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 24 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) Source Vals AI Original model label anthropic/claude-sonnet-5-5 Source version 1.1 Observed 2026-10-21T16:45:52.890Z Evidence SHA-256 `14681f4a2310e0f257df9ec07c9c81e5dadc8f6f71634f02e174a02a8096d868` Source locator `/tasks/tax/anthropic~1claude-sonnet-5-5` | | | | | | | | | [Terminal-Bench 4.0 / operations](https://benchize.com/benchmarks/vals-ai%3Aterminal-bench-4%3Aoperations.md) | 51.85% | SE ±7.41 pp | 2 / 36 | — | — | [Vals AI · 4.0](https://www.vals.ai/_astro/benchmark_view_terminal_bench_4.CAnGyzC_.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 51.852 \| percent \| reported \| Higher is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±7.41 pp sourcePopulation 52 stderr 7.407 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_terminal_bench_4.CAnGyzC_.json) Source Vals AI Original model label anthropic/claude-sonnet-5-5 Source version 4.0 Observed 2026-10-21T16:46:10.642Z Evidence SHA-256 `98f6964580e901f1aa143db664bd92aa558415f643569492545d67c47d7f519b` Source locator `/tasks/operations/anthropic~1claude-sonnet-5-5` | | | | | | | | | [Finance Agent (v2) / overall](https://benchize.com/benchmarks/vals-ai%3Afabv2%3Aoverall.md) | 58.1% | SE ±0.67 pp | 10 / 50 | $6.55 | 1,903.34source duration | [Vals AI · 2](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 58.103 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 6.552431 \| USD \| per_test \| Lower is better \| \| `latency` \| 1903.336 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.67 pp sourcePopulation 80 stderr 0.673 tokenTotals {"cache_read_tokens":555448124,"cache_write_tokens":566027579,"input_tokens":1121526513,"output_tokens":137870326,"reasoning_tokens":120529788} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) Source Vals AI Original model label anthropic/claude-sonnet-5-5 Source version 2 Observed 2026-10-21T16:45:57.035Z Evidence SHA-256 `f731b20a26c126725d125519f2b5841aa45254656dba859a9570d7ce7e5d8191` Source locator `/tasks/overall/anthropic~1claude-sonnet-5-5` | | | | | | | | | [EMB v1.1 / ma](https://benchize.com/benchmarks/vals-ai%3Aemb-v1-1%3Ama.md) | 82.29% | SE ±3.53 pp | 3 / 19 | $7.9 | 2,061.35source duration | [Vals AI · 1.1](https://www.vals.ai/_astro/benchmark_view_emb_v1_1.11L06J7L.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 82.294 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 7.900486 \| USD \| per_test \| Lower is better \| \| `latency` \| 2061.349 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±3.53 pp sourcePopulation 30 stderr 3.526 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb_v1_1.11L06J7L.json) Source Vals AI Original model label anthropic/claude-sonnet-5-5 Source version 1.1 Observed 2026-10-21T16:45:55.456Z Evidence SHA-256 `eea2ef2c83a6dc7d553c7c6531c287710ea7ffbc56ee1ccec06c85b104b2408e` Source locator `/tasks/ma/anthropic~1claude-sonnet-5-5` | | | | | | | | | [HLAB-Vals-1.1 / antitrust competition](https://benchize.com/benchmarks/vals-ai%3Ahlab%3Aantitrust_competition.md) | 10% | SE ±10 pp | 4 / 14 | $17.85 | 4,001.19source duration | [Vals AI · 1.1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 10 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 17.847147 \| USD \| per_test \| Lower is better \| \| `latency` \| 4001.192 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±10 pp sourcePopulation 24 stderr 10 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) Source Vals AI Original model label anthropic/claude-sonnet-5-5 Source version 1.1 Observed 2026-10-21T16:45:52.890Z Evidence SHA-256 `14681f4a2310e0f257df9ec07c9c81e5dadc8f6f71634f02e174a02a8096d868` Source locator `/tasks/antitrust_competition/anthropic~1claude-sonnet-5-5` | | | | | | | | | [CyberBench v1.1 / patch](https://benchize.com/benchmarks/benchmark%3Acyber%3Apatch.md) | 87.5% | SE ±4.42 pp | 4 / 28 | $6.51 | 3,076.33source duration | [Vals AI · 1.1](https://www.vals.ai/_astro/benchmark_view_cyberbench.DS6UOlu0.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 87.5 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 6.514246 \| USD \| per_test \| Lower is better \| \| `latency` \| 3076.325 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±4.42 pp sourcePopulation 47 stderr 4.419 tokenTotals {"cache_read_tokens":1250717995,"cache_write_tokens":15482728,"input_tokens":1266214579,"output_tokens":7588569,"reasoning_tokens":5454199} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_cyberbench.DS6UOlu0.json) Source Vals AI Original model label anthropic/claude-sonnet-5-5 Source version 1.1 Observed 2026-10-21T16:46:15.238Z Evidence SHA-256 `943f593f929f8971a6b746269a2851b5febc2aad4e2f1066895f715033cdc7f1` Source locator `/tasks/patch/anthropic~1claude-sonnet-5-5` | | | | | | | | | [Code Migration / cobol](https://benchize.com/benchmarks/vals-ai%3Acode-migration%3Acobol.md) | 70% | SE ±15.28 pp | 1 / 47 | $19.4 | 7,101.34source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_code_migration.Dttzoqk3.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 70 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 19.396217 \| USD \| per_test \| Lower is better \| \| `latency` \| 7101.336 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±15.28 pp sourcePopulation 79 stderr 15.275 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_code_migration.Dttzoqk3.json) Source Vals AI Original model label anthropic/claude-sonnet-5-5 Source version 1 Observed 2026-10-21T16:46:12.712Z Evidence SHA-256 `99194597f2c76c402195be47839409046e18bd30454616da6dd40b736a088fa4` Source locator `/tasks/cobol/anthropic~1claude-sonnet-5-5` | | | | | | | | | [HLAB-Vals-1.1 / criteria pass rate energy natural resources](https://benchize.com/benchmarks/vals-ai%3Ahlab%3Acriteria_pass_rate__energy_natural_resources.md) | 59.57% | SE ±2.82 pp | 14 / 14 | $17.61 | 5,117.25source duration | [Vals AI · 1.1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 59.571 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 17.609237 \| USD \| per_test \| Lower is better \| \| `latency` \| 5117.248 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.82 pp sourcePopulation 24 stderr 2.824 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) Source Vals AI Original model label anthropic/claude-sonnet-5-5 Source version 1.1 Observed 2026-10-21T16:45:52.890Z Evidence SHA-256 `14681f4a2310e0f257df9ec07c9c81e5dadc8f6f71634f02e174a02a8096d868` Source locator `/tasks/criteria_pass_rate__energy_natural_resources/anthropic~1claude-sonnet-5-5` | | | | | | | | | [HLAB-Vals-1.1 / criteria pass rate structured finance securitization](https://benchize.com/benchmarks/vals-ai%3Ahlab%3Acriteria_pass_rate__structured_finance_securitization.md) | 93.55% | SE ±1.4 pp | 3 / 14 | $19.76 | 4,095.53source duration | [Vals AI · 1.1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 93.548 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 19.757045 \| USD \| per_test \| Lower is better \| \| `latency` \| 4095.526 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.4 pp sourcePopulation 24 stderr 1.398 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) Source Vals AI Original model label anthropic/claude-sonnet-5-5 Source version 1.1 Observed 2026-10-21T16:45:52.890Z Evidence SHA-256 `14681f4a2310e0f257df9ec07c9c81e5dadc8f6f71634f02e174a02a8096d868` Source locator `/tasks/criteria_pass_rate__structured_finance_securitization/anthropic~1claude-sonnet-5-5` | | | | | | | | | [HLAB-Vals-1.1 / criteria pass rate emerging companies venture capital](https://benchize.com/benchmarks/vals-ai%3Ahlab%3Acriteria_pass_rate__emerging_companies_venture_capital.md) | 95.56% | SE ±1.21 pp | 3 / 14 | $16.61 | 4,124.45source duration | [Vals AI · 1.1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 95.563 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 16.606099 \| USD \| per_test \| Lower is better \| \| `latency` \| 4124.448 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.21 pp sourcePopulation 24 stderr 1.205 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) Source Vals AI Original model label anthropic/claude-sonnet-5-5 Source version 1.1 Observed 2026-10-21T16:45:52.890Z Evidence SHA-256 `14681f4a2310e0f257df9ec07c9c81e5dadc8f6f71634f02e174a02a8096d868` Source locator `/tasks/criteria_pass_rate__emerging_companies_venture_capital/anthropic~1claude-sonnet-5-5` | | | | | | | | | [SWE-bench Multimodal / bpmn js](https://benchize.com/benchmarks/vals-ai%3Aswebench-multimodal%3Abpmn_js.md) | 73.59% | SE ±6.11 pp | 3 / 14 | — | — | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_swebench_multimodal.C7UOLrm7.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 73.585 \| percent \| reported \| Higher is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±6.11 pp sourcePopulation 20 stderr 6.114 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_swebench_multimodal.C7UOLrm7.json) Source Vals AI Original model label anthropic/claude-sonnet-5-5 Source version 1 Observed 2026-10-21T16:46:12.211Z Evidence SHA-256 `5302c42065b017d7deaa43d9e34be1ec69893b7ec4cceb6c07bfbabbd20b1f82` Source locator `/tasks/bpmn_js/anthropic~1claude-sonnet-5-5` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / employment labor](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Aemployment_labor.md) | 0% | SE ±0 pp | 1 / 49 | $17.26 | 3,527.11source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 17.260921 \| USD \| per_test \| Lower is better \| \| `latency` \| 3527.114 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 81 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label anthropic/claude-sonnet-5-5 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/employment_labor/anthropic~1claude-sonnet-5-5` | | | | | | | | | [APEX-Agents](https://benchize.com/benchmarks/mercor%3Aapex-agents%3A1.1.md) | 75.5% | — | 2 / 38 | — | — | [mercor · 1.1](https://www.mercor.com/apex/apex-agents-leaderboard/) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `pass_1` \| 75.5 \| percent \| task_completion_pass_at_1 \| Higher is better \| \| `mean_score` \| 83.7 \| percent \| rubric_mean \| Higher is better \| ### Configuration datasetUrl https://huggingface.co/datasets/mercor/apex-agents-v1.1 datasetVersion 1.1 harness loop_truncated_tools_agent Reasoning effort max ### Reported statistics reportedErrors {"mean_score":3.6,"pass_1":4.7} sampleSize 955 ### Source [Original source](https://www.mercor.com/apex/apex-agents-leaderboard/) Source mercor Original model label Sonnet 5.5 Source version 1.1 Observed 2026-10-21T16:33:27.509Z Evidence SHA-256 `359c569c3b0b62ab375895eabf3c45846fa45551d419d5c9d6a4c9117db78184` Source locator `/__NEXT_DATA__/props/pageProps/leaderboardData/49` | | | | | | | | | [CyberBench v1.1 / overall](https://benchize.com/benchmarks/benchmark%3Acyber%3Aoverall.md) | 59.58% | SE ±5.21 pp | 20 / 28 | $4.42 | 2,032.63source duration | [Vals AI · 1.1](https://www.vals.ai/_astro/benchmark_view_cyberbench.DS6UOlu0.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 59.584 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 4.417372 \| USD \| per_test \| Lower is better \| \| `latency` \| 2032.625 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±5.21 pp sourcePopulation 47 stderr 5.212 tokenTotals {"cache_read_tokens":829966854,"cache_write_tokens":10626290,"input_tokens":840603874,"output_tokens":4907520,"reasoning_tokens":3557295} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_cyberbench.DS6UOlu0.json) Source Vals AI Original model label anthropic/claude-sonnet-5-5 Source version 1.1 Observed 2026-10-21T16:46:15.238Z Evidence SHA-256 `943f593f929f8971a6b746269a2851b5febc2aad4e2f1066895f715033cdc7f1` Source locator `/tasks/overall/anthropic~1claude-sonnet-5-5` | | | | | | | | | [Vals Index / terminal bench 4](https://benchize.com/benchmarks/vals-ai%3Avals_index%3Aterminal_bench_4.md) | 64.14% | SE ±1.01 pp | 2 / 14 | $16.51 | 4,928.93source duration | [Vals AI · 2.2](https://www.vals.ai/_astro/benchmark_view_vals_index.ye6C1wH2.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 64.141 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 16.513006 \| USD \| per_test \| Lower is better \| \| `latency` \| 4928.927 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.01 pp sourcePopulation 23 stderr 1.01 tokenTotals {"cache_read_tokens":3801766853,"cache_write_tokens":37747686,"input_tokens":3839535071,"output_tokens":23509474,"reasoning_tokens":17732859} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_vals_index.ye6C1wH2.json) Source Vals AI Original model label anthropic/claude-sonnet-5-5 Source version 2.2 Observed 2026-10-21T16:45:49.430Z Evidence SHA-256 `0e95aee94cd2bf34bc4b43a8f4bb16c7817de138613c21011873fea4e0786b45` Source locator `/tasks/terminal_bench_4/anthropic~1claude-sonnet-5-5` | | | | | | | | | [Terminal-Bench-Science / engineering sciences](https://benchize.com/benchmarks/vals-ai%3Aterminal-bench-science%3Aengineering_sciences.md) | 44.44% | SE ±17.57 pp | 1 / 30 | — | — | [Vals AI · 0.1](https://www.vals.ai/_astro/benchmark_view_terminal_bench_science.o-HR8Nyt.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 44.444 \| percent \| reported \| Higher is better \| ### Configuration computeEffort max harness Mini-SWE-agent maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±17.57 pp sourcePopulation 41 stderr 17.568 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_terminal_bench_science.o-HR8Nyt.json) Source Vals AI Original model label anthropic/claude-sonnet-5-5 Source version 0.1 Observed 2026-10-21T16:46:03.529Z Evidence SHA-256 `a30a8618deaecf018ea1c8c9f675576cdd9ab79742d05788067db49afcc5f91b` Source locator `/tasks/engineering_sciences/anthropic~1claude-sonnet-5-5` | | | | | | | | | [EMB v1.1 / dcf](https://benchize.com/benchmarks/vals-ai%3Aemb-v1-1%3Adcf.md) | 72.97% | SE ±6.73 pp | 1 / 19 | $9.08 | 2,289.8source duration | [Vals AI · 1.1](https://www.vals.ai/_astro/benchmark_view_emb_v1_1.11L06J7L.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 72.971 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 9.078529 \| USD \| per_test \| Lower is better \| \| `latency` \| 2289.802 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±6.73 pp sourcePopulation 30 stderr 6.726 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb_v1_1.11L06J7L.json) Source Vals AI Original model label anthropic/claude-sonnet-5-5 Source version 1.1 Observed 2026-10-21T16:45:55.456Z Evidence SHA-256 `eea2ef2c83a6dc7d553c7c6531c287710ea7ffbc56ee1ccec06c85b104b2408e` Source locator `/tasks/dcf/anthropic~1claude-sonnet-5-5` | | | | | | | | | [ClaimBench / direct support](https://benchize.com/benchmarks/vals-ai%3Aclaimbench%3Adirect_support.md) | 100% | SE ±0 pp | 1 / 5 | $0.04 | 3.92source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_claimbench.cK2bPt2E.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 100 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.044561 \| USD \| per_test \| Lower is better \| \| `latency` \| 3.916934 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 19 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_claimbench.cK2bPt2E.json) Source Vals AI Original model label anthropic/claude-sonnet-5-5 Source version 1 Observed 2026-10-21T16:45:56.452Z Evidence SHA-256 `c0b96e43a97ed7a9dbb35a06b2ff59446cf8fd70f6d50005b8c58aa860f229d8` Source locator `/tasks/direct_support/anthropic~1claude-sonnet-5-5` | | | | | | | | | [SpireClimb](https://benchize.com/benchmarks/epoch-ai%3Aspireclimb.md) | 1.33% | SE ±0.94 pp | 6 / 7 | — | — | [epoch ai · SpireClimb](https://epoch.ai/data/benchmark_data.zip#file=spireclimb.csv) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 0.013333333333333334 \| score_fraction \| Best score (across scorers) \| Higher is better \| \| `mean_score` \| 0.013333333333333334 \| score_fraction \| mean_score \| Higher is better \| ### Configuration benchmarkVersion SpireClimb reasoningBudget Not reported Reasoning effort max sourceModelVersion claude-sonnet-5-5_max ### Reported statistics Uncertainty Standard error: ±0.94 pp standardError 0.009364170421755878 ### Source [Original source](https://epoch.ai/data/benchmark_data.zip#file=spireclimb.csv) Source epoch ai Original model label claude-sonnet-5-5 Source version SpireClimb Evaluated 2026-09-29T15:20:30.511Z Observed 2026-10-21T16:30:24.199Z Evidence SHA-256 `8bb2bda1fd232b776c132133a46df08f18ceabdc5552e9d61eb5802dc85f4a5e` Source locator `/rows/1` | | | | | | | | | [IOI / ioi2026](https://benchize.com/benchmarks/vals-ai%3Aioi%3Aioi2026.md) | 75.83% | SE ±14.49 pp | 8 / 29 | $12.19 | 3,613.2source duration | [Vals AI · 2](https://www.vals.ai/_astro/benchmark_view_ioi.CuUolFNv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 75.833 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 12.192486 \| USD \| per_test \| Lower is better \| \| `latency` \| 3613.203 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±14.49 pp sourcePopulation 44 stderr 14.485 tokenTotals {"cache_read_tokens":210420244,"cache_write_tokens":4278377,"input_tokens":214699529,"output_tokens":2037311,"reasoning_tokens":1712396} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_ioi.CuUolFNv.json) Source Vals AI Original model label anthropic/claude-sonnet-5-5 Source version 2 Observed 2026-10-21T16:46:11.757Z Evidence SHA-256 `e1545b21b5b5c98d037aefa9d441b65859d07a10550f242d0e16162a49b8e8ab` Source locator `/tasks/ioi2026/anthropic~1claude-sonnet-5-5` | | | | | | | | | [HLAB-Vals-1.1 / immigration](https://benchize.com/benchmarks/vals-ai%3Ahlab%3Aimmigration.md) | 20% | SE ±20 pp | 2 / 14 | $13.84 | 3,787.05source duration | [Vals AI · 1.1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 20 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 13.84088 \| USD \| per_test \| Lower is better \| \| `latency` \| 3787.048 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±20 pp sourcePopulation 24 stderr 20 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) Source Vals AI Original model label anthropic/claude-sonnet-5-5 Source version 1.1 Observed 2026-10-21T16:45:52.890Z Evidence SHA-256 `14681f4a2310e0f257df9ec07c9c81e5dadc8f6f71634f02e174a02a8096d868` Source locator `/tasks/immigration/anthropic~1claude-sonnet-5-5` | | | | | | | | | [EMB / operating](https://benchize.com/benchmarks/vals-ai%3Aemb%3Aoperating.md) | 71.56% | SE ±7.07 pp | 9 / 47 | $7.81 | 2,013.79source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 71.555 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 7.810405 \| USD \| per_test \| Lower is better \| \| `latency` \| 2013.788 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±7.07 pp sourcePopulation 76 stderr 7.067 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) Source Vals AI Original model label anthropic/claude-sonnet-5-5 Source version 1 Observed 2026-10-21T16:45:57.495Z Evidence SHA-256 `d807aee620ea47152a1fca0257a8dd12a1be4512c76cc1744c1ae9e285180918` Source locator `/tasks/operating/anthropic~1claude-sonnet-5-5` | | | | | | | | | [EMB v1.1 / overall](https://benchize.com/benchmarks/vals-ai%3Aemb-v1-1%3Aoverall.md) | 75.9% | SE ±2.47 pp | 2 / 19 | $8.79 | 2,204.99source duration | [Vals AI · 1.1](https://www.vals.ai/_astro/benchmark_view_emb_v1_1.11L06J7L.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 75.899 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 8.792863 \| USD \| per_test \| Lower is better \| \| `latency` \| 2204.991 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.47 pp sourcePopulation 30 stderr 2.468 tokenTotals {"cache_read_tokens":2311427347,"cache_write_tokens":56507215,"input_tokens":2367951142,"output_tokens":29328538,"reasoning_tokens":22320203} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb_v1_1.11L06J7L.json) Source Vals AI Original model label anthropic/claude-sonnet-5-5 Source version 1.1 Observed 2026-10-21T16:45:55.456Z Evidence SHA-256 `eea2ef2c83a6dc7d553c7c6531c287710ea7ffbc56ee1ccec06c85b104b2408e` Source locator `/tasks/overall/anthropic~1claude-sonnet-5-5` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / real estate](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Areal_estate.md) | 0% | SE ±0 pp | 6 / 49 | $16.88 | 3,371.66source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 16.877102 \| USD \| per_test \| Lower is better \| \| `latency` \| 3371.66 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 81 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label anthropic/claude-sonnet-5-5 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/real_estate/anthropic~1claude-sonnet-5-5` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / criteria pass rate trusts estates private client](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__trusts_estates_private_client.md) | 75.57% | SE ±2.66 pp | 34 / 49 | $15.11 | 3,244.64source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 75.573 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 15.106108 \| USD \| per_test \| Lower is better \| \| `latency` \| 3244.644 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.66 pp sourcePopulation 81 stderr 2.66 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label anthropic/claude-sonnet-5-5 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__trusts_estates_private_client/anthropic~1claude-sonnet-5-5` | | | | | | | | | [HLAB-Vals-1.1 / criteria pass rate corporate governance](https://benchize.com/benchmarks/vals-ai%3Ahlab%3Acriteria_pass_rate__corporate_governance.md) | 91.44% | SE ±1.64 pp | 7 / 14 | $16.23 | 5,172.94source duration | [Vals AI · 1.1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 91.438 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 16.225332 \| USD \| per_test \| Lower is better \| \| `latency` \| 5172.942 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.64 pp sourcePopulation 24 stderr 1.64 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) Source Vals AI Original model label anthropic/claude-sonnet-5-5 Source version 1.1 Observed 2026-10-21T16:45:52.890Z Evidence SHA-256 `14681f4a2310e0f257df9ec07c9c81e5dadc8f6f71634f02e174a02a8096d868` Source locator `/tasks/criteria_pass_rate__corporate_governance/anthropic~1claude-sonnet-5-5` | | | | | | | | | [SWE-bench Multimodal / grommet](https://benchize.com/benchmarks/vals-ai%3Aswebench-multimodal%3Agrommet.md) | 30% | SE ±10.51 pp | 5 / 14 | — | — | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_swebench_multimodal.C7UOLrm7.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 30 \| percent \| reported \| Higher is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±10.51 pp sourcePopulation 20 stderr 10.513 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_swebench_multimodal.C7UOLrm7.json) Source Vals AI Original model label anthropic/claude-sonnet-5-5 Source version 1 Observed 2026-10-21T16:46:12.211Z Evidence SHA-256 `5302c42065b017d7deaa43d9e34be1ec69893b7ec4cceb6c07bfbabbd20b1f82` Source locator `/tasks/grommet/anthropic~1claude-sonnet-5-5` | | | | | | | | | [EMB / comps](https://benchize.com/benchmarks/vals-ai%3Aemb%3Acomps.md) | 80.04% | SE ±4.26 pp | 3 / 47 | $10.25 | 2,463.28source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 80.042 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 10.248719 \| USD \| per_test \| Lower is better \| \| `latency` \| 2463.278 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±4.26 pp sourcePopulation 76 stderr 4.256 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) Source Vals AI Original model label anthropic/claude-sonnet-5-5 Source version 1 Observed 2026-10-21T16:45:57.495Z Evidence SHA-256 `d807aee620ea47152a1fca0257a8dd12a1be4512c76cc1744c1ae9e285180918` Source locator `/tasks/comps/anthropic~1claude-sonnet-5-5` | | | | | | | | | [Finance Agent (v2) / general qualitative analysis](https://benchize.com/benchmarks/vals-ai%3Afabv2%3Ageneral_qualitative_analysis.md) | 67.06% | SE ±0.14 pp | 19 / 50 | $10.63 | 3,366.45source duration | [Vals AI · 2](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 67.06 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 10.63213 \| USD \| per_test \| Lower is better \| \| `latency` \| 3366.451 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.14 pp sourcePopulation 80 stderr 0.137 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) Source Vals AI Original model label anthropic/claude-sonnet-5-5 Source version 2 Observed 2026-10-21T16:45:57.035Z Evidence SHA-256 `f731b20a26c126725d125519f2b5841aa45254656dba859a9570d7ce7e5d8191` Source locator `/tasks/general_qualitative_analysis/anthropic~1claude-sonnet-5-5` | | | | | | | | | [Legal Research Bench / weighted pass civil litigation](https://benchize.com/benchmarks/vals-ai%3Alegal_research%3Aweighted_pass__civil_litigation.md) | 81.7% | SE ±6.13 pp | 36 / 50 | $16.42 | 6,231.02source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 81.699 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 16.419596 \| USD \| per_test \| Lower is better \| \| `latency` \| 6231.017 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±6.13 pp sourcePopulation 82 stderr 6.129 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_research.DiuL0R8P.json) Source Vals AI Original model label anthropic/claude-sonnet-5-5 Source version 1 Observed 2026-10-21T16:45:52.389Z Evidence SHA-256 `26c58dfd47d0d2dc86f33c6d056f30309e0ee63570e7cd752c593b385f9d862e` Source locator `/tasks/weighted_pass__civil_litigation/anthropic~1claude-sonnet-5-5` | | | | | | | | | [Vals Index / legal agent benchmark](https://benchize.com/benchmarks/vals-ai%3Avals_index%3Alegal_agent_benchmark.md) | 10.42% | SE ±2.57 pp | 8 / 14 | $16.61 | 4,397.95source duration | [Vals AI · 2.2](https://www.vals.ai/_astro/benchmark_view_vals_index.ye6C1wH2.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 10.417 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 16.612583 \| USD \| per_test \| Lower is better \| \| `latency` \| 4397.953 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.57 pp sourcePopulation 23 stderr 2.572 tokenTotals {"cache_read_tokens":7328811056,"cache_write_tokens":271040714,"input_tokens":7599893900,"output_tokens":71504693,"reasoning_tokens":47513756} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_vals_index.ye6C1wH2.json) Source Vals AI Original model label anthropic/claude-sonnet-5-5 Source version 2.2 Observed 2026-10-21T16:45:49.430Z Evidence SHA-256 `0e95aee94cd2bf34bc4b43a8f4bb16c7817de138613c21011873fea4e0786b45` Source locator `/tasks/legal_agent_benchmark/anthropic~1claude-sonnet-5-5` | | | | | | | | | [HLAB-Vals-1.1 / criteria pass rate litigation dispute resolution](https://benchize.com/benchmarks/vals-ai%3Ahlab%3Acriteria_pass_rate__litigation_dispute_resolution.md) | 75% | SE ±2.66 pp | 14 / 14 | $13.91 | 4,470.47source duration | [Vals AI · 1.1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 75 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 13.912957 \| USD \| per_test \| Lower is better \| \| `latency` \| 4470.472 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.66 pp sourcePopulation 24 stderr 2.66 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) Source Vals AI Original model label anthropic/claude-sonnet-5-5 Source version 1.1 Observed 2026-10-21T16:45:52.890Z Evidence SHA-256 `14681f4a2310e0f257df9ec07c9c81e5dadc8f6f71634f02e174a02a8096d868` Source locator `/tasks/criteria_pass_rate__litigation_dispute_resolution/anthropic~1claude-sonnet-5-5` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / capital markets](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acapital_markets.md) | 0% | SE ±0 pp | 16 / 49 | $14.68 | 2,856.81source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 14.684627 \| USD \| per_test \| Lower is better \| \| `latency` \| 2856.812 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 81 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label anthropic/claude-sonnet-5-5 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/capital_markets/anthropic~1claude-sonnet-5-5` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / data privacy cybersecurity](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Adata_privacy_cybersecurity.md) | 20% | SE ±20 pp | 4 / 49 | $16.21 | 3,526.54source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 20 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 16.205227 \| USD \| per_test \| Lower is better \| \| `latency` \| 3526.54 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±20 pp sourcePopulation 81 stderr 20 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label anthropic/claude-sonnet-5-5 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/data_privacy_cybersecurity/anthropic~1claude-sonnet-5-5` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / criteria pass rate real estate](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__real_estate.md) | 36.44% | SE ±2.6 pp | 48 / 49 | $16.88 | 3,371.66source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 36.443 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 16.877102 \| USD \| per_test \| Lower is better \| \| `latency` \| 3371.66 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.6 pp sourcePopulation 81 stderr 2.602 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label anthropic/claude-sonnet-5-5 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__real_estate/anthropic~1claude-sonnet-5-5` | | | | | | | | | [Finance Agent (v2) / market analysis](https://benchize.com/benchmarks/vals-ai%3Afabv2%3Amarket_analysis.md) | 71.44% | SE ±0.98 pp | 6 / 50 | $4.19 | 1,298.75source duration | [Vals AI · 2](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 71.435 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 4.192346 \| USD \| per_test \| Lower is better \| \| `latency` \| 1298.75 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.98 pp sourcePopulation 80 stderr 0.979 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) Source Vals AI Original model label anthropic/claude-sonnet-5-5 Source version 2 Observed 2026-10-21T16:45:57.035Z Evidence SHA-256 `f731b20a26c126725d125519f2b5841aa45254656dba859a9570d7ce7e5d8191` Source locator `/tasks/market_analysis/anthropic~1claude-sonnet-5-5` | | | | | | | | | [AutomationBench](https://benchize.com/benchmarks/zapier%3Aautomationbench.md) | 44.75% | — | 3 / 113 | $1.14 | — | [zapier](https://framerusercontent.com/sites/4WTSl4BNjd1q9QFEFibC6h/lOvLnhItj8S8-1vyjH_s33Jn-WmUXPmiJ5Diajl0Q38.CMutaHYl.mjs) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 44.75 \| percent \| reported \| Higher is better \| \| `cost` \| 1.14 \| USD \| per_task \| Lower is better \| ### Configuration benchmarkVersion 1.0.6 costFootnote false Fallback policy Provider default Reasoning effort max Model configuration Claude Sonnet 5.5 (default fallbacks, Max) thinkingEnabled true Fallback is enabled for refusals; this does not mean every task used a fallback model. ### Reported statistics rank 3 ### Source [Original source](https://framerusercontent.com/sites/4WTSl4BNjd1q9QFEFibC6h/lOvLnhItj8S8-1vyjH_s33Jn-WmUXPmiJ5Diajl0Q38.CMutaHYl.mjs) Source zapier Original model label Claude Sonnet 5.5 (default fallbacks, Max) Observed 2026-10-21T16:34:56.783Z Evidence SHA-256 `43978e38f018789fa59c11de5328abe9e5905bfa9a5a4d601fa3aa4d9c153b0d` Source locator `/rows/2` | | | | | | | | | [Tax Agent Bench / forms filings](https://benchize.com/benchmarks/vals-ai%3Atax_agent_bench%3Aforms_filings.md) | 40% | SE ±9.1 pp | 1 / 46 | $10.25 | 3,778.7source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 40 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 10.250628 \| USD \| per_test \| Lower is better \| \| `latency` \| 3778.699 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±9.1 pp sourcePopulation 73 stderr 9.097 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) Source Vals AI Original model label anthropic/claude-sonnet-5-5 Source version 1 Observed 2026-10-21T16:45:55.958Z Evidence SHA-256 `446db182b67efb7923b3187a4040ec3a3ea492b8ad699a2b6d708d714cc063ac` Source locator `/tasks/forms_filings/anthropic~1claude-sonnet-5-5` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / criteria pass rate](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate.md) | 69.95% | SE ±0.55 pp | 38 / 49 | $16.61 | 3,464.43source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 69.952 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 16.612583 \| USD \| per_test \| Lower is better \| \| `latency` \| 3464.426 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.55 pp sourcePopulation 81 stderr 0.552 tokenTotals {"cache_read_tokens":4582495471,"cache_write_tokens":168486127,"input_tokens":4750998856,"output_tokens":53560846,"reasoning_tokens":35373065} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label anthropic/claude-sonnet-5-5 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate/anthropic~1claude-sonnet-5-5` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / tax](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Atax.md) | 0% | SE ±0 pp | 3 / 49 | $16.09 | 3,703.61source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 16.089025 \| USD \| per_test \| Lower is better \| \| `latency` \| 3703.608 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 81 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label anthropic/claude-sonnet-5-5 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/tax/anthropic~1claude-sonnet-5-5` | | | | | | | | | [Mystery Game Puzzles](https://benchize.com/benchmarks/epoch-ai%3Amystery_game_puzzles.md) | 65% | SE ±4.79 pp | 4 / 71 | — | — | [epoch ai · Mystery Game Puzzles](https://epoch.ai/data/benchmark_data.zip#file=mystery_game_puzzles.csv) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 0.65 \| score_fraction \| Best score (across scorers) \| Higher is better \| \| `mean_score` \| 0.65 \| score_fraction \| mean_score \| Higher is better \| ### Configuration benchmarkVersion Mystery Game Puzzles reasoningBudget Not reported Reasoning effort max sourceModelVersion claude-sonnet-5-5_max ### Reported statistics Uncertainty Standard error: ±4.79 pp standardError 0.04793724854411022 ### Source [Original source](https://epoch.ai/data/benchmark_data.zip#file=mystery_game_puzzles.csv) Source epoch ai Original model label claude-sonnet-5-5 Source version Mystery Game Puzzles Evaluated 2026-09-29T15:15:14.000Z Observed 2026-10-21T16:30:24.199Z Evidence SHA-256 `795b977bc82c460a5e2c1d6dd7b2569ddb21853f3829c4f6fdd16edfaf47bfcf` Source locator `/rows/7` | | | | | | | | | [HLAB-Vals-1.1 / criteria pass rate healthcare life sciences](https://benchize.com/benchmarks/vals-ai%3Ahlab%3Acriteria_pass_rate__healthcare_life_sciences.md) | 94.13% | SE ±1.36 pp | 8 / 14 | $19.89 | 5,352.56source duration | [Vals AI · 1.1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 94.128 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 19.894952 \| USD \| per_test \| Lower is better \| \| `latency` \| 5352.558 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.36 pp sourcePopulation 24 stderr 1.364 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) Source Vals AI Original model label anthropic/claude-sonnet-5-5 Source version 1.1 Observed 2026-10-21T16:45:52.890Z Evidence SHA-256 `14681f4a2310e0f257df9ec07c9c81e5dadc8f6f71634f02e174a02a8096d868` Source locator `/tasks/criteria_pass_rate__healthcare_life_sciences/anthropic~1claude-sonnet-5-5` | | | | | | | | | [FrontierCode 1.1 extended](https://benchize.com/benchmarks/frontiercode%3Av1_1%3Aextended.md) | 59.09% | — | 40 / 125 | $16.29 | — | [frontiercode](https://cognition.com/data/frontiercode-leaderboard/data.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 0.5909 \| score_fraction \| blocking_criteria_adjusted \| Higher is better \| \| `correct` \| 0.6463 \| score_fraction \| raw_rubric \| Higher is better \| \| `cost` \| 16.2877 \| USD \| per_rollout \| Lower is better \| ### Configuration harness claude-code Reasoning effort max sourceVersion v1_1 subset extended ### Reported statistics flaggedRate 0.0337 sampleSize 150 ### Source [Original source](https://cognition.com/data/frontiercode-leaderboard/data.json) Source frontiercode Original model label Claude Sonnet 5.5 Observed 2026-10-21T16:31:00.878Z Evidence SHA-256 `ff0040778b7bdf69c705ad7474f771fb5e53cfaf5cf8573b3a4614153543d75f` Source locator `/v1_1/data/Claude%20Sonnet%205.5/max/extended` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / criteria pass rate employment labor](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__employment_labor.md) | 83.15% | SE ±2.3 pp | 11 / 49 | $17.26 | 3,527.11source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 83.146 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 17.260921 \| USD \| per_test \| Lower is better \| \| `latency` \| 3527.114 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.3 pp sourcePopulation 81 stderr 2.295 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label anthropic/claude-sonnet-5-5 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__employment_labor/anthropic~1claude-sonnet-5-5` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / criteria pass rate tax](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__tax.md) | 83.95% | SE ±2.13 pp | 13 / 49 | $16.09 | 3,703.61source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 83.946 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 16.089025 \| USD \| per_test \| Lower is better \| \| `latency` \| 3703.608 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.13 pp sourcePopulation 81 stderr 2.127 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label anthropic/claude-sonnet-5-5 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__tax/anthropic~1claude-sonnet-5-5` | | | | | | | | | [Terminal-Bench 4.0 / media](https://benchize.com/benchmarks/vals-ai%3Aterminal-bench-4%3Amedia.md) | 75% | SE ±0 pp | 2 / 36 | — | — | [Vals AI · 4.0](https://www.vals.ai/_astro/benchmark_view_terminal_bench_4.CAnGyzC_.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 75 \| percent \| reported \| Higher is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 52 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_terminal_bench_4.CAnGyzC_.json) Source Vals AI Original model label anthropic/claude-sonnet-5-5 Source version 4.0 Observed 2026-10-21T16:46:10.642Z Evidence SHA-256 `98f6964580e901f1aa143db664bd92aa558415f643569492545d67c47d7f519b` Source locator `/tasks/media/anthropic~1claude-sonnet-5-5` | | | | | | | | | [SWE-bench Multimodal / scratch gui](https://benchize.com/benchmarks/vals-ai%3Aswebench-multimodal%3Ascratch_gui.md) | 50% | SE ±22.36 pp | 1 / 14 | — | — | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_swebench_multimodal.C7UOLrm7.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 50 \| percent \| reported \| Higher is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±22.36 pp sourcePopulation 20 stderr 22.361 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_swebench_multimodal.C7UOLrm7.json) Source Vals AI Original model label anthropic/claude-sonnet-5-5 Source version 1 Observed 2026-10-21T16:46:12.211Z Evidence SHA-256 `5302c42065b017d7deaa43d9e34be1ec69893b7ec4cceb6c07bfbabbd20b1f82` Source locator `/tasks/scratch_gui/anthropic~1claude-sonnet-5-5` | | | | | | | | | [Finance Agent (v2) / general quantitative analysis](https://benchize.com/benchmarks/vals-ai%3Afabv2%3Ageneral_quantitative_analysis.md) | 79.44% | SE ±1.34 pp | 9 / 50 | $5.66 | 1,525.61source duration | [Vals AI · 2](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 79.44 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 5.661267 \| USD \| per_test \| Lower is better \| \| `latency` \| 1525.614 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.34 pp sourcePopulation 80 stderr 1.343 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) Source Vals AI Original model label anthropic/claude-sonnet-5-5 Source version 2 Observed 2026-10-21T16:45:57.035Z Evidence SHA-256 `f731b20a26c126725d125519f2b5841aa45254656dba859a9570d7ce7e5d8191` Source locator `/tasks/general_quantitative_analysis/anthropic~1claude-sonnet-5-5` | | | | | | | | | [HLAB-Vals-1.1 / corporate ma](https://benchize.com/benchmarks/vals-ai%3Ahlab%3Acorporate_ma.md) | 20% | SE ±20 pp | 4 / 14 | $15.02 | 4,191.2source duration | [Vals AI · 1.1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 20 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 15.021633 \| USD \| per_test \| Lower is better \| \| `latency` \| 4191.198 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±20 pp sourcePopulation 24 stderr 20 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) Source Vals AI Original model label anthropic/claude-sonnet-5-5 Source version 1.1 Observed 2026-10-21T16:45:52.890Z Evidence SHA-256 `14681f4a2310e0f257df9ec07c9c81e5dadc8f6f71634f02e174a02a8096d868` Source locator `/tasks/corporate_ma/anthropic~1claude-sonnet-5-5` | | | | | | | | | [EMB v1.1 / operating](https://benchize.com/benchmarks/vals-ai%3Aemb-v1-1%3Aoperating.md) | 71.44% | SE ±7.17 pp | 7 / 19 | $7.81 | 2,013.79source duration | [Vals AI · 1.1](https://www.vals.ai/_astro/benchmark_view_emb_v1_1.11L06J7L.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 71.435 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 7.810405 \| USD \| per_test \| Lower is better \| \| `latency` \| 2013.788 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±7.17 pp sourcePopulation 30 stderr 7.166 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb_v1_1.11L06J7L.json) Source Vals AI Original model label anthropic/claude-sonnet-5-5 Source version 1.1 Observed 2026-10-21T16:45:55.456Z Evidence SHA-256 `eea2ef2c83a6dc7d553c7c6531c287710ea7ffbc56ee1ccec06c85b104b2408e` Source locator `/tasks/operating/anthropic~1claude-sonnet-5-5` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / funds asset management](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Afunds_asset_management.md) | 0% | SE ±0 pp | 11 / 49 | $19.38 | 4,008.23source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 19.376575 \| USD \| per_test \| Lower is better \| \| `latency` \| 4008.234 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 81 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label anthropic/claude-sonnet-5-5 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/funds_asset_management/anthropic~1claude-sonnet-5-5` | | | | | | | | | [EMB / lbo](https://benchize.com/benchmarks/vals-ai%3Aemb%3Albo.md) | 69.98% | SE ±4.38 pp | 1 / 47 | $11.17 | 2,529.29source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 69.979 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 11.166488 \| USD \| per_test \| Lower is better \| \| `latency` \| 2529.291 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±4.38 pp sourcePopulation 76 stderr 4.376 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) Source Vals AI Original model label anthropic/claude-sonnet-5-5 Source version 1 Observed 2026-10-21T16:45:57.495Z Evidence SHA-256 `d807aee620ea47152a1fca0257a8dd12a1be4512c76cc1744c1ae9e285180918` Source locator `/tasks/lbo/anthropic~1claude-sonnet-5-5` | | | | | | | | | [EMB / template](https://benchize.com/benchmarks/vals-ai%3Aemb%3Atemplate.md) | 80.94% | SE ±2.62 pp | 2 / 47 | $9.01 | 2,158.23source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 80.939 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 9.010402 \| USD \| per_test \| Lower is better \| \| `latency` \| 2158.229 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.62 pp sourcePopulation 76 stderr 2.616 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_emb.BY1LXsAw.json) Source Vals AI Original model label anthropic/claude-sonnet-5-5 Source version 1 Observed 2026-10-21T16:45:57.495Z Evidence SHA-256 `d807aee620ea47152a1fca0257a8dd12a1be4512c76cc1744c1ae9e285180918` Source locator `/tasks/template/anthropic~1claude-sonnet-5-5` | | | | | | | | | [SAGE / overall](https://benchize.com/benchmarks/vals-ai%3Asage%3Aoverall.md) | 51.77% | SE ±3.41 pp | 8 / 41 | $0.18 | 113.42source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_sage.RRwIIzO5.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 51.772 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.178566 \| USD \| per_test \| Lower is better \| \| `latency` \| 113.423 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±3.41 pp sourcePopulation 91 stderr 3.412 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_sage.RRwIIzO5.json) Source Vals AI Original model label anthropic/claude-sonnet-5-5 Source version 1 Observed 2026-10-21T16:46:07.515Z Evidence SHA-256 `cc5fb3fecdb0382d492ff167a42ad84b9c5c3883373c78ed87baaa972b8b6f7c` Source locator `/tasks/overall/anthropic~1claude-sonnet-5-5` | | | | | | | | | [SAGE / calculus](https://benchize.com/benchmarks/vals-ai%3Asage%3Acalculus.md) | 78.4% | SE ±5.28 pp | 9 / 36 | $0.29 | 86.27source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_sage.RRwIIzO5.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 78.395 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.292098 \| USD \| per_test \| Lower is better \| \| `latency` \| 86.271 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±5.28 pp sourcePopulation 91 stderr 5.28 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_sage.RRwIIzO5.json) Source Vals AI Original model label anthropic/claude-sonnet-5-5 Source version 1 Observed 2026-10-21T16:46:07.515Z Evidence SHA-256 `cc5fb3fecdb0382d492ff167a42ad84b9c5c3883373c78ed87baaa972b8b6f7c` Source locator `/tasks/calculus/anthropic~1claude-sonnet-5-5` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / criteria pass rate bankruptcy restructuring](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__bankruptcy_restructuring.md) | 59.17% | SE ±2.9 pp | 40 / 49 | $15.13 | 2,967.02source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 59.17 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 15.129321 \| USD \| per_test \| Lower is better \| \| `latency` \| 2967.024 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.9 pp sourcePopulation 81 stderr 2.896 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label anthropic/claude-sonnet-5-5 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__bankruptcy_restructuring/anthropic~1claude-sonnet-5-5` | | | | | | | | | [Finance Agent v2.1 / financial modeling](https://benchize.com/benchmarks/vals-ai%3Afabv2-1%3Afinancial_modeling.md) | 48.95% | SE ±6.83 pp | 4 / 18 | $4.46 | 1,172.34source duration | [Vals AI · 2.1](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2_1.ZbMhy9xo.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 48.954 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 4.457192 \| USD \| per_test \| Lower is better \| \| `latency` \| 1172.341 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±6.83 pp sourcePopulation 28 stderr 6.834 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2_1.ZbMhy9xo.json) Source Vals AI Original model label anthropic/claude-sonnet-5-5 Source version 2.1 Observed 2026-10-21T16:45:54.954Z Evidence SHA-256 `f1b0c2d7dca4e73b52136862a9be059f7c8481c69f15d705ce90bf3bd9f78429` Source locator `/tasks/financial_modeling/anthropic~1claude-sonnet-5-5` | | | | | | | | | [Vals Index / emb](https://benchize.com/benchmarks/vals-ai%3Avals_index%3Aemb.md) | 75.9% | SE ±2.47 pp | 1 / 14 | $8.79 | 2,204.99source duration | [Vals AI · 2.2](https://www.vals.ai/_astro/benchmark_view_vals_index.ye6C1wH2.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 75.899 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 8.792863 \| USD \| per_test \| Lower is better \| \| `latency` \| 2204.991 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.47 pp sourcePopulation 23 stderr 2.468 tokenTotals {"cache_read_tokens":2311427347,"cache_write_tokens":56507215,"input_tokens":2367951142,"output_tokens":29328538,"reasoning_tokens":22320203} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_vals_index.ye6C1wH2.json) Source Vals AI Original model label anthropic/claude-sonnet-5-5 Source version 2.2 Observed 2026-10-21T16:45:49.430Z Evidence SHA-256 `0e95aee94cd2bf34bc4b43a8f4bb16c7817de138613c21011873fea4e0786b45` Source locator `/tasks/emb/anthropic~1claude-sonnet-5-5` | | | | | | | | | [Tax Agent Bench / weighted pass fact pattern analysis](https://benchize.com/benchmarks/vals-ai%3Atax_agent_bench%3Aweighted_pass__fact_pattern_analysis.md) | 73.85% | SE ±6.31 pp | 6 / 46 | $7.97 | 3,212.21source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 73.852 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 7.966557 \| USD \| per_test \| Lower is better \| \| `latency` \| 3212.209 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±6.31 pp sourcePopulation 73 stderr 6.306 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_tax_agent_bench.BFpexnpS.json) Source Vals AI Original model label anthropic/claude-sonnet-5-5 Source version 1 Observed 2026-10-21T16:45:55.958Z Evidence SHA-256 `446db182b67efb7923b3187a4040ec3a3ea492b8ad699a2b6d708d714cc063ac` Source locator `/tasks/weighted_pass__fact_pattern_analysis/anthropic~1claude-sonnet-5-5` | | | | | | | | | [Terminal-Bench 4.0 / software](https://benchize.com/benchmarks/vals-ai%3Aterminal-bench-4%3Asoftware.md) | 59.26% | SE ±1.85 pp | 2 / 36 | — | — | [Vals AI · 4.0](https://www.vals.ai/_astro/benchmark_view_terminal_bench_4.CAnGyzC_.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 59.259 \| percent \| reported \| Higher is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.85 pp sourcePopulation 52 stderr 1.852 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_terminal_bench_4.CAnGyzC_.json) Source Vals AI Original model label anthropic/claude-sonnet-5-5 Source version 4.0 Observed 2026-10-21T16:46:10.642Z Evidence SHA-256 `98f6964580e901f1aa143db664bd92aa558415f643569492545d67c47d7f519b` Source locator `/tasks/software/anthropic~1claude-sonnet-5-5` | | | | | | | | | [Terminal-Bench-Science / earth sciences](https://benchize.com/benchmarks/vals-ai%3Aterminal-bench-science%3Aearth_sciences.md) | 12.5% | SE ±12.5 pp | 11 / 30 | — | — | [Vals AI · 0.1](https://www.vals.ai/_astro/benchmark_view_terminal_bench_science.o-HR8Nyt.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 12.5 \| percent \| reported \| Higher is better \| ### Configuration computeEffort max harness Mini-SWE-agent maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±12.5 pp sourcePopulation 41 stderr 12.5 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_terminal_bench_science.o-HR8Nyt.json) Source Vals AI Original model label anthropic/claude-sonnet-5-5 Source version 0.1 Observed 2026-10-21T16:46:03.529Z Evidence SHA-256 `a30a8618deaecf018ea1c8c9f675576cdd9ab79742d05788067db49afcc5f91b` Source locator `/tasks/earth_sciences/anthropic~1claude-sonnet-5-5` | | | | | | | | | [Code Migration v1.1 / overall](https://benchize.com/benchmarks/vals-ai%3Acode-migration-v1-1%3Aoverall.md) | 86.88% | SE ±1.52 pp | 1 / 17 | $43.88 | 14,805.74source duration | [Vals AI · 1.1](https://www.vals.ai/_astro/benchmark_view_code_migration_v1_1.bJHk6HEY.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 86.875 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 43.879213 \| USD \| per_test \| Lower is better \| \| `latency` \| 14805.742 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.52 pp sourcePopulation 26 stderr 1.521 tokenTotals {"cache_read_tokens":45297461218,"cache_write_tokens":464036227,"input_tokens":45761702147,"output_tokens":167144986,"reasoning_tokens":79172021} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_code_migration_v1_1.bJHk6HEY.json) Source Vals AI Original model label anthropic/claude-sonnet-5-5 Source version 1.1 Observed 2026-10-21T16:46:09.631Z Evidence SHA-256 `352faac603c73f05cc01e89214fb4adcafa05ea7432ea71ea9903a6b5fe36b08` Source locator `/tasks/overall/anthropic~1claude-sonnet-5-5` | | | | | | | | | [MysteryMechanism / overall](https://benchize.com/benchmarks/vals-ai%3Amysterymechanism%3Aoverall.md) | 49.1% | SE ±3.36 pp | 3 / 19 | $3.45 | 1,410.24source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_mysterymechanism.75uyyp4P.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 49.099 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 3.45323 \| USD \| per_test \| Lower is better \| \| `latency` \| 1410.236 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±3.36 pp sourcePopulation 27 stderr 3.363 tokenTotals {"cache_read_tokens":1040231244,"cache_write_tokens":51657746,"input_tokens":1091907970,"output_tokens":42938841,"reasoning_tokens":35532465} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_mysterymechanism.75uyyp4P.json) Source Vals AI Original model label anthropic/claude-sonnet-5-5 Source version 1 Observed 2026-10-21T16:46:04.766Z Evidence SHA-256 `1ef09cd0a0ee8690b0c07b2272057461242359b1680e24975ea42971e03acea9` Source locator `/tasks/overall/anthropic~1claude-sonnet-5-5` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / criteria pass rate arbitration international dispute resolution](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acriteria_pass_rate__arbitration_international_dispute_resolution.md) | 36.92% | SE ±2.89 pp | 47 / 49 | $16.01 | 3,597.47source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 36.918 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 16.01159 \| USD \| per_test \| Lower is better \| \| `latency` \| 3597.47 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.89 pp sourcePopulation 81 stderr 2.894 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label anthropic/claude-sonnet-5-5 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/criteria_pass_rate__arbitration_international_dispute_resolution/anthropic~1claude-sonnet-5-5` | | | | | | | | | [Terminal-Bench 4.0 / ml](https://benchize.com/benchmarks/vals-ai%3Aterminal-bench-4%3Aml.md) | 81.82% | SE ±5.25 pp | 1 / 36 | — | — | [Vals AI · 4.0](https://www.vals.ai/_astro/benchmark_view_terminal_bench_4.CAnGyzC_.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 81.818 \| percent \| reported \| Higher is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±5.25 pp sourcePopulation 52 stderr 5.249 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_terminal_bench_4.CAnGyzC_.json) Source Vals AI Original model label anthropic/claude-sonnet-5-5 Source version 4.0 Observed 2026-10-21T16:46:10.642Z Evidence SHA-256 `98f6964580e901f1aa143db664bd92aa558415f643569492545d67c47d7f519b` Source locator `/tasks/ml/anthropic~1claude-sonnet-5-5` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / corporate ma](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Acorporate_ma.md) | 0% | SE ±0 pp | 21 / 49 | $15.02 | 3,174.42source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 15.021633 \| USD \| per_test \| Lower is better \| \| `latency` \| 3174.424 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 81 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label anthropic/claude-sonnet-5-5 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/corporate_ma/anthropic~1claude-sonnet-5-5` | | | | | | | | | [HLAB-Vals-1.1 / criteria pass rate immigration](https://benchize.com/benchmarks/vals-ai%3Ahlab%3Acriteria_pass_rate__immigration.md) | 94% | SE ±1.44 pp | 3 / 14 | $13.84 | 3,787.05source duration | [Vals AI · 1.1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 94 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 13.84088 \| USD \| per_test \| Lower is better \| \| `latency` \| 3787.048 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.44 pp sourcePopulation 24 stderr 1.435 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) Source Vals AI Original model label anthropic/claude-sonnet-5-5 Source version 1.1 Observed 2026-10-21T16:45:52.890Z Evidence SHA-256 `14681f4a2310e0f257df9ec07c9c81e5dadc8f6f71634f02e174a02a8096d868` Source locator `/tasks/criteria_pass_rate__immigration/anthropic~1claude-sonnet-5-5` | | | | | | | | | [HLAB-Vals-1.1 / criteria pass rate corporate ma](https://benchize.com/benchmarks/vals-ai%3Ahlab%3Acriteria_pass_rate__corporate_ma.md) | 95.66% | SE ±1.16 pp | 8 / 14 | $15.02 | 4,191.2source duration | [Vals AI · 1.1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 95.659 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 15.021633 \| USD \| per_test \| Lower is better \| \| `latency` \| 4191.198 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.16 pp sourcePopulation 24 stderr 1.157 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) Source Vals AI Original model label anthropic/claude-sonnet-5-5 Source version 1.1 Observed 2026-10-21T16:45:52.890Z Evidence SHA-256 `14681f4a2310e0f257df9ec07c9c81e5dadc8f6f71634f02e174a02a8096d868` Source locator `/tasks/criteria_pass_rate__corporate_ma/anthropic~1claude-sonnet-5-5` | | | | | | | | | [Harvey's Legal Agent Benchmark v1 / bankruptcy restructuring](https://benchize.com/benchmarks/vals-ai%3Ahlab-v1%3Abankruptcy_restructuring.md) | 0% | SE ±0 pp | 3 / 49 | $15.13 | 2,967.02source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 0 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 15.129321 \| USD \| per_test \| Lower is better \| \| `latency` \| 2967.024 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0 pp sourcePopulation 81 stderr 0 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark_v1.ZWRzZKcv.json) Source Vals AI Original model label anthropic/claude-sonnet-5-5 Source version 1 Observed 2026-10-21T16:45:53.909Z Evidence SHA-256 `65e7cd51e83066a42c9d782635dba5f2e9e1b189bd8118d3510cd17d2ddcba3f` Source locator `/tasks/bankruptcy_restructuring/anthropic~1claude-sonnet-5-5` | | | | | | | | | [Code Migration / code quality](https://benchize.com/benchmarks/vals-ai%3Acode-migration%3Acode_quality.md) | 4.6% | SE ±0.03 pp | 5 / 47 | $75.83 | 12,004.48source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_code_migration.Dttzoqk3.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 4.601 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 75.826721 \| USD \| per_test \| Lower is better \| \| `latency` \| 12004.483 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±0.03 pp sourcePopulation 79 stderr 0.034 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_code_migration.Dttzoqk3.json) Source Vals AI Original model label anthropic/claude-sonnet-5-5 Source version 1 Observed 2026-10-21T16:46:12.712Z Evidence SHA-256 `99194597f2c76c402195be47839409046e18bd30454616da6dd40b736a088fa4` Source locator `/tasks/code_quality/anthropic~1claude-sonnet-5-5` | | | | | | | | | [Finance Agent (v2) / financial modeling](https://benchize.com/benchmarks/vals-ai%3Afabv2%3Afinancial_modeling.md) | 25.51% | SE ±1.11 pp | 11 / 50 | $4.34 | 1,145.39source duration | [Vals AI · 2](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 25.507 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 4.340797 \| USD \| per_test \| Lower is better \| \| `latency` \| 1145.389 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.11 pp sourcePopulation 80 stderr 1.111 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2.DypCOnwd.json) Source Vals AI Original model label anthropic/claude-sonnet-5-5 Source version 2 Observed 2026-10-21T16:45:57.035Z Evidence SHA-256 `f731b20a26c126725d125519f2b5841aa45254656dba859a9570d7ce7e5d8191` Source locator `/tasks/financial_modeling/anthropic~1claude-sonnet-5-5` | | | | | | | | | [HLAB-Vals-1.1 / criteria pass rate capital markets](https://benchize.com/benchmarks/vals-ai%3Ahlab%3Acriteria_pass_rate__capital_markets.md) | 83.53% | SE ±2.33 pp | 14 / 14 | $14.68 | 3,458.52source duration | [Vals AI · 1.1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 83.529 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 14.684627 \| USD \| per_test \| Lower is better \| \| `latency` \| 3458.518 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.33 pp sourcePopulation 24 stderr 2.327 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) Source Vals AI Original model label anthropic/claude-sonnet-5-5 Source version 1.1 Observed 2026-10-21T16:45:52.890Z Evidence SHA-256 `14681f4a2310e0f257df9ec07c9c81e5dadc8f6f71634f02e174a02a8096d868` Source locator `/tasks/criteria_pass_rate__capital_markets/anthropic~1claude-sonnet-5-5` | | | | | | | | | [SWE-bench Multimodal / eslint](https://benchize.com/benchmarks/vals-ai%3Aswebench-multimodal%3Aeslint.md) | 81.82% | SE ±12.2 pp | 2 / 14 | — | — | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_swebench_multimodal.C7UOLrm7.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 81.818 \| percent \| reported \| Higher is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±12.2 pp sourcePopulation 20 stderr 12.197 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_swebench_multimodal.C7UOLrm7.json) Source Vals AI Original model label anthropic/claude-sonnet-5-5 Source version 1 Observed 2026-10-21T16:46:12.211Z Evidence SHA-256 `5302c42065b017d7deaa43d9e34be1ec69893b7ec4cceb6c07bfbabbd20b1f82` Source locator `/tasks/eslint/anthropic~1claude-sonnet-5-5` | | | | | | | | | [HLAB-Vals-1.1 / criteria pass rate real estate](https://benchize.com/benchmarks/vals-ai%3Ahlab%3Acriteria_pass_rate__real_estate.md) | 88.05% | SE ±1.75 pp | 14 / 14 | $16.88 | 5,189.84source duration | [Vals AI · 1.1](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 88.047 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 16.877102 \| USD \| per_test \| Lower is better \| \| `latency` \| 5189.844 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±1.75 pp sourcePopulation 24 stderr 1.754 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_legal_agent_benchmark.CFXFrRQl.json) Source Vals AI Original model label anthropic/claude-sonnet-5-5 Source version 1.1 Observed 2026-10-21T16:45:52.890Z Evidence SHA-256 `14681f4a2310e0f257df9ec07c9c81e5dadc8f6f71634f02e174a02a8096d868` Source locator `/tasks/criteria_pass_rate__real_estate/anthropic~1claude-sonnet-5-5` | | | | | | | | | [Finance Agent v2.1 / earnings analysis](https://benchize.com/benchmarks/vals-ai%3Afabv2-1%3Aearnings_analysis.md) | 86.5% | SE ±4.7 pp | 7 / 18 | $2.74 | 1,103.74source duration | [Vals AI · 2.1](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2_1.ZbMhy9xo.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 86.503 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 2.739073 \| USD \| per_test \| Lower is better \| \| `latency` \| 1103.738 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±4.7 pp sourcePopulation 28 stderr 4.702 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_finance_agent_v2_1.ZbMhy9xo.json) Source Vals AI Original model label anthropic/claude-sonnet-5-5 Source version 2.1 Observed 2026-10-21T16:45:54.954Z Evidence SHA-256 `f1b0c2d7dca4e73b52136862a9be059f7c8481c69f15d705ce90bf3bd9f78429` Source locator `/tasks/earnings_analysis/anthropic~1claude-sonnet-5-5` | | | | | | | | | [ProgramBench / partial](https://benchize.com/benchmarks/vals-ai%3Aprogrambench%3Apartial.md) | 76.3% | SE ±2.42 pp | 11 / 40 | $35.86 | 7,602.08source duration | [Vals AI · 1](https://www.vals.ai/_astro/benchmark_view_programbench.C_J7ZU4X.json) | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 76.304 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 35.860741 \| USD \| per_test \| Lower is better \| \| `latency` \| 7602.078 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.42 pp sourcePopulation 64 stderr 2.42 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_programbench.C_J7ZU4X.json) Source Vals AI Original model label anthropic/claude-sonnet-5-5 Source version 1 Observed 2026-10-21T16:46:10.137Z Evidence SHA-256 `e9bbb4378c7a058441a21d456baf9316b96c2cf5b58d371a3c7c803f5000bd8c` Source locator `/tasks/partial/anthropic~1claude-sonnet-5-5` | | | | | | | | [Next page](https://benchize.com/models/claude-sonnet-5-5-max.md?cursor=65e010f4f0c5884663af6264d78a50fabe326669805af460eacf0e663a842749) --- Cite as: Claude Sonnet 5.5Max — Benchize, https://benchize.com/models/claude-sonnet-5-5-max (retrieved 2026-10-21)