# Claude Sonnet 4

> Source: Benchize (https://benchize.com) — https://benchize.com/models/claude-sonnet-4. When citing or reusing this content, credit "Benchize" and link to https://benchize.com/models/claude-sonnet-4.

Anthropic

Canonical URL: https://benchize.com/models/claude-sonnet-4
Last updated: Thu, 22 Oct 2026 07:32:33 GMT

Reproduction and quotation must name Benchize, link to the canonical HTML URL, and retain individual source citations.

Anthropic

## Benchmarks

| Benchmark | Score | ± | Rank | Cost | Source |  |
| --- | --- | --- | --- | --- | --- | --- |
| [SWE-bench Verified](https://benchize.com/benchmarks/swe-bench%3Averified.md) | 74.8% | — | 3 / 63 | — | [SWE-bench](https://www.swebench.com/) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `resolved` \| 74.8 \| percent \| verified \| Higher is better \| ### Configuration agent Harness AI agentOrganization Harness models ["claude-sonnet-4-20250514"] Reasoning effort Not reported sourceSubmission 20250731_harness_ai subset Verified system ["System: Attempts - 1"] ### Source [Original source](https://www.swebench.com/) Source SWE-bench Original model label Claude Sonnet 4 Observed 2026-10-22T02:30:05.773Z Evidence SHA-256 `c862011a4ee1d1a7199fad6ca0905fa5c64f58216f7217ac1eddf37f28788433` Source locator `/leaderboard-data/2/results/13` |  |  |  |  |  |  |
| [SWE-bench Multimodal](https://benchize.com/benchmarks/swe-bench%3Amultimodal.md) | 34.43% | — | 3 / 14 | — | [SWE-bench](https://www.swebench.com/) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `resolved` \| 34.43 \| percent \| multimodal \| Higher is better \| ### Configuration agent OpenHands-Versa agentOrganization OpenHands models ["claude-4-sonnet-20250514"] Reasoning effort Not reported sourceSubmission 20250528_OpenHands-Versa-claude4 subset Multimodal system ["System: Attempts - 1"] ### Source [Original source](https://www.swebench.com/) Source SWE-bench Original model label Claude Sonnet 4 Observed 2026-10-22T02:30:05.773Z Evidence SHA-256 `c862011a4ee1d1a7199fad6ca0905fa5c64f58216f7217ac1eddf37f28788433` Source locator `/leaderboard-data/4/results/3` |  |  |  |  |  |  |
| [Humanity's Last Exam (Text Only)](https://benchize.com/benchmarks/scale%3Ahumanitys_last_exam_text_only.md) | 5.42source score | CI 4.46–6.38 | 21 / 25 | — | [scale](https://labs.scale.com/leaderboard/humanitys_last_exam_text_only) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 5.42 \| source_score \| reported \| Higher is better \| ### Configuration Reasoning effort unknown Model configuration Claude Sonnet 4 sourceModelVersion Not reported thinkingEnabled Not reported ### Reported statistics Uncertainty Confidence interval: 4.46–6.38 contaminationWarning Potential contamination warning: This model was evaluated after the public release of HLE, allowing model builder access to the prompts and solutions. Sampled at temperature 0.3. rank 54 reportedConfidenceHalfWidth 0.96 ### Source [Original source](https://labs.scale.com/leaderboard/humanitys_last_exam_text_only) Source scale Original model label Claude Sonnet 4 Observed 2026-10-22T03:30:09.372Z Evidence SHA-256 `16bd91dbd41a73afd31ff09fba3b9b08e9e3e2cfe682c50cc3b2bf8397610af6` Source locator `/rows/53` |  |  |  |  |  |  |
| [ARC-AGI-2](https://benchize.com/benchmarks/arc-prize%3Av2_Semi_Private.md) | 1.27% | — | 211 / 243 | 0.13usd per task | [arc prize · v2](https://arcprize.org/media/data/leaderboard/v2.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 0.0127 \| score_fraction \| v2_Semi_Private \| Higher is better \| \| `cost_per_task` \| 0.1272 \| usd_per_task \| v2_Semi_Private \| Lower is better \| ### Configuration evaluationSplit v2_Semi_Private harness Not reported modelGroup Not reported modelType Base LLM sourceModelVersion claude-sonnet-4-20250514 ### Source [Original source](https://arcprize.org/media/data/leaderboard/v2.json) Source arc prize Original model label Claude Sonnet 4 Source version v2 Observed 2026-10-22T05:20:04.306Z Evidence SHA-256 `4351eb696e78113b0459e9bb5f0258318b905efef34f473dcea51ce641b0d94d` Source locator `/evaluations/36` |  |  |  |  |  |  |
| [SWE-bench Verified](https://benchize.com/benchmarks/swe-bench%3Averified.md) | 57% | — | 31 / 63 | — | [SWE-bench](https://www.swebench.com/) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `resolved` \| 57 \| percent \| verified \| Higher is better \| ### Configuration agent Artemis Agent v2 agentOrganization Turintech models ["claude-sonnet-4-20250514"] Reasoning effort Not reported sourceSubmission 20250924_artemis_agent_v2 subset Verified system ["System: Attempts - 1"] ### Source [Original source](https://www.swebench.com/) Source SWE-bench Original model label Claude Sonnet 4 Observed 2026-10-22T02:30:05.773Z Evidence SHA-256 `c862011a4ee1d1a7199fad6ca0905fa5c64f58216f7217ac1eddf37f28788433` Source locator `/leaderboard-data/2/results/85` |  |  |  |  |  |  |
| [Spiral-Bench v1.0](https://benchize.com/benchmarks/eq-bench%3Aspiral-bench-v1.0.md) | 41.1% | CI 41.1–41.1% | 10 / 11 | — | [eq bench · spiral-bench-v1.0](https://eqbench.com/spiral-bench_v1.0.js) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score_norm` \| 41.1 \| percent \| overall \| Higher is better \| ### Configuration interactiveAdjustments none protocol spiral-bench-v1.0 sourceModelVersion claude-sonnet-4 ### Reported statistics Uncertainty Confidence interval: 41.1–41.1% confidenceIntervalHigh 41.1 confidenceIntervalLow 41.1 ### Source [Original source](https://eqbench.com/spiral-bench_v1.0.js) Source eq bench Original model label claude-sonnet-4 Source version spiral-bench-v1.0 Observed 2026-10-22T05:10:07.395Z Evidence SHA-256 `d939ed9ba6d42226765fb4acf3e440127472566e1b5c2894d4ca25baa5ae75d2` Source locator `/leaderboardDataDelusion/2` |  |  |  |  |  |  |
| [Creative Writing v3](https://benchize.com/benchmarks/eq-bench%3Acreative-writing-v3.md) | 1,482.8elo | — | 53 / 90 | — | [eq bench · creative-writing-v3](https://eqbench.com/creative_writing.js) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `elo_score` \| 1482.8 \| elo \| overall \| Higher is better \| \| `creative_writing_score` \| 15.77 \| rubric_score \| overall \| Higher is better \| ### Configuration interactiveAdjustments none protocol creative-writing-v3 sourceModelVersion claude-sonnet-4 ### Source [Original source](https://eqbench.com/creative_writing.js) Source eq bench Original model label claude-sonnet-4 Source version creative-writing-v3 Observed 2026-10-22T05:10:05.433Z Evidence SHA-256 `722ddb132e8d3a5e5c39b8d445f8475f3f7e24893a007260c6f3e3cce5c1ee18` Source locator `/leaderboardDataCreativeWritingV3/66` |  |  |  |  |  |  |
| [DesignArena Website generation](https://benchize.com/benchmarks/designarena%3Awebsite.md) | 1,146elo | — | 86 / 123 | — | [designarena · public](https://www.designarena.ai/api/leaderboard#board=website) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `elo` \| 1146 \| elo \| website \| Higher is better \| ### Configuration arenaType models category website inputModality text sourceModelVersion claude-sonnet-4 variationName public ### Reported statistics averageGenerationTimeMs 74782 battles 13465 losses 6406 standardError Not reported winRate 52.4 wins 7059 ### Source [Original source](https://www.designarena.ai/api/leaderboard#board=website) Source designarena Original model label Claude Sonnet 4 Source version public Observed 2026-10-21T16:37:57.757Z Evidence SHA-256 `51873e97776a3c5ae706b24cc2c60384793dd18398df970e213a8e7c276d8ac0` Source locator `/data/124` |  |  |  |  |  |  |
| [EQ-Bench 3](https://benchize.com/benchmarks/eq-bench%3Aeqbench3.md) | 1,242.9elo | CI 1,215.2–1,270.5 | 28 / 50 | — | [eq bench · eqbench3](https://eqbench.com/eqbench3.js) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `elo_norm` \| 1242.9 \| elo \| overall \| Higher is better \| \| `rubric_0_100` \| 66.6 \| percent \| overall \| Higher is better \| ### Configuration interactiveAdjustments none protocol eqbench3 sourceModelVersion claude-sonnet-4 ### Reported statistics Uncertainty Confidence interval: 1,215.2–1,270.5 confidenceIntervalHigh 1270.5 confidenceIntervalLow 1215.2 ### Source [Original source](https://eqbench.com/eqbench3.js) Source eq bench Original model label claude-sonnet-4 Source version eqbench3 Observed 2026-10-22T05:10:04.922Z Evidence SHA-256 `3672ff093e739f07ff664a2e6013bc38a6977a146e4092b4cef80ba1780542b2` Source locator `/leaderboardDataEQBench3/38` |  |  |  |  |  |  |
| [NC Bench](https://benchize.com/benchmarks/nc-bench%3Aoverall.md) | 87.64% | — | 78 / 144 | — | [nc bench](https://www.nc-bench.com/models) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 87.64 \| percent \| reported \| Higher is better \| \| `tooling` \| 100 \| percent \| category \| Higher is better \| \| `creative_writing` \| 79.21 \| percent \| category \| Higher is better \| \| `language` \| 91.31 \| percent \| category \| Higher is better \| \| `utility` \| 84.02 \| percent \| category \| Higher is better \| \| `reasoning` \| 75.86 \| percent \| category \| Higher is better \| \| `text_editing` \| 99.13 \| percent \| category \| Higher is better \| \| `rule_following` \| 81.52 \| percent \| category \| Higher is better \| \| `hallucination` \| 90.04 \| percent \| category \| Higher is better \| ### Configuration Reasoning effort unknown Model configuration Claude Sonnet 4 thinkingEnabled Not reported ### Source [Original source](https://www.nc-bench.com/models) Source nc bench Original model label Claude Sonnet 4 Observed 2026-10-22T06:10:03.426Z Evidence SHA-256 `ec261022458543624b08a136d01f5a2f0666cd52ffb49c64d6ca983e8543d1b5` Source locator `/rows/102` |  |  |  |  |  |  |
| [τ-bench retail (legacy)](https://benchize.com/benchmarks/sierra%3Alegacy%3Aretail.md) | 80.5% | — | 7 / 14 | — | [sierra](https://sierra-tau-bench-public.s3.us-west-2.amazonaws.com/submissions/claude-sonnet-4_anthropic_2025-05-22/submission.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `pass_1` \| 80.5 \| percent \| consistent_success_across_k_trials \| Higher is better \| ### Configuration benchmarkVersion original tau-bench harness tau-bench methodology {"evaluation_date":"2025-05-22","notes":"Scores were achieved with a prompt addendum to both the Airline and Retail Agent Policy instructing Claude to better leverage its reasoning abilities while using extended thinking with tool use. The model is encouraged to write down its thoughts as it solves the problem distinct from our usual thinking mode, during the multi-turn trajectories to best leverage its reasoning abilities. To accommodate the additional steps Claude incurs by utilizing more thinking, the maximum number of steps (counted by model completions) was increased from 30 to 100 (most trajectories completed under 30 steps with only one trajectory reaching above 50 steps).","tau2_bench_version":"original tau-bench","user_simulator":null,"verification":{"details":"Unverified submission - no trajectory data available, unknown evaluation methodology, telecom domain omitted, incomplete evaluation (Pass^1 only)","modified_prompts":null,"omitted_questions":null}} modality text Reasoning effort unknown retrieval Not reported sourceSubmission claude-sonnet-4_anthropic_2025-05-22 ### Source [Original source](https://sierra-tau-bench-public.s3.us-west-2.amazonaws.com/submissions/claude-sonnet-4_anthropic_2025-05-22/submission.json) Source sierra Original model label Claude Sonnet 4 Evaluated 2025-05-22 Observed 2026-10-21T16:55:01.001Z Evidence SHA-256 `c87bd245d5b2c2ac6494b650c1804239f742fb49524d0e5f17a1dab415220b31` Source locator `/results/retail` |  |  |  |  |  |  |
| [Spiral-Bench v1.2](https://benchize.com/benchmarks/eq-bench%3Aspiral-bench-v1.2.md) | 32.1% | CI 32.1–32.1% | 10 / 16 | — | [eq bench · spiral-bench-v1.2](https://eqbench.com/spiral-bench.js) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score_norm` \| 32.1 \| percent \| overall \| Higher is better \| ### Configuration interactiveAdjustments none protocol spiral-bench-v1.2 sourceModelVersion claude-sonnet-4 ### Reported statistics Uncertainty Confidence interval: 32.1–32.1% confidenceIntervalHigh 32.1 confidenceIntervalLow 32.1 ### Source [Original source](https://eqbench.com/spiral-bench.js) Source eq bench Original model label claude-sonnet-4 Source version spiral-bench-v1.2 Observed 2026-10-22T05:10:07.179Z Evidence SHA-256 `2f0402654b01e7363c0d02bbc708afd7849e669dc5da63d9f85c995ed8f0a97f` Source locator `/leaderboardDataDelusion/3` |  |  |  |  |  |  |
| [DesignArena UI component generation](https://benchize.com/benchmarks/designarena%3Aui-components.md) | 1,129elo | — | 80 / 112 | — | [designarena · public](https://www.designarena.ai/api/leaderboard#board=ui-components) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `elo` \| 1129 \| elo \| ui-components \| Higher is better \| ### Configuration arenaType models category uicomponent inputModality Not reported sourceModelVersion claude-sonnet-4 variationName public ### Reported statistics averageGenerationTimeMs 59552 battles 598 losses 252 standardError Not reported winRate 57.9 wins 346 ### Source [Original source](https://www.designarena.ai/api/leaderboard#board=ui-components) Source designarena Original model label Claude Sonnet 4 Source version public Observed 2026-10-21T16:37:57.993Z Evidence SHA-256 `46482ec1b4012638eace65493daea4750e2088098018fd9d5fc46126fc1f8557` Source locator `/data/118` |  |  |  |  |  |  |
| [DesignArena 3D design generation](https://benchize.com/benchmarks/designarena%3A3d-design.md) | 1,139elo | — | 55 / 110 | — | [designarena · public](https://www.designarena.ai/api/leaderboard#board=3d-design) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `elo` \| 1139 \| elo \| 3d-design \| Higher is better \| ### Configuration arenaType models category 3d inputModality Not reported sourceModelVersion claude-sonnet-4 variationName public ### Reported statistics averageGenerationTimeMs 57012 battles 1574 losses 665 standardError Not reported winRate 57.8 wins 909 ### Source [Original source](https://www.designarena.ai/api/leaderboard#board=3d-design) Source designarena Original model label Claude Sonnet 4 Source version public Observed 2026-10-21T16:37:58.515Z Evidence SHA-256 `bcc0267f7bc7e9ec112306b3a35413cf78bbccbfaa838356232d5d5ed0b83d0f` Source locator `/data/88` |  |  |  |  |  |  |
| [Humanity’s Last Exam (HLE)](https://benchize.com/benchmarks/hle.md) | 5.52source score | CI 4.62–6.42 | 14 / 17 | — | [scale](https://labs.scale.com/leaderboard/humanitys_last_exam) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 5.52 \| source_score \| reported \| Higher is better \| ### Configuration Reasoning effort unknown Model configuration Claude Sonnet 4 sourceModelVersion Not reported thinkingEnabled Not reported ### Reported statistics Uncertainty Confidence interval: 4.62–6.42 contaminationWarning Potential contamination warning: This model was evaluated after the public release of HLE, allowing model builder access to the prompts and solutions. rank 47 reportedConfidenceHalfWidth 0.9 ### Source [Original source](https://labs.scale.com/leaderboard/humanitys_last_exam) Source scale Original model label Claude Sonnet 4 Observed 2026-10-22T03:30:08.991Z Evidence SHA-256 `2e7bc7983b3fe2fab5270a4dc00f51a40b87f5318a686fbae4291beda672f604` Source locator `/rows/46` |  |  |  |  |  |  |
| [VISTA](https://benchize.com/benchmarks/scale%3Avisual_language_understanding.md) | 43.21source score | CI 42.69–43.73 | 8 / 16 | — | [scale](https://labs.scale.com/leaderboard/visual_language_understanding) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 43.21 \| source_score \| reported \| Higher is better \| ### Configuration Reasoning effort unknown Model configuration Claude Sonnet 4 sourceModelVersion Not reported thinkingEnabled Not reported ### Reported statistics Uncertainty Confidence interval: 42.69–43.73 contaminationWarning Not reported rank 37 reportedConfidenceHalfWidth 0.52 ### Source [Original source](https://labs.scale.com/leaderboard/visual_language_understanding) Source scale Original model label Claude Sonnet 4 Observed 2026-10-22T03:30:13.295Z Evidence SHA-256 `d373f192d42603b3d2b7b0b509c801be37a4fd65b7c73742f980d9f2f0a4e668` Source locator `/rows/36` |  |  |  |  |  |  |
| [DesignArena SVG generation](https://benchize.com/benchmarks/designarena%3Asvgs.md) | 1,063elo | — | 62 / 77 | — | [designarena · public](https://www.designarena.ai/api/leaderboard#board=svgs) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `elo` \| 1063 \| elo \| svgs \| Higher is better \| ### Configuration arenaType models category svg inputModality Not reported sourceModelVersion claude-sonnet-4 variationName public ### Reported statistics averageGenerationTimeMs 17004 battles 1724 losses 900 standardError Not reported winRate 47.8 wins 824 ### Source [Original source](https://www.designarena.ai/api/leaderboard#board=svgs) Source designarena Original model label Claude Sonnet 4 Source version public Observed 2026-10-21T16:38:03.249Z Evidence SHA-256 `8cbc34a474a66ef5c64f3ffe557df9b55bf7e13368c07c0214223e3ed8f3f21e` Source locator `/data/100` |  |  |  |  |  |  |
| [Fortress](https://benchize.com/benchmarks/scale%3Afortress.md) | 24.37source score | CI 21.3–27.44 | 17 / 31 | — | [scale](https://labs.scale.com/leaderboard/fortress) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 24.37 \| source_score \| reported \| Higher is better \| ### Configuration Reasoning effort unknown Model configuration Claude Sonnet 4 sourceModelVersion Not reported thinkingEnabled Not reported ### Reported statistics Uncertainty Confidence interval: 21.3–27.44 contaminationWarning ORS score: 5.04 rank 36 reportedConfidenceHalfWidth 3.07 ### Source [Original source](https://labs.scale.com/leaderboard/fortress) Source scale Original model label Claude Sonnet 4 Observed 2026-10-22T03:30:12.378Z Evidence SHA-256 `2c005102a7346206e5d31d9249d525620bef39f6d514668ed48398e710616b0f` Source locator `/rows/35` |  |  |  |  |  |  |
| [Longform Creative Writing](https://benchize.com/benchmarks/eq-bench%3Alongform-writing.md) | 58.4% | — | 57 / 93 | — | [eq bench · longform-writing](https://eqbench.com/creative_writing_longform.js) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `overall_score_100` \| 58.4 \| percent \| overall \| Higher is better \| \| `chapter1_avg` \| 14.29 \| rubric_score \| overall \| Higher is better \| \| `chapter2_avg` \| 13.42 \| rubric_score \| overall \| Higher is better \| \| `chapter3_avg` \| 13.41 \| rubric_score \| overall \| Higher is better \| \| `chapter4_avg` \| 11.91 \| rubric_score \| overall \| Higher is better \| \| `chapter5_avg` \| 12.02 \| rubric_score \| overall \| Higher is better \| \| `chapter6_avg` \| 11.66 \| rubric_score \| overall \| Higher is better \| \| `chapter7_avg` \| 10.18 \| rubric_score \| overall \| Higher is better \| \| `chapter8_avg` \| 9.83 \| rubric_score \| overall \| Higher is better \| \| `final_judgement_avg` \| 10.77 \| rubric_score \| overall \| Higher is better \| ### Configuration interactiveAdjustments none protocol longform-writing sourceModelVersion claude-sonnet-4 ### Source [Original source](https://eqbench.com/creative_writing_longform.js) Source eq bench Original model label claude-sonnet-4 Source version longform-writing Observed 2026-10-22T05:10:05.903Z Evidence SHA-256 `3ba283ce357ac7620e16db640a961879c30fe3ddc50a27de9a8ed4290784825c` Source locator `/leaderboardDataLongformV3/31` |  |  |  |  |  |  |
| [τ-bench airline (legacy)](https://benchize.com/benchmarks/sierra%3Alegacy%3Aairline.md) | 60% | — | 7 / 14 | — | [sierra](https://sierra-tau-bench-public.s3.us-west-2.amazonaws.com/submissions/claude-sonnet-4_anthropic_2025-05-22/submission.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `pass_1` \| 60 \| percent \| consistent_success_across_k_trials \| Higher is better \| ### Configuration benchmarkVersion original tau-bench harness tau-bench methodology {"evaluation_date":"2025-05-22","notes":"Scores were achieved with a prompt addendum to both the Airline and Retail Agent Policy instructing Claude to better leverage its reasoning abilities while using extended thinking with tool use. The model is encouraged to write down its thoughts as it solves the problem distinct from our usual thinking mode, during the multi-turn trajectories to best leverage its reasoning abilities. To accommodate the additional steps Claude incurs by utilizing more thinking, the maximum number of steps (counted by model completions) was increased from 30 to 100 (most trajectories completed under 30 steps with only one trajectory reaching above 50 steps).","tau2_bench_version":"original tau-bench","user_simulator":null,"verification":{"details":"Unverified submission - no trajectory data available, unknown evaluation methodology, telecom domain omitted, incomplete evaluation (Pass^1 only)","modified_prompts":null,"omitted_questions":null}} modality text Reasoning effort unknown retrieval Not reported sourceSubmission claude-sonnet-4_anthropic_2025-05-22 ### Source [Original source](https://sierra-tau-bench-public.s3.us-west-2.amazonaws.com/submissions/claude-sonnet-4_anthropic_2025-05-22/submission.json) Source sierra Original model label Claude Sonnet 4 Evaluated 2025-05-22 Observed 2026-10-21T16:55:01.001Z Evidence SHA-256 `c87bd245d5b2c2ac6494b650c1804239f742fb49524d0e5f17a1dab415220b31` Source locator `/results/airline` |  |  |  |  |  |  |
| [Judgemark v2.1](https://benchize.com/benchmarks/eq-bench%3Ajudgemark-v2.md) | 81.99% | — | 10 / 39 | — | [eq bench · judgemark-v2](https://eqbench.com/judgemark-v2.js) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `judgemark_score` \| 81.99 \| percent \| overall \| Higher is better \| \| `stability` \| 0.726 \| score_fraction \| overall \| Higher is better \| \| `separability` \| 0.844 \| score_fraction \| overall \| Higher is better \| \| `human_corr` \| 0.818 \| correlation \| overall \| Higher is better \| ### Configuration interactiveAdjustments none protocol judgemark-v2 sourceModelVersion claude-sonnet-4 ### Source [Original source](https://eqbench.com/judgemark-v2.js) Source eq bench Original model label claude-sonnet-4 Source version judgemark-v2 Observed 2026-10-22T05:10:06.632Z Evidence SHA-256 `070cc3fbd6f0333dde2b52bec4329ea3fba7eccbf016bbbbef95e0f8b2197781` Source locator `/leaderboardDataV2/1` |  |  |  |  |  |  |
| [VisualToolBench (VTB)](https://benchize.com/benchmarks/scale%3Avtb.md) | 4.48source score | CI 4.07–4.89 | 5 / 5 | — | [scale](https://labs.scale.com/leaderboard/vtb) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 4.48 \| source_score \| reported \| Higher is better \| ### Configuration Reasoning effort unknown Model configuration claude-sonnet-4 sourceModelVersion Not reported thinkingEnabled Not reported ### Reported statistics Uncertainty Confidence interval: 4.07–4.89 contaminationWarning Not reported rank 17 reportedConfidenceHalfWidth 0.41 ### Source [Original source](https://labs.scale.com/leaderboard/vtb) Source scale Original model label claude-sonnet-4 Observed 2026-10-22T03:30:11.441Z Evidence SHA-256 `93d98857da4966e6fc30066502e67359a9f484ad3cbbcc076cff6cec6ee4d03c` Source locator `/rows/16` |  |  |  |  |  |  |
| [ARC-AGI-1](https://benchize.com/benchmarks/arc-prize%3Av1_Semi_Private.md) | 23.83% | — | 206 / 245 | 0.08usd per task | [arc prize · v1](https://arcprize.org/media/data/leaderboard/v1.json) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 0.2383 \| score_fraction \| v1_Semi_Private \| Higher is better \| \| `cost_per_task` \| 0.0806 \| usd_per_task \| v1_Semi_Private \| Lower is better \| ### Configuration evaluationSplit v1_Semi_Private harness Not reported modelGroup Not reported modelType Base LLM sourceModelVersion claude-sonnet-4-20250514 ### Source [Original source](https://arcprize.org/media/data/leaderboard/v1.json) Source arc prize Original model label Claude Sonnet 4 Source version v1 Observed 2026-10-22T05:20:04.066Z Evidence SHA-256 `2bf68d22dafba37047d07ad833ebc0e8199c9aa2f9eeaed50c75c1b704462836` Source locator `/evaluations/34` |  |  |  |  |  |  |
| [DesignArena Overall frontend coding](https://benchize.com/benchmarks/designarena%3Acode.md) | 1,141elo | SE ±2.77 | 83 / 116 | — | [designarena · public](https://www.designarena.ai/api/leaderboard#board=code) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `elo` \| 1141 \| elo \| code \| Higher is better \| ### Configuration arenaType models category allcategories inputModality text sourceModelVersion claude-sonnet-4 variationName public ### Reported statistics Uncertainty Standard error: ±2.77 averageGenerationTimeMs 72935 battles 17542 losses 8178 standardError 2.7675 winRate 53.4 wins 9364 ### Source [Original source](https://www.designarena.ai/api/leaderboard#board=code) Source designarena Original model label Claude Sonnet 4 Source version public Observed 2026-10-21T16:37:57.301Z Evidence SHA-256 `0c897b58670f5e352bd123db999a40c0d1781a443964c502b90774a76a3a7758` Source locator `/data/121` |  |  |  |  |  |  |
| [SVGBench Arena](https://benchize.com/benchmarks/svgbench%3Aarena.md) | 1,579 | — | 41 / 191 | — | [svgbench](https://svgbench.ai/api/leaderboard?shape=2) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `rating` \| 1579 \| rating \| reported \| Higher is better \| ### Configuration Reasoning effort unknown sourceModelId model_anthropic_claude_sonnet_4 ### Reported statistics coverageCount 27 draws 1 judged 68 losses 28 rank 49 ratingDeviation 28 wins 39 ### Source [Original source](https://svgbench.ai/api/leaderboard?shape=2) Source svgbench Original model label Claude Sonnet 4 Observed 2026-10-22T04:30:04.538Z Evidence SHA-256 `5c0a9c869593820d57d2c05f6d416e1b44a740b4835e66c66e9ad8b8edc59e2c` Source locator `/rows/47` |  |  |  |  |  |  |
| [MASK](https://benchize.com/benchmarks/scale%3Amask.md) | 89.27source score | CI 87.26–91.28 | 4 / 25 | — | [scale](https://labs.scale.com/leaderboard/mask) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `score` \| 89.27 \| source_score \| reported \| Higher is better \| ### Configuration Reasoning effort unknown Model configuration Claude Sonnet 4 sourceModelVersion claude-sonnet-4-20250514 thinkingEnabled Not reported ### Reported statistics Uncertainty Confidence interval: 87.26–91.28 contaminationWarning Not reported rank 9 reportedConfidenceHalfWidth 2.01 ### Source [Original source](https://labs.scale.com/leaderboard/mask) Source scale Original model label Claude Sonnet 4 Observed 2026-10-22T03:30:12.706Z Evidence SHA-256 `fe1699303dff9a092aff10128cfc39d63ddc0680c0d241345fa747c450597b67` Source locator `/rows/8` |  |  |  |  |  |  |
| [DesignArena Browser game generation](https://benchize.com/benchmarks/designarena%3Agame-dev.md) | 1,133elo | — | 75 / 117 | — | [designarena · public](https://www.designarena.ai/api/leaderboard#board=game-dev) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `elo` \| 1133 \| elo \| game-dev \| Higher is better \| ### Configuration arenaType models category gamedev inputModality Not reported sourceModelVersion claude-sonnet-4 variationName public ### Reported statistics averageGenerationTimeMs 79031 battles 1689 losses 754 standardError Not reported winRate 55.4 wins 935 ### Source [Original source](https://www.designarena.ai/api/leaderboard#board=game-dev) Source designarena Original model label Claude Sonnet 4 Source version public Observed 2026-10-21T16:38:01.100Z Evidence SHA-256 `53db6a89112ce8cdf644568c666d4b8309aece764cfcedb291ffd52ae3934e30` Source locator `/data/113` |  |  |  |  |  |  |
| [DesignArena Data visualization generation](https://benchize.com/benchmarks/designarena%3Adata-viz.md) | 1,112elo | — | 83 / 99 | — | [designarena · public](https://www.designarena.ai/api/leaderboard#board=data-viz) |  |
| ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `elo` \| 1112 \| elo \| data-viz \| Higher is better \| ### Configuration arenaType models category dataviz inputModality Not reported sourceModelVersion claude-sonnet-4 variationName public ### Reported statistics averageGenerationTimeMs 60498 battles 216 losses 101 standardError Not reported winRate 53.2 wins 115 ### Source [Original source](https://www.designarena.ai/api/leaderboard#board=data-viz) Source designarena Original model label Claude Sonnet 4 Source version public Observed 2026-10-21T16:37:58.264Z Evidence SHA-256 `a895f6f1a25fa8eb01f821c2097ddec4f884de7bd22f1b8a22e82e6679b71849` Source locator `/data/121` |  |  |  |  |  |  |

---
Cite as: Claude Sonnet 4 — Benchize, https://benchize.com/models/claude-sonnet-4 (retrieved 2026-10-22)