# CyberBench v1.1 / overall > Source: Benchize (https://benchize.com) — https://benchize.com/benchmarks/benchmark%3Acyber%3Aoverall. When citing or reusing this content, credit "Benchize" and link to https://benchize.com/benchmarks/benchmark%3Acyber%3Aoverall. Can autonomous agents craft PoC inputs that trigger OSS-Fuzz vulnerabilities—and produce source patches that fix them? Canonical URL: https://benchize.com/benchmarks/benchmark%3Acyber%3Aoverall Last updated: Wed, 21 Oct 2026 18:52:20 GMT Reproduction and quotation must name Benchize, link to the canonical HTML URL, and retain individual source citations. Can autonomous agents craft PoC inputs that trigger OSS-Fuzz vulnerabilities—and produce source patches that fix them? Safety & robustness · Models tested: 28 · Score publishers: Vals AI ## Model results | # | Model | Score | ± | Cost | Latency | | | --- | --- | --- | --- | --- | --- | --- | | 1 | [Muse Wonder](https://benchize.com/models/muse-wonder-max.md)Max | 80.36% | SE ±5.13 pp | $3.66 | 824.81source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 80.357 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 3.658792 \| USD \| per_test \| Lower is better \| \| `latency` \| 824.81 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 131072 provider Meta reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±5.13 pp sourcePopulation 47 stderr 5.133 tokenTotals {"cache_read_tokens":732733498,"cache_write_tokens":0,"input_tokens":741790778,"output_tokens":1678020,"reasoning_tokens":1010968} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_cyberbench.DS6UOlu0.json) Source Vals AI Original model label meta/muse-wonder Source version 1.1 Observed 2026-10-21T16:46:15.238Z Evidence SHA-256 `943f593f929f8971a6b746269a2851b5febc2aad4e2f1066895f715033cdc7f1` Source locator `/tasks/overall/meta~1muse-wonder` | | | | | | | | 2 | [GPT-6 Sol](https://benchize.com/models/gpt-6-sol-max.md)Max | 77.98% | SE ±5.11 pp | $1.34 | 734.45source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 77.976 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 1.338572 \| USD \| per_test \| Lower is better \| \| `latency` \| 734.454 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±5.11 pp sourcePopulation 47 stderr 5.11 tokenTotals {"cache_read_tokens":199421634,"cache_write_tokens":5785534,"input_tokens":205333516,"output_tokens":1528208,"reasoning_tokens":603092} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_cyberbench.DS6UOlu0.json) Source Vals AI Original model label openai/gpt-6-sol Source version 1.1 Observed 2026-10-21T16:46:15.238Z Evidence SHA-256 `943f593f929f8971a6b746269a2851b5febc2aad4e2f1066895f715033cdc7f1` Source locator `/tasks/overall/openai~1gpt-6-sol` | | | | | | | | 3 | [Gemini 4 Argon](https://benchize.com/models/gemini-4-argon-high.md)High | 77.86% | SE ±5.3 pp | $5.11 | 1,150.37source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 77.857 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 5.113798 \| USD \| per_test \| Lower is better \| \| `latency` \| 1150.368 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 262144 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±5.3 pp sourcePopulation 47 stderr 5.296 tokenTotals {"cache_read_tokens":439040212,"cache_write_tokens":null,"input_tokens":479358504,"output_tokens":2387515,"reasoning_tokens":2080086} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_cyberbench.DS6UOlu0.json) Source Vals AI Original model label google/gemini-4-argon Source version 1.1 Observed 2026-10-21T16:46:15.238Z Evidence SHA-256 `943f593f929f8971a6b746269a2851b5febc2aad4e2f1066895f715033cdc7f1` Source locator `/tasks/overall/google~1gemini-4-argon` | | | | | | | | 4 | [GPT-5.6 Sol](https://benchize.com/models/gpt-5-6-sol-max.md)Max | 76.31% | SE ±5.18 pp | $3.05 | 646.76source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 76.31 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 3.048302 \| USD \| per_test \| Lower is better \| \| `latency` \| 646.763 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±5.18 pp sourcePopulation 47 stderr 5.177 tokenTotals {"cache_read_tokens":251598292,"cache_write_tokens":5293402,"input_tokens":256943358,"output_tokens":714135,"reasoning_tokens":383090} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_cyberbench.DS6UOlu0.json) Source Vals AI Original model label openai/gpt-5.6-sol Source version 1.1 Observed 2026-10-21T16:46:15.238Z Evidence SHA-256 `943f593f929f8971a6b746269a2851b5febc2aad4e2f1066895f715033cdc7f1` Source locator `/tasks/overall/openai~1gpt-5.6-sol` | | | | | | | | 5 | [GPT-6 Luna](https://benchize.com/models/gpt-6-luna-max.md)Max | 76.25% | SE ±5.29 pp | $0.13 | 949.8source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 76.25 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.133802 \| USD \| per_test \| Lower is better \| \| `latency` \| 949.797 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±5.29 pp sourcePopulation 47 stderr 5.288 tokenTotals {"cache_read_tokens":370663750,"cache_write_tokens":6898760,"input_tokens":377615597,"output_tokens":2701710,"reasoning_tokens":1292614} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_cyberbench.DS6UOlu0.json) Source Vals AI Original model label openai/gpt-6-luna Source version 1.1 Observed 2026-10-21T16:46:15.238Z Evidence SHA-256 `943f593f929f8971a6b746269a2851b5febc2aad4e2f1066895f715033cdc7f1` Source locator `/tasks/overall/openai~1gpt-6-luna` | | | | | | | | 6 | [Claude Haiku 5.5](https://benchize.com/models/claude-haiku-5-5-max.md)Max | 75.48% | SE ±5.21 pp | $1.25 | 1,959.18source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 75.476 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 1.248102 \| USD \| per_test \| Lower is better \| \| `latency` \| 1959.181 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±5.21 pp sourcePopulation 47 stderr 5.205 tokenTotals {"cache_read_tokens":1119059176,"cache_write_tokens":14352266,"input_tokens":1133424162,"output_tokens":7935697,"reasoning_tokens":5942212} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_cyberbench.DS6UOlu0.json) Source Vals AI Original model label anthropic/claude-haiku-5-5 Source version 1.1 Observed 2026-10-21T16:46:15.238Z Evidence SHA-256 `943f593f929f8971a6b746269a2851b5febc2aad4e2f1066895f715033cdc7f1` Source locator `/tasks/overall/anthropic~1claude-haiku-5-5` | | | | | | | | 7 | [MiMo-V2.6-Flash](https://benchize.com/models/mimo-v2-6-flash.md) | 75.36% | SE ±5.42 pp | $0.05 | 1,702source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 75.357 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.046276 \| USD \| per_test \| Lower is better \| \| `latency` \| 1701.995 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider Xiaomi reasoning Not reported Reasoning effort unknown Temperature 1 topP 0.95 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±5.42 pp sourcePopulation 47 stderr 5.417 tokenTotals {"cache_read_tokens":303130496,"cache_write_tokens":null,"input_tokens":310670084,"output_tokens":2926093,"reasoning_tokens":2349308} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_cyberbench.DS6UOlu0.json) Source Vals AI Original model label xiaomi/mimo-v2.6-flash Source version 1.1 Observed 2026-10-21T16:46:15.238Z Evidence SHA-256 `943f593f929f8971a6b746269a2851b5febc2aad4e2f1066895f715033cdc7f1` Source locator `/tasks/overall/xiaomi~1mimo-v2.6-flash` | | | | | | | | 8 | [DeepSeek V4.1 Flash](https://benchize.com/models/deepseek-v4-1-flash-high.md)High | 73.69% | SE ±5.48 pp | $0.07 | 830.11source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 73.69 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.069681 \| USD \| per_test \| Lower is better \| \| `latency` \| 830.108 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±5.48 pp sourcePopulation 47 stderr 5.476 tokenTotals {"cache_read_tokens":390720704,"cache_write_tokens":null,"input_tokens":395046929,"output_tokens":2123954,"reasoning_tokens":1603878} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_cyberbench.DS6UOlu0.json) Source Vals AI Original model label deepseek/deepseek-v4.1-flash Source version 1.1 Observed 2026-10-21T16:46:15.238Z Evidence SHA-256 `943f593f929f8971a6b746269a2851b5febc2aad4e2f1066895f715033cdc7f1` Source locator `/tasks/overall/deepseek~1deepseek-v4.1-flash` | | | | | | | | 9 | [GPT-5.6 Luna](https://benchize.com/models/gpt-5-6-luna-max.md)Max | 73.63% | SE ±5.56 pp | $0.4 | 975.2source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 73.631 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.404245 \| USD \| per_test \| Lower is better \| \| `latency` \| 975.203 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±5.56 pp sourcePopulation 47 stderr 5.564 tokenTotals {"cache_read_tokens":599627434,"cache_write_tokens":7892275,"input_tokens":607574721,"output_tokens":1597682,"reasoning_tokens":662688} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_cyberbench.DS6UOlu0.json) Source Vals AI Original model label openai/gpt-5.6-luna Source version 1.1 Observed 2026-10-21T16:46:15.238Z Evidence SHA-256 `943f593f929f8971a6b746269a2851b5febc2aad4e2f1066895f715033cdc7f1` Source locator `/tasks/overall/openai~1gpt-5.6-luna` | | | | | | | | 10 | [DeepSeek V4 Pro 0813](https://benchize.com/models/deepseek-v4-pro-0813-max.md)Max | 72.86% | SE ±5.5 pp | $0.59 | 1,353.19source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 72.857 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.593028 \| USD \| per_test \| Lower is better \| \| `latency` \| 1353.188 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±5.5 pp sourcePopulation 47 stderr 5.5 tokenTotals {"cache_read_tokens":490223040,"cache_write_tokens":null,"input_tokens":494905048,"output_tokens":1832767,"reasoning_tokens":1144956} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_cyberbench.DS6UOlu0.json) Source Vals AI Original model label deepseek/deepseek-v4-pro-0813 Source version 1.1 Observed 2026-10-21T16:46:15.238Z Evidence SHA-256 `943f593f929f8971a6b746269a2851b5febc2aad4e2f1066895f715033cdc7f1` Source locator `/tasks/overall/deepseek~1deepseek-v4-pro-0813` | | | | | | | | 10 | [MiMo-V2.6-Pro](https://benchize.com/models/mimo-v2-6-pro.md) | 72.86% | SE ±5.5 pp | $0.09 | 1,661.32source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 72.857 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.094131 \| USD \| per_test \| Lower is better \| \| `latency` \| 1661.322 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider Xiaomi reasoning Not reported Reasoning effort unknown Temperature 1 topP 0.95 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±5.5 pp sourcePopulation 47 stderr 5.5 tokenTotals {"cache_read_tokens":212179328,"cache_write_tokens":null,"input_tokens":217915344,"output_tokens":2598046,"reasoning_tokens":2158056} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_cyberbench.DS6UOlu0.json) Source Vals AI Original model label xiaomi/mimo-v2.6-pro Source version 1.1 Observed 2026-10-21T16:46:15.238Z Evidence SHA-256 `943f593f929f8971a6b746269a2851b5febc2aad4e2f1066895f715033cdc7f1` Source locator `/tasks/overall/xiaomi~1mimo-v2.6-pro` | | | | | | | | 12 | [GPT-5.6 Terra](https://benchize.com/models/gpt-5-6-terra-max.md)Max | 72.08% | SE ±5.41 pp | $3.31 | 1,023.93source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 72.084 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 3.308232 \| USD \| per_test \| Lower is better \| \| `latency` \| 1023.932 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±5.41 pp sourcePopulation 47 stderr 5.408 tokenTotals {"cache_read_tokens":494433896,"cache_write_tokens":7792076,"input_tokens":502284611,"output_tokens":1204709,"reasoning_tokens":696584} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_cyberbench.DS6UOlu0.json) Source Vals AI Original model label openai/gpt-5.6-terra Source version 1.1 Observed 2026-10-21T16:46:15.238Z Evidence SHA-256 `943f593f929f8971a6b746269a2851b5febc2aad4e2f1066895f715033cdc7f1` Source locator `/tasks/overall/openai~1gpt-5.6-terra` | | | | | | | | 12 | [DeepSeek V4 Flash 0731](https://benchize.com/models/deepseek-v4-flash-0731-high.md)High | 72.08% | SE ±5.41 pp | $0.19 | 927.82source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 72.084 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.19286 \| USD \| per_test \| Lower is better \| \| `latency` \| 927.815 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 384000 provider DeepSeek reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±5.41 pp sourcePopulation 47 stderr 5.408 tokenTotals {"cache_read_tokens":472399680,"cache_write_tokens":null,"input_tokens":476749147,"output_tokens":2146386,"reasoning_tokens":1595078} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_cyberbench.DS6UOlu0.json) Source Vals AI Original model label deepseek/deepseek-v4-flash-0731 Source version 1.1 Observed 2026-10-21T16:46:15.238Z Evidence SHA-256 `943f593f929f8971a6b746269a2851b5febc2aad4e2f1066895f715033cdc7f1` Source locator `/tasks/overall/deepseek~1deepseek-v4-flash-0731` | | | | | | | | 14 | [Claude Fable 5.1](https://benchize.com/models/claude-fable-5-1-max.md)Max | 70.42% | SE ±5.43 pp | $4.01 | 1,112.1source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 70.416 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 4.010688 \| USD \| per_test \| Lower is better \| \| `latency` \| 1112.095 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±5.43 pp sourcePopulation 47 stderr 5.43 tokenTotals {"cache_read_tokens":256689486,"cache_write_tokens":6229710,"input_tokens":262925117,"output_tokens":2088268,"reasoning_tokens":1387300} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_cyberbench.DS6UOlu0.json) Source Vals AI Original model label anthropic/claude-fable-5-1 Source version 1.1 Observed 2026-10-21T16:46:15.238Z Evidence SHA-256 `943f593f929f8971a6b746269a2851b5febc2aad4e2f1066895f715033cdc7f1` Source locator `/tasks/overall/anthropic~1claude-fable-5-1` | | | | | | | | 15 | [Muse Spark 1.2](https://benchize.com/models/muse-spark-1-2-xhigh.md)Extra high | 69.52% | SE ±5.56 pp | $2.15 | 1,220.58source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 69.524 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 2.149077 \| USD \| per_test \| Lower is better \| \| `latency` \| 1220.583 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 131072 provider Meta reasoning Not reported Reasoning effort xhigh Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±5.56 pp sourcePopulation 47 stderr 5.558 tokenTotals {"cache_read_tokens":626573058,"cache_write_tokens":0,"input_tokens":642905824,"output_tokens":2957384,"reasoning_tokens":1860750} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_cyberbench.DS6UOlu0.json) Source Vals AI Original model label meta/muse_spark_1_2 Source version 1.1 Observed 2026-10-21T16:46:15.238Z Evidence SHA-256 `943f593f929f8971a6b746269a2851b5febc2aad4e2f1066895f715033cdc7f1` Source locator `/tasks/overall/meta~1muse_spark_1_2` | | | | | | | | 16 | [Muse Spark 1.3](https://benchize.com/models/muse-spark-1-3-xhigh.md)Extra high | 69.41% | SE ±5.76 pp | $4.26 | 1,492.54source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 69.405 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 4.258407 \| USD \| per_test \| Lower is better \| \| `latency` \| 1492.536 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 131072 provider Meta reasoning Not reported Reasoning effort xhigh Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±5.76 pp sourcePopulation 47 stderr 5.758 tokenTotals {"cache_read_tokens":1443226114,"cache_write_tokens":0,"input_tokens":1460743688,"output_tokens":3506556,"reasoning_tokens":2251884} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_cyberbench.DS6UOlu0.json) Source Vals AI Original model label meta/muse_spark_1_3 Source version 1.1 Observed 2026-10-21T16:46:15.238Z Evidence SHA-256 `943f593f929f8971a6b746269a2851b5febc2aad4e2f1066895f715033cdc7f1` Source locator `/tasks/overall/meta~1muse_spark_1_3` | | | | | | | | 17 | [Claude Opus 5](https://benchize.com/models/claude-opus-5-max.md)Max | 65.36% | SE ±5.55 pp | $2.87 | 1,165.4source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 65.357 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 2.868267 \| USD \| per_test \| Lower is better \| \| `latency` \| 1165.404 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±5.55 pp sourcePopulation 47 stderr 5.55 tokenTotals {"cache_read_tokens":248088934,"cache_write_tokens":4961952,"input_tokens":253056659,"output_tokens":2019202,"reasoning_tokens":1354102} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_cyberbench.DS6UOlu0.json) Source Vals AI Original model label anthropic/claude-opus-5 Source version 1.1 Observed 2026-10-21T16:46:15.238Z Evidence SHA-256 `943f593f929f8971a6b746269a2851b5febc2aad4e2f1066895f715033cdc7f1` Source locator `/tasks/overall/anthropic~1claude-opus-5` | | | | | | | | 18 | [MiniMax M3](https://benchize.com/models/minimax-m3.md) | 65% | SE ±6.1 pp | $3.41 | 2,456.4source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 65 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 3.408292 \| USD \| per_test \| Lower is better \| \| `latency` \| 2456.402 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 512000 provider MiniMax reasoning Not reported Reasoning effort unknown Temperature 1 topP 0.95 verbosity Not reported ### Reported statistics Uncertainty Standard error: ±6.1 pp sourcePopulation 47 stderr 6.104 tokenTotals {"cache_read_tokens":1494349503,"cache_write_tokens":0,"input_tokens":1519598089,"output_tokens":2352544,"reasoning_tokens":498876} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_cyberbench.DS6UOlu0.json) Source Vals AI Original model label minimax/MiniMax-M3 Source version 1.1 Observed 2026-10-21T16:46:15.238Z Evidence SHA-256 `943f593f929f8971a6b746269a2851b5febc2aad4e2f1066895f715033cdc7f1` Source locator `/tasks/overall/minimax~1MiniMax-M3` | | | | | | | | 19 | [Claude Sonnet 5](https://benchize.com/models/claude-sonnet-5-max.md)Max | 61.91% | SE ±5.74 pp | $1.89 | 1,017.03source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 61.905 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 1.888947 \| USD \| per_test \| Lower is better \| \| `latency` \| 1017.033 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±5.74 pp sourcePopulation 47 stderr 5.742 tokenTotals {"cache_read_tokens":340300626,"cache_write_tokens":6922111,"input_tokens":347229444,"output_tokens":2459614,"reasoning_tokens":1787702} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_cyberbench.DS6UOlu0.json) Source Vals AI Original model label anthropic/claude-sonnet-5 Source version 1.1 Observed 2026-10-21T16:46:15.238Z Evidence SHA-256 `943f593f929f8971a6b746269a2851b5febc2aad4e2f1066895f715033cdc7f1` Source locator `/tasks/overall/anthropic~1claude-sonnet-5` | | | | | | | | 20 | [Claude Sonnet 5.5](https://benchize.com/models/claude-sonnet-5-5-max.md)Max | 59.58% | SE ±5.21 pp | $4.42 | 2,032.63source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 59.584 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 4.417372 \| USD \| per_test \| Lower is better \| \| `latency` \| 2032.625 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±5.21 pp sourcePopulation 47 stderr 5.212 tokenTotals {"cache_read_tokens":829966854,"cache_write_tokens":10626290,"input_tokens":840603874,"output_tokens":4907520,"reasoning_tokens":3557295} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_cyberbench.DS6UOlu0.json) Source Vals AI Original model label anthropic/claude-sonnet-5-5 Source version 1.1 Observed 2026-10-21T16:46:15.238Z Evidence SHA-256 `943f593f929f8971a6b746269a2851b5febc2aad4e2f1066895f715033cdc7f1` Source locator `/tasks/overall/anthropic~1claude-sonnet-5-5` | | | | | | | | 21 | [Gemini 3.5 Flash-Lite](https://benchize.com/models/gemini-3-5-flash-lite-high.md)High | 59.11% | SE ±6.17 pp | $0.43 | 1,075.89source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 59.107 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.425124 \| USD \| per_test \| Lower is better \| \| `latency` \| 1075.887 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±6.17 pp sourcePopulation 47 stderr 6.17 tokenTotals {"cache_read_tokens":254177185,"cache_write_tokens":null,"input_tokens":299786791,"output_tokens":1388894,"reasoning_tokens":1079110} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_cyberbench.DS6UOlu0.json) Source Vals AI Original model label google/gemini-3.5-flash-lite Source version 1.1 Observed 2026-10-21T16:46:15.238Z Evidence SHA-256 `943f593f929f8971a6b746269a2851b5febc2aad4e2f1066895f715033cdc7f1` Source locator `/tasks/overall/google~1gemini-3.5-flash-lite` | | | | | | | | 22 | [Claude Opus 5.5](https://benchize.com/models/claude-opus-5-5-max.md)Max | 55.36% | SE ±5.13 pp | $3.48 | 1,091.13source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 55.357 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 3.476604 \| USD \| per_test \| Lower is better \| \| `latency` \| 1091.126 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort max harness Not reported maxOutputTokens 128000 provider Anthropic reasoning Not reported Reasoning effort max Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±5.13 pp sourcePopulation 47 stderr 5.133 tokenTotals {"cache_read_tokens":265061380,"cache_write_tokens":5933202,"input_tokens":271001170,"output_tokens":2436060,"reasoning_tokens":1656850} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_cyberbench.DS6UOlu0.json) Source Vals AI Original model label anthropic/claude-opus-5-5 Source version 1.1 Observed 2026-10-21T16:46:15.238Z Evidence SHA-256 `943f593f929f8971a6b746269a2851b5febc2aad4e2f1066895f715033cdc7f1` Source locator `/tasks/overall/anthropic~1claude-opus-5-5` | | | | | | | | 23 | [Mercury 2.5](https://benchize.com/models/mercury-2-5-high.md)High | 45.48% | SE ±6.06 pp | $0.12 | 1,139.17source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 45.476 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.12462 \| USD \| per_test \| Lower is better \| \| `latency` \| 1139.168 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Inception reasoning Not reported Reasoning effort high Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±6.06 pp sourcePopulation 47 stderr 6.056 tokenTotals {"cache_read_tokens":138406142,"cache_write_tokens":null,"input_tokens":157635879,"output_tokens":814145,"reasoning_tokens":748273} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_cyberbench.DS6UOlu0.json) Source Vals AI Original model label inception/mercury-2.5 Source version 1.1 Observed 2026-10-21T16:46:15.238Z Evidence SHA-256 `943f593f929f8971a6b746269a2851b5febc2aad4e2f1066895f715033cdc7f1` Source locator `/tasks/overall/inception~1mercury-2.5` | | | | | | | | 24 | [Gemini 3.6 Flash](https://benchize.com/models/gemini-3-6-flash-high.md)High | 45.36% | SE ±3.75 pp | $1.31 | 660.04source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 45.357 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 1.306182 \| USD \| per_test \| Lower is better \| \| `latency` \| 660.036 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±3.75 pp sourcePopulation 47 stderr 3.745 tokenTotals {"cache_read_tokens":243499364,"cache_write_tokens":null,"input_tokens":262805643,"output_tokens":1382568,"reasoning_tokens":850728} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_cyberbench.DS6UOlu0.json) Source Vals AI Original model label google/gemini-3.6-flash Source version 1.1 Observed 2026-10-21T16:46:15.238Z Evidence SHA-256 `943f593f929f8971a6b746269a2851b5febc2aad4e2f1066895f715033cdc7f1` Source locator `/tasks/overall/google~1gemini-3.6-flash` | | | | | | | | 25 | [Gemini 3.8 Flash](https://benchize.com/models/gemini-3-8-flash-high.md)High | 43.75% | SE ±2.21 pp | $1.31 | 588.85source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 43.75 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 1.305189 \| USD \| per_test \| Lower is better \| \| `latency` \| 588.848 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.21 pp sourcePopulation 47 stderr 2.21 tokenTotals {"cache_read_tokens":450352484,"cache_write_tokens":null,"input_tokens":470052820,"output_tokens":1962374,"reasoning_tokens":1730282} ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_cyberbench.DS6UOlu0.json) Source Vals AI Original model label google/gemini-3.8-flash Source version 1.1 Observed 2026-10-21T16:46:15.238Z Evidence SHA-256 `943f593f929f8971a6b746269a2851b5febc2aad4e2f1066895f715033cdc7f1` Source locator `/tasks/overall/google~1gemini-3.8-flash` | | | | | | | | 25 | [Gemini 3.7 Flash](https://benchize.com/models/gemini-3-7-flash-high.md)High | 43.75% | SE ±2.21 pp | $2.15 | 453.76source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 43.75 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 2.146121 \| USD \| per_test \| Lower is better \| \| `latency` \| 453.761 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 65536 provider Google reasoning Not reported Reasoning effort high Temperature 1 topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.21 pp sourcePopulation 47 stderr 2.21 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_cyberbench.DS6UOlu0.json) Source Vals AI Original model label google/gemini-3.7-flash Source version 1.1 Observed 2026-10-21T16:46:15.238Z Evidence SHA-256 `943f593f929f8971a6b746269a2851b5febc2aad4e2f1066895f715033cdc7f1` Source locator `/tasks/overall/google~1gemini-3.7-flash` | | | | | | | | 27 | [GPT-6 Astra](https://benchize.com/models/gpt-6-astra-max.md)Max | 41.07% | SE ±2.56 pp | $1.84 | 250.42source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 41.072 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 1.840737 \| USD \| per_test \| Lower is better \| \| `latency` \| 250.415 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.56 pp sourcePopulation 47 stderr 2.559 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_cyberbench.DS6UOlu0.json) Source Vals AI Original model label openai/gpt-6-astra Source version 1.1 Observed 2026-10-21T16:46:15.238Z Evidence SHA-256 `943f593f929f8971a6b746269a2851b5febc2aad4e2f1066895f715033cdc7f1` Source locator `/tasks/overall/openai~1gpt-6-astra` | | | | | | | | 28 | [GPT-6.1 Sol](https://benchize.com/models/gpt-6-1-sol-max.md)Max | 39.29% | SE ±2.74 pp | $0.29 | 221.99source duration | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `accuracy` \| 39.286 \| percent \| reported \| Higher is better \| \| `cost_per_test` \| 0.285372 \| USD \| per_test \| Lower is better \| \| `latency` \| 221.992 \| source_duration \| reported_duration \| Lower is better \| ### Configuration computeEffort Not reported harness Not reported maxOutputTokens 128000 provider OpenAI reasoning Not reported Reasoning effort max Temperature Not reported topP Not reported verbosity Not reported ### Reported statistics Uncertainty Standard error: ±2.74 pp sourcePopulation 47 stderr 2.742 tokenTotals Not reported ### Source [Original source](https://www.vals.ai/_astro/benchmark_view_cyberbench.DS6UOlu0.json) Source Vals AI Original model label openai/gpt-6.1-sol Source version 1.1 Observed 2026-10-21T16:46:15.238Z Evidence SHA-256 `943f593f929f8971a6b746269a2851b5febc2aad4e2f1066895f715033cdc7f1` Source locator `/tasks/overall/openai~1gpt-6.1-sol` | | | | | | | --- Cite as: CyberBench v1.1 / overall — Benchize, https://benchize.com/benchmarks/benchmark%3Acyber%3Aoverall (retrieved 2026-10-21)